From 602cb33d41088c238ed9102bd44a0510497d3460 Mon Sep 17 00:00:00 2001 From: Jonni Liljamo Date: Sat, 26 Oct 2024 21:09:53 +0300 Subject: [PATCH] docs: update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index b14fc28..09fc91d 100644 --- a/README.md +++ b/README.md @@ -3,16 +3,16 @@ This repository contains my very opinionated (and possibly overengineered) robots.txt generation. -`lists` contains lists of user agents to disallow at root +`lists/` contains lists of user agents to disallow at root: * .txt files, line by line listings of user agents * empty lines are ignored * lines starting with # are ignored -`bases` contains site specific base robots.txt files - * base robots.txt files in the robots.txt format +`bases/` contains site specific base robots.txt files: + * in the robots.txt format * names represent the domains they're served at -`generate.sh` is a bash script for generating output robots.txt files +`generate.sh` is a bash script for generating output robots.txt files: * arg $1 is a required path to the out file * arg $2 is an optional path to a base file -- 2.44.1