diff options
| author | Danny Holman <dholman@gymli.org> | 2022-10-19 09:51:45 -0500 |
|---|---|---|
| committer | Danny Holman <dholman@gymli.org> | 2022-10-19 09:51:45 -0500 |
| commit | 45df3836538f2298517d651a771e778b468c7349 (patch) | |
| tree | c344556cccf7d89e77eb006b1309a1ad27b43746 | |
| parent | f6a51f9c5e9000da139ba78bd8c1b8333c247a52 (diff) | |
Add robots.txt file
Add a robots.txt file for search engine crawlers
| -rw-r--r-- | robots.txt | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..93d096d --- /dev/null +++ b/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Crawl-delay: 10 |