diff options
| author | Alyssa Ross <hi@alyssa.is> | 2023-11-05 09:27:05 +0100 |
|---|---|---|
| committer | Alyssa Ross <hi@alyssa.is> | 2023-11-05 09:28:07 +0100 |
| commit | 05f40ff2bfe9c68198664c38d65816f677ac7ed4 (patch) | |
| tree | c971a33fde9ce595bb5924e3c443cc8a23657be1 /modules | |
| parent | 1473201e3385250844b04338932072c7c79f291c (diff) | |
| download | nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar.gz nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar.bz2 nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar.lz nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar.xz nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.tar.zst nixlib-05f40ff2bfe9c68198664c38d65816f677ac7ed4.zip | |
modules/spectrum/nginx: add robots.txt Crawl-delay
Really this should be done for git.qyliss.net too, but that's harder because currently it's just using cgit's built-in one, and I'm not going to let perfect be the enemy of good.
Diffstat (limited to 'modules')
| -rw-r--r-- | modules/server/spectrum/nginx/robots.txt | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/modules/server/spectrum/nginx/robots.txt b/modules/server/spectrum/nginx/robots.txt index 6fba45644c62..b2d1429a929b 100644 --- a/modules/server/spectrum/nginx/robots.txt +++ b/modules/server/spectrum/nginx/robots.txt @@ -2,3 +2,4 @@ User-agent: * # Wildcards are not supported in the robots.txt spec, # but let's hope they work anyway. Disallow: /git/*/snapshot/ +Crawl-delay: 5 |