## Tranco ranks 1-1,000: 517 sites returned a text robots.txt
GPTBot: blocked at / 110 (21.3%), named 128
OAI-SearchBot: blocked at / 63 (12.2%), named 85
ChatGPT-User: blocked at / 74 (14.3%), named 95
ClaudeBot: blocked at / 113 (21.9%), named 127
Claude-SearchBot: blocked at / 71 (13.7%), named 67
Claude-User: blocked at / 74 (14.3%), named 67
PerplexityBot: blocked at / 89 (17.2%), named 116
Google-Extended: blocked at / 97 (18.8%), named 111
CCBot: blocked at / 121 (23.4%), named 112
Googlebot: blocked at / 19 (3.7%), named 102
Bingbot: blocked at / 19 (3.7%), named 67
GPTBot blocked AND OAI-SearchBot allowed: 47 of 110
ClaudeBot blocked AND Claude-SearchBot allowed: 42 of 113
Any of OAI-SearchBot/Claude-SearchBot/PerplexityBot blocked: 93
robots.txt returned 403 to our fetcher: 62
## Tranco ranks 200,001-201,000: 587 sites returned a text robots.txt
GPTBot: blocked at / 102 (17.4%), named 114
OAI-SearchBot: blocked at / 28 (4.8%), named 36
ChatGPT-User: blocked at / 34 (5.8%), named 39
ClaudeBot: blocked at / 101 (17.2%), named 108
Claude-SearchBot: blocked at / 22 (3.7%), named 21
Claude-User: blocked at / 21 (3.6%), named 17
PerplexityBot: blocked at / 34 (5.8%), named 43
Google-Extended: blocked at / 87 (14.8%), named 93
CCBot: blocked at / 104 (17.7%), named 105
Googlebot: blocked at / 11 (1.9%), named 45
Bingbot: blocked at / 13 (2.2%), named 23
GPTBot blocked AND OAI-SearchBot allowed: 74 of 102
ClaudeBot blocked AND Claude-SearchBot allowed: 80 of 101
Any of OAI-SearchBot/Claude-SearchBot/PerplexityBot blocked: 35
robots.txt returned 403 to our fetcher: 58