某爬虫自称 ClaudeBot，七天内二十次索要 .env 文件

In the last seven days, something asked this site for its .env file twenty times. It said it was ClaudeBot. An .env file is where a web application keeps its secrets: database passwords, API keys, cloud credentials. It is not meant to be reachable over HTTP at all. On this site it is not: the path returns a 404, as it should. No training crawler wants that file. There is nothing in it to train on. A crawler that asks for it is not reading your site; it is checking whether you left your keys in the door. What the list actually looked like This is the panel that shows which pages AI crawlers read, for the seven days ending September 18. I have left it in the order the dashboard shows it. Path (requests) Top crawler / (27) Bytespider /.env (20) ClaudeBot /tag/news/ (12) Bytespider /pickup-info/mypal-wifi/ (9) ChatGPT-User /tag/ココナッツ/ (7) meta-externalagent /tag/フィリピンイーグル/ (7) meta-externalagent /read-document (6) PerplexityBot Most of that is a small Japanese site being read the way you would expect: the front page, a few tag pages, an article about pocket wifi. Two rows are not like the others. Neither /.env nor /read-document exists here. Both return 404. My own instrument could not tell me The obvious question is whether those twenty requests really came from Anthropic. My dashboard has a Verified column for exactly this, and over those seven days ClaudeBot came out at 68% verified — meaning about a third of its requests could not be confirmed as Anthropic. Which does not answer the question. The column reports a percentage per crawler, not per path. I cannot ask it whether the /.env requests were among the verified ones or the unverified ones. That is the same defect I described two weeks ago and have still not fixed. 68% Share of ClaudeBot requests confirmed as coming from Anthropic that week. Which tells you nothing about the twenty that asked for the .env file. Per-path verification is the thing my dashboard is missing. I did not have to guess On August 31, the security firm GreyNoise published what it had been watching. Between July 28 and August 23 it counted 824 addresses running scans that forged AI crawler names — six crawler names belonging to four AI companies, with two Amazon crawler names appearing in greater volume still. What they asked for: /.env, /.env.production, /.env.bak, /.aws/credentials. The detail that stopped me is not the volume. It is the tell they found. 0 Requests for robots.txt in that entire scanning campaign. The real ClaudeBot asks for robots.txt more than anything else — it is 12% of its traffic. The impostors never asked once. That is a beautifully simple signature. A crawler that respects your site checks the rules first; it is the most boring thing it does, and it does it constantly. Something that wants your credentials has no reason to look at a file telling it where it may not go. Why the crawler name was the target These names were not chosen at random. Site owners have spent the past two years being told to add AI crawlers to their allowlists — to let GPTBot and ClaudeBot and PerplexityBot through so their content gets read, or to block them deliberately if they would rather not be. Either way, the name became a thing that firewalls and plugins make decisions about. A user agent is a header the client writes about itself. It is a claim, not an identity. I demonstrated this on my own site two weeks ago by asking it for a page while calling myself ClaudeBot, and watching my dashboard record the visit as ClaudeBot. That took one line of curl. The scanners are doing the same thing, at scale, against the specific names that people have been trained to wave through. An allowlist keyed on a name that anybody can type is not a control. It is a convention being mistaken for one. What to actually do Do not serve secrets over HTTP, ever. This is the one that matters. If .env is unreachable, the scan is a wasted 404 and nothing else. Everything below is secondary to this. Check addresses, not names. OpenAI, Anthropic, Perplexity, Common Crawl and Google all publish IP ranges or reverse-DNS schemes now. If a request claims a vendor that publishes one and does not match it, the name is worth nothing. Watch for what is missing. A client that never fetches robots.txt but does fetch config paths is not a crawler, whatever it calls itself. That signature costs nothing to look for. Treat crawler traffic as traffic. The habit of filing "AI crawler" away as a separate, benign category is precisely what this campaign is built to exploit. What this instrument cannot see Verification is per crawler, not per path. The central question of this article — were those twenty requests Anthropic or not — is one my own dashboard cannot answer. I am reporting the limit rather than writing around it. One site, seven days. Twenty requests is a tiny sample. It is consistent with a campaign that GreyNoise measured at 824 addresses; it does not independently prove anything. I have not attributed anything. I do not know who sent these, and nothing here should be read as a claim about Anthropic, whose name was used without its involvement. Cached pages are invisible. When a page cache answers, PHP never runs and nothing is recorded. Every count here is a lower bound. How to measure your own site The instrument is a WordPress plugin called AILYS Lens, and it is free. I build it, which you should factor in. Every calculation runs on your own server, nothing is transmitted anywhere, and human visitors are never recorded. The page list described here is on the main dashboard. If a path you have never heard of appears on it, that is worth ten seconds of your attention. A note on AILYS Doctor AILYS Lens is free permanently. That is a design commitment rather than a pricing stage — every feature runs locally on your server, so there is nothing for a paid tier to unlock. The diagnostic service alongside it, AILYS Doctor, is a different thing. Lens tells you what happened; Doctor tells you why, and what to change. It is free to use during its data-collection period, and any future pricing will be announced on the site. See a sample diagnosis · AILYS Doctor Illustration generated with AI and selected by the author.