prerender[ OBSERVATORY ]
[ Machine ] [ AI training ]

ClaudeBot

Anthropic's training crawler. It collects web content that could contribute to training, and blocking it excludes a site's future materials from Anthropic's training datasets.

OperatorAnthropic
What it's forTraining data
Renders JavaScriptNot yet observed
Verify byIP list

Who runs it

Anthropic. It names itself with the token ClaudeBot. Anthropic doesn't publish a full user agent string for it.

Token, as Anthropic documents it
ClaudeBot

What it does

It collects web content that could potentially contribute to model training. Anthropic runs two other machines, Claude-SearchBot and Claude-User, with their own tokens.

Renders JavaScript

Not yet observed. Anthropic doesn't say whether ClaudeBot runs scripts, and this site hasn't recorded a verified visit yet.

How to verify it

Match the visiting address against Anthropic's published list. It is one list for all three Anthropic machines. Anthropic advises against blocking by IP address, which may not work persistently as an opt-out.

bots.json, Anthropic's IP list

robots.txt

Rules under User-agent: ClaudeBot apply to it. Anthropic says its machines honor industry standard robots.txt directives, and it supports the non-standard Crawl-delay. Put the rule at the top level of every subdomain.

Questions

Do Anthropic's three machines share one robots.txt rule?

No. ClaudeBot, Claude-SearchBot and Claude-User each have their own token.

Last checked against Anthropic's documentation: 31 Oct 2026Verified visits recorded here: shown once the site is live