Who runs it
Common Crawl. It names itself with the token CCBot.
User agent, as Common Crawl documents it
CCBot/2.0 (https://commoncrawl.org/faq/)What it does
It fetches pages with HTTP GET requests after checking robots.txt, to build Common Crawl's open archive of the web.
Renders JavaScript
No. Common Crawl says JavaScript is not executed and cookies are not used.
How to verify it
Run a reverse DNS lookup on the visiting address. The name should be under crawl.commoncrawl.org, and a forward lookup should return the same address. Reverse DNS isn't yet supported over IPv6, so use the published list for those addresses.
robots.txt
Rules under User-agent: CCBot apply to it. Common Crawl says CCBot checks robots.txt first, obeys Crawl-delay and honors nofollow, and it keeps an opt-out registry.
Questions
Does CCBot run my JavaScript?
No. Common Crawl says JavaScript is not executed.
Last checked against Common Crawl's documentation: 31 Oct 2026Verified visits recorded here: shown once the site is live