Download websites that use Cloudflare Bot Management

About Cloudflare Bot Management

A control designed to be inconspicuous is a strange thing to try to count. Requests are scored at the edge before they ever reach the origin server, so what comes back in an ordinary response is indirect: a scoring cookie, a challenge path, a header added on the way out. Recording Cloudflare Bot Management means reading those leftovers rather than reading a tag somebody pasted into a template.

Presence therefore carries an unusual amount of commercial information for a security control. Bot defence of this class is not a free tier feature and not something a developer enables while nobody is looking. Somebody signed a contract, and in most organisations the same conversation covered a firewall, rate limiting and a support agreement. A domain that shows the product is a domain with a security budget, an owner who has been scraped or credential stuffed at least once, and a procurement process reaching beyond the hosting invoice.

As of September 2026 the dataset carries 8,901,484 sites with the product in place, 7,010,958 of them in United States, and .com is heavily represented across the domains. Concentration in one country and one suffix says less about where automated traffic is a problem than about where Cloudflare Inc. sells and where large commercial properties register. Ticketing, travel, retail and financial services cluster in the same places for the same reason.

The product has been on sale since 2019 and it is Proprietary, which matters more here than the license field usually does. There is no self hosted equivalent to install and no open fork to adopt, so a detection is a statement about a supplier relationship rather than about an engineering choice a team could reproduce internally. Sites lose the flag when they leave the vendor, which makes the column a workable proxy for enterprise account movement.

Detection has a floor worth stating before anyone builds a forecast on the number. A well tuned deployment that never challenges ordinary browser traffic can be almost silent, and our crawler is a polite client that is often waved through. We record the product where signals specific to it appear in the response we receive, and we do not infer it from the mere fact that a site sits behind Cloudflare Inc., because the ordinary delivery network and the bot product are separate purchases. The consequence is undercount rather than false positives. Sites on the list are sites where the evidence was actually present.

Two audiences use the population in opposite directions. Security vendors read it as a competitive map, since a domain already paying for commercial bot defence has conceded that the problem is real and has a renewal date attached. Data, pricing and integration teams read it as a constraint map, because a property that scores automated traffic is a property where partner access, price monitoring and any form of collection need to be arranged rather than assumed. Compliance and procurement teams use it a third way, checking which suppliers in a portfolio run controls comparable to their own.

A lookup service answers one hostname and returns a yes or a no. The questions that justify a dataset are shaped differently: which segment of an industry has adopted edge bot defence, how far the share moved over a year, which hosting bases and registrars are overrepresented, and what else runs on the sites that adopted it. The download answers those in one pass, with country, suffix, Security peers and the rest of each site's detected stack sitting on the same row as the flag itself.