Heuristics on the Ruleset Engine

Cloudflare’s bot detection relies on a layer of static rules, called heuristics, beneath every machine learning model. These heuristics serve as ground truth for bot identification, as labeled training data for new models, and as a benchmark for validating model accuracy. Historically, these rules were written in Lua within Cloudflare’s reverse proxy, but that engine limited rule complexity and slowed deployment of new detections.

Cloudflare has now migrated its bot heuristics to the Cloudflare Ruleset Engine, the same framework that powers WAF, Load Balancing, and Transform rules. The move gives Bot Management customers new visibility into why requests are flagged, and lets Cloudflare write more precise rules that reduce false positives.

Why the old engine fell short

The previous Lua-based heuristics engine was bundled with code that also ran ML models and other critical logic. Every change went through the same review and release pipeline, so deploying a new detection during an attack could be delayed by unrelated pending changes. More importantly, the engine only supported rules built from a small set of request attributes. As bots grew more sophisticated, Cloudflare found itself rejecting new heuristic candidates because it couldn't write precise enough rules.

One example: Cloudflare found a Golang TLS fingerprint that was commonly used by bots, but also by a small number of corporate VPNs. Under the old platform, blocking the bots would also block the legitimate VPN traffic, because the engine couldn't express a rule that included enough context to separate the two.

More expressive rules, fewer false positives

The Ruleset Engine uses a Wireshark-inspired syntax that allows arbitrary sub-conditions in YAML. This lets Cloudflare's engineers and analysts write heuristics that combine request attributes like user agent, ASN, or JA4 fingerprint in a single rule. New detections inherit the same rigorous testing and rapid release process used for WAF rules, which is critical for "under attack" scenarios.

The Golang TLS fingerprint heuristic was one of the first migrated to the new engine. Before migration, it had a false positive rate of 0.01% — meaning 100 real users per million bots blocked saw an unnecessary challenge page. After analyzing the traffic, Cloudflare found that the vast majority of attack traffic came from a small number of abusive networks. By narrowing the rule to flag the Golang fingerprint only when sourced by those networks, the false positive rate dropped to 0.0001% (one out of 1 million), while still blocking millions of bots weekly and providing ample training data for ML models.

Bot Detection IDs

For Enterprise Bot Management customers, the migration brings a new field: Bot Detection IDs. Every heuristic now has a unique ID, visible in analytics, logs, and firewall events. Customers can use these IDs to write precise rules for individual bots in Custom Rules, Rate Limiting Rules, Transform Rules, and Workers.

Detections also carry a tag describing the heuristic class, plotted over time in analytics. An example Firewall Event entry now shows not just a bot score of 1, but the reason — such as an empty user-agent — giving customers real visibility into why a request was flagged.

Account takeover detections

Cloudflare has also added two detection IDs focused on account takeover (ATO). These complement the Leaked Credentials Check launched during Birthday Week, and they monitor login activity on the zone:

  • 201326592 — flags an anomalous rise in login failures, typically associated with brute force attacks.
  • 201326593 — flags an anomalous rise in login attempts, typically associated with credential stuffing.

These two detections do not currently affect the bot score, but will begin to later in 2025. Customers can already use them to identify ATO attempts in Security Analytics and create rules to block suspicious traffic.