Keeping bot detection models honest after deployment
Cloudflare’s Bot Management service uses machine learning models to classify HTTP requests as human, benign bot, or malicious bot. With more than 55 million requests per second flowing through Cloudflare’s network, those models have to perform at a scale where even small degradations can have significant impact. The models are updated regularly to incorporate new threat intelligence, which means every change needs rigorous validation both before and after it reaches production.
Pre-deployment validation is thorough: models are tested across many traffic segments — by browser, HTTP protocol, and other dimensions — to confirm that detections behave as expected before a gradual rollout. But once a model is live, the problem changes. Outcome-based metrics like bot score distributions or challenge solve rates can show that something shifted, but they don’t reveal which traffic segments were affected or whether the shift was anticipated.
Training a model for the internet means training against a moving target. A model trained on static data can achieve excellent results only as long as the input doesn’t change. Building a model that generalizes into the future — with new threats, browsers, and bots — is harder. Monitoring closes that gap by providing a repeatable process to verify that production models continue to generalize against current traffic.
Before machine learning monitoring was in place, the team tracked the aggregate proportion of requests scored as bot or human. That high-level view was useful but didn’t show how the model treated specific traffic types. Digging into behavior for, say, Chrome traffic or iOS clients required manual investigation. With monitoring, the team gets granular insight into model behavior without that extra work, answering the core question: how are bot detection models performing in production?
Monitoring has proven invaluable in several scenarios:
- Investigating bot score anomalies: When customers report false positives or negatives and a broader issue is suspected, monitoring helps engineers pinpoint the cause — either through a global dashboard or by focusing on a specific dataset.
- Monitoring any model prediction or request field: The monitoring service adds an observability layer over any request artifact stored in the web request databases. If a prediction or outcome is logged, it can be monitored, and the team can work with other engineering groups to enable tracking for any result.
- Deploying new models: Rollouts happen gradually, with checks at each stage. Monitoring compares the new model against the previous version across granular traffic segments at every deployment step, providing confidence before moving to the next release phase.
How monitoring works
The system starts with a ground-truth dataset — labeled traffic known to be either human or bot. If the model scores a request as bot traffic but the label says human, that’s a miscategorization. The same labeled data used to train the model is the basis for evaluating it.
Training-time datasets only validate a model for a snapshot in time. To continuously evaluate production performance, the team needs real-time labeled data to compare against bot scores. One source is the heuristics engine, which provides high-confidence labels for certain requests. Customer feedback and attack pattern research also supply reliable labeled data. Model scores are then compared against these freshly labeled datasets to judge ongoing performance.
Consistency matters: while the underlying data changes between sampling windows, the methodology, conditions, and filters must stay identical. The process is automated, generating labeled datasets in real time for an up-to-the-minute view of model performance.
Granular performance metrics
Suppose accuracy drops suddenly on a dataset labeled as bot traffic — meaning bots are being scored as humans. The question becomes which subset of traffic is responsible. Is it a specific Chrome browser version or a particular ASN? Monitoring uses specializations, which are filters applied to a dataset that focus on one dimension of interest, such as browser type or ASN. Each specialization yields an expectation for how traffic should have been scored along with insight into the dimension causing the miss.
The monitoring platform
Monitoring runs on Endeavor, a unified platform for bots-related machine learning that handles model training, validation, interpretability, and delivery of detection data to production servers. Monitoring breaks down into three tasks: rendering monitoring queries to fetch datasets, computing performance metrics, and storing those metrics. The platform runs on Airflow with Kubernetes and GPUs, backed by Postgres and ClickHouse databases.
A monitoring query is a SQL query against the ClickHouse web request database asking how machine learning scoring looks right now. Adding dataset and specialization conditions makes the question more precise: for a set of known automated or human traffic, how does scoring look along specific dimensions?
Training and validation datasets are defined by SQL queries that capture traffic segments, such as requests flagged as bots by heuristics. For monitoring, those queries are adapted to measure performance metrics like accuracy and continuously updated to reflect the latest time range. Each training dataset can thus generate a monitoring query that gives real-time insight into model performance.
Computing and storing metrics
Once a monitoring query is rendered, bot score distributions are fetched from the request database. The MetricsComputer takes those distributions as input and produces performance metrics such as accuracy over a configurable time interval. Any metric can be evaluated by implementing the MetricInterface Python interface, which requires only a compute_metric method defining the calculation.
After each run, metrics are stored by dataset, model version, and specialization value in the ml_performance ClickHouse table. Precomputing metrics allows long retention periods, so performance can be reviewed across model versions or dimensions over time. Because the table also stores the raw score distributions, new metrics can be backfilled as needed.
Running on GPUs
Metrics computation is load balanced across endeavour-worker instances running on GPUs. The airflow-scheduler adds monitoring tasks to a Redis Queue, and Airflow Celery workers on each GPU pull tasks for processing. Keeping a production monitoring service constantly running on GPUs has a side benefit: it acts as a health check for the infrastructure, ensuring GPUs stay current and ready for model training and validation workloads.
What Monitoring Caught That Training Missed
Shortly after the monitoring system went live, it flagged a blind spot: the production model was producing unreliable bot scores for HTTP/3 traffic. At the time, HTTP/3 was still rare enough that the model had never been trained on it. The heuristics engine, a separate detection layer, continued to catch bots on HTTP/3 accurately, so customer protection was never compromised — but the monitoring data made the gap explicit and pointed directly at what the next training round needed to address.
The fix was measurable. The subsequent model iteration distinguished bot from human HTTP/3 requests with over 3.5x higher accuracy than its predecessor. As more specialized datasets are enabled, the same kind of monitoring can expose weak spots along other dimensions — specific browsers, operating systems, or other traffic slices — before they become customer-facing problems.
Catching a Bad Deployment Before It Spread
Running machine learning in production across data centers in more than 100 countries means deployments rarely go perfectly. A couple of years ago, an update to the bot detection models caused exactly that: a rise in false positives, with legitimate traffic on residential ASNs being flagged as automated.
The failure mode was visible in the monitoring dashboards almost immediately. The performance drop followed the staged rollout pattern precisely — software updates begin at tier 3 colocation centers and graduate upward to tier 1 — and the release was just reaching the largest data centers. The dashboards made the cause unambiguous, and the update was reverted before it could spread further. Normal model performance was restored quickly. The episode is a textbook case of why monitoring matters: it turned a potentially global incident into a localized one by surfacing the anomaly, pointing at the root cause, and enabling a fast rollback.
From Internal Tool to Product Feature
The same monitoring and deployment discipline that protects bot detection models is now being generalized for external use. Over the next few months, AI Gateway — the proxy for observing and controlling AI applications — will gain more advanced features built on exactly these strategies. The goal is a single control plane where anyone can deploy, test, monitor, and iterate on their own AI or machine learning models the same way Cloudflare does internally.
What's Next for Model Monitoring
Today's monitoring is already catching performance issues during rollouts and guiding model iteration. The roadmap adds alerting based on model performance, so the system will proactively notify engineers if drift pushes metrics outside expected bounds — no manual dashboard checks required.
The compute side is also changing. With GPUs now deployed in 100+ cities alongside the Workers AI launch, inference runs closer to users, which improves both latency and the ability to experiment with more powerful, cutting-edge bot detection models. Faster iteration cycles make the monitoring and alerting pipeline more critical than ever — it's the safety net that lets new models ship quickly without risking production traffic.
