From blind spots to positive security models
Cloudflare has rolled out automatic API endpoint discovery and schema learning for all API Gateway customers. The two features work together so that teams can enforce a positive security model on their APIs even when they have little or no existing inventory of those APIs.
Knowing your API hostnames and endpoints is the necessary first step in securing them. In practice, many organizations start that process by asking developers to list endpoints in a spreadsheet. That approach produces a point-in-time inventory that goes stale with the next code release, depends on tribal knowledge that walks out the door when people leave, and is vulnerable to human error. Even with an accurate inventory, building a schema for each endpoint to validate that the API is being used as intended requires yet more collective knowledge.
API Gateway's new API Discovery and Schema Learning features are designed to remove that manual burden by automatically protecting APIs across Cloudflare's global network.
Discovery without session identifiers
API Discovery previously relied on customer-specific session identifiers, such as HTTP headers or cookies, to identify API endpoints and show analytics. That approach worked but had three drawbacks:
- Customers had to know which header or cookie delineated their sessions, and finding the right token could take time.
- Session-based discovery could not monitor or report on completely unauthenticated APIs, even though customers wanted visibility into session-less traffic to document all endpoints and police abuse.
- After inputting the session identifier, customers waited up to 24 hours for discovery to complete.
The session-based product delivered value quickly, but as traffic and labeled data accumulated, Cloudflare saw an opportunity to train a machine learning model using existing API metadata and the new labeled data. That model would eliminate the need for a session identifier to pinpoint API endpoints. The result is Machine Learning-based API Discovery, which continually finds all API traffic routed through Cloudflare's network with no prerequisite input from customers. API Gateway customers can now start with Discovery immediately and can uncover unauthenticated APIs that were previously invisible to them.
Session identifiers remain important to API Gateway for other purposes: they are the basis of volumetric abuse prevention rate limits and Sequence Analytics.
Learning schemas from discovered endpoints
Once APIs are discovered, the question is how to protect them. Developers can programmatically generate an API schema file that codifies acceptable input, then upload it to API Gateway's Schema Validation. But many customers cannot find their APIs as fast as developers build them, and security teams often see only the HTTP request method and path in their logs — not enough detail to build unique OpenAPI schemas for potentially hundreds of endpoints.
Usage patterns showed customers discovering APIs but rarely enforcing a schema. The reason was consistently a lack of time and expertise: tracking down the owner of each API to request a schema was hard to prioritize against other security work. To close that gap, Cloudflare applied the same learning process used for endpoint discovery to the discovered endpoints themselves. The result is Schema Learning, which generates an OpenAPI formatted schema for every discovered endpoint in real time. Customers can upload that schema into Schema Validation to enforce a positive security model.
Why pattern matching isn’t enough
REST APIs tend to follow predictable path conventions — a version prefix, a service name, identifiers, and feature names. The Cloudflare API, for instance, consistently begins with /client/v4. A naive heuristic like “path starts with /client” might catch that specific case, but defining what makes a request an API call in general is far less straightforward.
Consider two near-identical paths: /users/7f577081-7003-451e-9abe-eb2e8a0f103d.jpg and /users/7f577081-7003-451e-9abe-eb2e8a0f103d. The first looks like a static thumbnail resource; the second offers little clue from the path alone. Hand-crafting rules that separate these cases across the scale of traffic Cloudflare processes daily quickly becomes unmanageable.
Cloudflare’s new Discovery feature instead uses a machine-learned classifier to score HTTP transactions. The model was trained on labeled API and non-API traffic samples, with care taken to avoid overfitting so it generalizes beyond training data. It builds on the CatBoost library, the same technology behind Cloudflare’s Bot Management ML models, and can be understood as a decision flow that checks successive conditions — such as whether the path contains “api” and then whether a file extension is present — ultimately producing a likelihood score.
Feature extraction and scoring run in Rust and take only microseconds per request. Since Discovery draws from Cloudflare’s data pipeline rather than live traffic, only transactions that will be sampled are scored, keeping CPU overhead low. Classification results feed into the existing API Discovery mechanism used for session identifier-based discovery, which simplifies comparing both approaches.
Once raw paths are collected, Discovery normalizes them into variable patterns before presenting results. This is nontrivial because identifier schemes vary widely across the network, from straightforward GUIDs to custom formats. The system uses variable classifiers and supervised learning to handle the normalization.
Comparing discovery methods
ML Discovery was expected to find a similar set of endpoints to the earlier session identifier-based method, with two known gaps. Session-based discovery requires an identifier header or cookie, so it misses endpoints outside of sessions — logins or unauthenticated calls, for example. It also surfaces non-API traffic when customers couldn’t cleanly segment their API traffic.
At a high level, the distribution of endpoints per domain looks similar across both methods, confirming ML Discovery behaves as intended. A domain-by-domain view, however, shows differences.
For roughly 46% of domains there is no change in the number of discovered endpoints. About 15% of domains see an increase of 1 to 50 endpoints, while 9% see a comparable drop. More notably, about 28% of domains gain over 50 additional endpoints — ones that previously went undetected because they weren’t tied to a session identifier.
Learning API schemas from traffic
Discovering endpoints is only half the job. Protecting them requires knowing what legitimate requests look like. API Gateway’s Schema Validation enforces expected formats for body, path, and query parameters to mitigate OWASP Top 10 API attacks — but that assumes you already have a schema.
Schema Learning reconstructs that schema automatically by examining successful requests to an endpoint. Clients that interact with an API successfully are effectively conforming to its implicit schema; Schema Learning infers the rules from their behavior. For example, if an API expects user IDs in the format id12345-a, client requests will consistently reflect that, even if the constraint was never documented.
The process identifies recent successful requests, parses each input parameter by position and type, and then looks for common characteristics across all values. It applies a statistical test to the value distribution and only generates a schema when it is confident those constraints are bounded, preventing false positives that would block valid traffic.
This matters in practice. A manually written schema might assume a timestamp parameter is always a Unix integer, while the API actually accepts ISO 8601 strings too. A rule forcing an integer would produce false positives on valid requests. Schema Learning catches such variability automatically.
What the learned parameters look like
Across observed traffic, slightly more than half of learned parameters are strings, followed by integers at nearly a third. Arrays, booleans, and floats make up most of the remaining 17%, while object parameters are uncommon in path and query positions.
Paths typically carry very few parameters — 94% of endpoints have at most one.
Query strings are much denser, with some endpoints exposing up to 50 parameters. Parameter learning estimates numeric constraints such as value ranges, lengths, or allowable unique values with 99.9% confidence for the majority of parameters it observes.
Getting started
API Gateway customers can begin using Discovery immediately from the Cloudflare dashboard. Discovered endpoints appear under the Discovery tab without any setup, and can be added to Endpoint Management. Schema Learning runs automatically on those endpoints, and after 24 hours a learned schema can be exported and uploaded into Schema Validation. Enterprise customers without an API Gateway subscription can start through the dashboard trial or by contacting their account team.
Roadmap
Planned enhancements include support for additional parameter formats — POST body parameters in JSON and URL-encoded form, plus header and cookie schemas — and proactive notifications when a learned schema changes so a refreshed version can be applied. Cloudflare is also soliciting feedback from users to prioritize further improvements.
