A second pair of eyes on AI traffic

An AI bill that has quietly doubled is a clue, not an answer. To know whether something is wrong, you need a baseline of what's normal and an identity attached to each request so a spike isn't anonymous. Cloudflare is addressing both with two updates to its AI Gateway: an identity-aware gateway protected by Cloudflare Access, and a new User Insights view that scores sessions against each account's own history.

The releases target a problem Stanford's AI Index report highlighted: 59% of organizations said knowledge gaps were their biggest obstacle to responsible AI governance. When usage is spread across shared keys and multiple tools, nobody can say who did what — which is as much a security issue as a cost one.

Putting a name on every request

AI Gateway sits between your applications and model providers like OpenAI, Anthropic, and Google, giving you a single place to observe and control AI usage. It works for custom apps and for coding agents like Claude Code, Codex, and GitHub Copilot, so all traffic falls under the same visibility.

The new integration with Cloudflare Access (now in open beta) lets you put a custom domain in front of the gateway and authenticate requests with any SAML-supported identity provider, such as Okta or Entra. That removes the need to generate and circulate Cloudflare API keys. Administrators can set policies on who can reach the gateway, and requests go to a clean hostname like ai.example.com without account or gateway IDs in the URL.

Once authenticated, Cloudflare adds the verified Access user ID to request metadata as cf.user_id, so logs, analytics, and spend can be filtered by the actual person. That identity also becomes a budgeting tool: combined with spend limits, you can now assign each user their own budget bucket and block further requests or fall back to a cheaper model once the limit is hit.

Flexport, an early adopter, saw the value immediately. According to Staff Security Engineer Max Baumgarten, shared API keys made it nearly impossible to tell who was using a service or to apply existing employee access rules. Placing Cloudflare Access in front of AI Gateway gave each request an authenticated identity and let Flexport reuse its identity policies at the gateway.

Learning what 'normal' looks like

The second piece, User Insights, is generally available to all AI Gateway customers at no extra cost. It turns the traffic already flowing through the gateway into a behavioral baseline for every account — person or agent. It tracks cost and waste like low cache-hit rates and oversized context windows, but its focus is anomaly detection: flagging accounts that break their own pattern.

The approach scores sessions, not single requests. Absolute thresholds miss the nuance. A $500 jump from a heavy user might be routine, while a $50 session from an agent that usually spends $5 is a 10x deviation. So each session is compared against the account's own history, using the p95 session cost over the last 30 days. A session above 2x of that p95 is a candidate for anomalous behavior.

The system also applies an account-wide ceiling at the p99 session cost, so an anomaly must be both a relative spike for the user and among the most expensive 1% of sessions across the organization. Two thresholds are used for the categories:

  • High relative spike and above the account p99 — triggers an alert.
  • High relative spike but below the absolute ceiling — ignored to avoid alerting on small-dollar shifts.
  • High absolute spend but consistent with the user's history — treated as routine behavior.
  • Normal activity within both baselines — filtered out.

Baselines are not static. The rolling p95 and the 2x threshold move as an account's habits change, so alerts always reflect recent behavior. A dollar floor is applied so that a spike must be both statistically unusual and worth an administrator's time; a micro-user's 500x blip over a few cents never fires an alert.

Spotting misuse as it happens

The view administrators get is a filtered feed of accounts that broke their own pattern, with everything normal removed. This catches behavior that policy rules miss: a trusted account doing more of what it's already allowed to do. It might be a service account suddenly running more expensive sessions, or a person whose usage jumps well past their own norm and stays there for days.

None of that trips a policy, but all of it breaks a behavioral baseline — often the first observable sign of a compromised credential or an agent going off the rails. User Insights does not judge intent or block anyone. It surfaces the handful of accounts behaving strangely so someone can investigate. That might lead to a real security incident, or simply to coaching a developer who dumps an entire codebase into every prompt.

What's on the roadmap

Two features are in active development. Task-based smart routing would analyze an incoming request and route it to the model that offers the best result at the lowest cost, rather than sending everything to a frontier model. At the organizational level, it would show where the most savings could be captured by shifting to more efficient models.

Prompt classification is the other focus. It would sort requests into categories like coding, writing, and others, adding context that anomaly detection lacks. A spend spike in coding from an engineer might be acceptable; the same spike in a category that account has never touched is not. Classification would also help separate business traffic from personal use — a task that currently looks indistinguishable from someone quietly moving data out through a model.

Getting started

User Insights is available today in the AI Gateway dashboard for any customer sending traffic through the gateway. Anomaly and spend views work without identity attached, but Cloudflare recommends placing the gateway behind Cloudflare Access — now in open beta — so an anonymous account ID becomes a name an administrator can act on. Start in monitoring mode to learn baselines before enforcing limits.