Securing LLM Applications at the Edge: Cloudflare's Firewall for AI

LLM-powered assistants trained on proprietary documentation can be a double-edged sword. While they streamline customer support and improve user experience, they also introduce a significant security risk: if the training data contains sensitive employee details or internal discussions, attackers may manipulate the assistant into leaking that information. Cloudflare aims to address this with Firewall for AI, an inline security solution announced as an open beta during Security Week 2025.

Discovery: Identifying LLM-Powered Endpoints

The first challenge is simply knowing where your LLM applications are. Cloudflare's approach, built on its reverse-proxy position, begins with detection. Merely searching for known API paths like /chat/completions used by major providers yields few results across global traffic. To broaden the net, Cloudflare uses two key heuristics:

  • Response Headers: Matching on text/event-stream content types for server-sent events, and later application/json for most other exchanges.
  • Origin Response Time: When the proxy identifies potential LLM responses, it plots the time for the origin server to respond. This data shows a clear distinction: applications serving LLM endpoints mostly require over one second to respond, while the majority of other traffic responds in under a second.

Response size proves less reliable, as LLM output intentionally overlaps with regular traffic. However, calculating an effective bitrate—dividing response size by duration—reveals a strong signal: approximately 80% of LLM endpoints operate slower than 4 KB/s.

Applying this bitrate heuristic globally flags roughly 3% of all origin responses. To filter out known false positives—such as GraphQL endpoints, health checks, and generators for QR codes or invoices—Cloudflare introduced a second filter based on variance of response size over time. Template-based generators show low variance, whereas LLM conversations are far more varied. The combination yields approximately 30,000 endpoints now labelled cf-llm in API Shield and Web assets, giving customers a starting point to review and protect their AI traffic.

Detection: Finding PII in Prompts

Once endpoints are discovered, the next step is analyzing incoming prompts for threats. A common method uses regular expressions, which excel at detecting structured entities like credit card numbers but fail on contextual information. For instance, a prompt like “I just booked a flight using my Chase card, ending in 1111” wouldn't trigger a regex match even though it leaks partial financial data.

To address this, Cloudflare augments its WAF's Sensitive Data Detection (which uses regex on HTTP responses) with a Named Entity Recognition (NER) model. The design relies on Presidio, an open-source PII framework, deployed via Cloudflare Workers AI. This setup processes requests inline, flagging sensitive data before it ever reaches the model. The output metadata—"Was PII found?" and "What type of PII entity?"—is then handled by Firewall for AI and passed to other systems like Security Analytics for visibility and WAF Custom rules for enforcement.

Mitigation: Enforcement Through the WAF

Firewall for AI isn't a standalone product; it's integrated directly into Cloudflare's Web Application Firewall. This adds new fields to Custom and Rate Limiting rules, enabling immediate actions like blocking or logging risky prompts in real time.

Customers can design tailored policies for their AI traffic. For instance, a rule can interface with the new metadata fields to block requests containing detected PII prompts. Since enforcement happens before the request reaches your origin, this allows for a layered approach: existing WAF capabilities for traditional threats run in parallel with new AI-focused safeguards. Security Analytics and Security Events provide a centralized view into LLM interactions to spot risks.

The beta focuses on discovery and PII detection, with an enterprise beta available through your customer team. Looking ahead, Cloudflare plans to introduce token counting to analyze prompt length and structure. This would allow you to use the count in rate limiting rules to prevent expensive or abusive queries from reaching third-party models. Further out, AI-powered detection will move into content moderation, providing more flexible guardrail options.