Mitigating a token-length side-channel attack in our AI products

A Token-Length Side Channel in Streaming AI Responses

Length-based side-channel attacks have been a practical concern since CRIME, BREACH, TIME, and LUCKY-13 demonstrated that encrypted traffic can leak plaintext information through metadata like packet size. Researchers at Ben Gurion University recently applied this class of attack to AI assistants. Their paper, “What Was Your Prompt? A Remote Keylogging Attack on AI Assistants,” describes how an attacker can reconstruct text from encrypted LLM responses by observing token lengths in network traffic.

Cloudflare was contacted through its Public Bug Bounty program, and the Workers AI and AI Gateway teams worked with the researchers to reproduce the findings, validate the risk, and roll out mitigations. The vulnerability has been fully patched across both products.

How the Attack Leaks Token Lengths

Streaming is the standard mode for LLM chat interfaces: tokens are emitted sequentially as they are generated. For an attacker, each emitted token becomes a distinct network packet whose length roughly corresponds to the token’s size. By measuring the packet lengths of an encrypted session, an attacker can infer how many tokens were sent, their order, and their individual lengths. With that sequence and a dedicated LLM, the attacker can attempt to reconstruct the underlying text.

The attack requires two conditions:

• An AI chat client running in streaming mode.
• An attacker positioned to capture network traffic between the client and the AI service, such as on the same public network or through an ISP.

Inspecting the packets in a tool like Wireshark makes the leak visible. For example, a packet length of 95 may correspond to a four-character token like "Port", while the next packet at 93 corresponds to a two-character token like "ug". Once the token overhead is subtracted from the packet length, the transmitted tokens’ sizes are exposed. The researchers report that, in ideal conditions, their system can reconstruct 29% of an AI assistant’s responses and correctly infer the topic from 55% of them.

The attack only affects text-generation models in streaming mode, which includes most common LLM interactions. It also has limitations: the attacker must be on the network path, accuracy depends on knowing the target model’s writing style, and there is no ground truth against which to verify predictions. A reconstructed response may be highly accurate or only match on common conjunctions.

Padding to Hide Token Boundaries

Because the attack depends on inferring token length from packet size, it can be neutralized by obscuring that signal. The research paper proposes several mitigations, the simplest being to add random-length padding to each streamed response. Cloudflare evaluated the researchers’ testing notebook and reproduced the results on their examples, though accuracy varied greatly across different prompts and models. Still, the risk was deemed significant enough to act on.

Cloudflare adopted the padding approach and applied it to both of its AI products.

Protection at the Inference Layer

For Workers AI, where Cloudflare controls the inference requests end to end, the fix adds a new property, "p", to each streamed JSON object. This property holds a string of random variable length. The change is a variation of the suggested mitigation, applied to the JSON envelope rather than the raw tokens. It requires no SDK or client changes, is invisible to end users, and introduces network-level variability that destroys the attacker’s signal. All Workers AI customers are automatically protected, and no malicious exploitation was observed beyond the researchers’ ethical testing.

Extending Coverage Through AI Gateway

Cloudflare also applied the mitigation to AI Gateway, a proxy that sits between users and any supported inference provider. Many customers use third-party providers where Cloudflare does not control the upstream response format. By adding random-length padding to all streaming responses proxied through AI Gateway, the product protects customers even when the upstream provider has not independently patched the vulnerability. This is now enabled by default for all AI Gateway users.

For developers using an LLM provider that has not yet addressed this attack, routing traffic through AI Gateway provides the needed protection without altering the application code or the end-user experience.