HTTP/2 Rapid Reset: Anatomy of a 201 Million RPS Attack
On August 25, 2023, our automated DDoS mitigation systems began flagging unusually large HTTP attacks against multiple customers. Within days, these campaigns escalated to record-breaking volumes, ultimately peaking at just over 201 million requests per second—nearly three times the size of the largest attack we had previously documented.
What made this wave particularly alarming was the attacker's efficiency: the entire assault was generated by a botnet of only about 20,000 machines. For context, the global web typically handles between 1–3 billion requests per second, and many botnets today comprise hundreds of thousands or millions of devices. This suggests that a sufficiently large botnet could, in theory, concentrate an entire web's worth of traffic against a small set of targets using this same method.
Why This Attack Was Different
The attack exploited a fundamental weakness in the HTTP/2 protocol specification rather than a bug in any single implementation. As detailed in CVE-2023-44487, the underlying issue affects any vendor that has implemented HTTP/2—which includes every modern web server. Because of this, we believe all HTTP/2 implementations are susceptible to this attack vector.
We observed these attacks at the same time as two other major industry players—Google and AWS—and coordinated with them on a responsible disclosure to affected vendors and critical infrastructure providers. Our joint efforts have focused on informing web server vendors so they can implement patches. In the interim, the most effective defense is to place a DDoS mitigation service in front of any web-facing web or API server.
Initial Impact and Current Protections
During the first wave, roughly 1% of customer requests experienced some impact. However, we have since refined our mitigation methods so that no Cloudflare customer is affected by this attack, and our systems remain stable even under sustained assault. These protections are now active for all customers by default.
The remainder of this analysis explains the specific HTTP/2 features that attackers weaponized and the mitigation strategies we deployed. By publishing these details, we aim to equip other web servers and services with the information needed to defend against this attack—and to inform the HTTP/2 standards team and future protocol designers about how to prevent similar abuse.
How rapid resets crash the concurrency model
The HTTP/2 protocol solved a fundamental inefficiency in HTTP/1.1: the serial, strictly ordered exchange of whole messages over a single TCP connection. HTTP/1.1 forced browsers to juggle pools of connections (typically up to six per host) and to choose between waiting out queued responses or tearing down connections to cancel unwanted requests.
HTTP/2 replaces that with a frame-based wire format. Every message is broken into frames carrying a type, length, flags, and stream identifier. Because the stream ID tells the receiver which message each frame belongs to, many streams can be multiplexed and interleaved over one connection. Streams are bidirectional: the client sends frames on odd-numbered IDs (1, 3, 5, ...) and the server replies on the same IDs. A simple GET becomes one HEADERS frame from the client, followed by the server's HEADERS and one or more DATA frames on the same stream.
Multiplexing makes better use of a single TCP connection, but it also lets a client launch a much larger amount of parallel work than HTTP/1.1 ever allowed — an obvious avenue for denial-of-service. To keep that in check, HTTP/2 defines a concurrency limit via the SETTINGS_MAX_CONCURRENT_STREAMS parameter. A server announces, say, a limit of 100; if a client tries to open more active streams than that, the server rejects the excess with a RST_STREAM frame, without disturbing other in-flight streams.
The catch is which streams count against the limit. Only streams in the open or half-closed states count toward the advertised maximum. When a client cancels a stream, that stream transitions to closed and immediately stops consuming concurrency quota. The client can then open a fresh stream in its place. This lifecycle fact is the foundation of CVE-2023-44487.
The request cancellation feature under stress
HTTP/2's request cancellation is normally a welcome feature. A browser scrolling through a page of images can send an RST_STREAM for images that have scrolled out of view, freeing server resources and bandwidth so that newly visible images load faster. In HTTP/1.1, the same action required closing the whole connection.
On the wire, a canceled stream passes through its lifecycle quickly. The client's HEADERS frame with END_STREAM set moves the stream from idle to open to half-closed almost instantly. A subsequent RST_STREAM immediately moves it to closed. Once closed, the stream no longer contributes to the concurrency count, so the client is free to open another stream and send another request immediately. Nothing in the protocol forces the client to wait for the server to acknowledge the cancellation.
That mechanism becomes an attack when the client churns through an unbounded number of streams, each with a request followed by a rapid reset. If the server can process RST_STREAM frames and clean up local state fast enough, the traffic is harmless. Problems appear when there is any lag between reading a reset and actually tearing down the associated work.
Reverse proxies and load balancers are especially exposed. A typical deployment terminates HTTP/2 at the proxy and dispatches each request upstream as asynchronous work. That design lets the proxy handle client connections efficiently, but it also makes cleanup harder: when the proxy reads an RST_STREAM, it must notify the upstream and tear down local state. A malicious client can send a long chain of request-and-reset pairs at the start of a connection. Cloudflare's own reverse proxies, which read buffered socket data in order and dispatch each request upstream as it is parsed, were vulnerable to exactly this pattern: eagerly consuming an enormous chain of requests and resets created stress on upstream servers until they could no longer accept new incoming requests.
The key point is that the SETTINGS_MAX_CONCURRENT_STREAMS value cannot stop this attack. Because canceled streams immediately free their concurrency slot, the client can generate arbitrarily high request rates regardless of the advertised maximum.
Anatomy of an attack trace
A proof-of-concept client targeting an unmitigated off-the-shelf server illustrates the mechanics. The client attempts a total of 1,000 requests in a test environment, and the traffic is captured with Wireshark:
The sheer number of frames makes the trace hard to read at a glance. Wireshark's statistics summary shows the shape of the attack:
The server's opening SETTINGS frame in packet 14 advertises a concurrency limit of 100. In packet 15, the client sends control frames and then begins a rapid sequence of requests, each immediately reset. The first HEADERS frame is 26 bytes; all subsequent ones are only 9 bytes. The difference is the result of HPACK compression — after the first request establishes the compression context, the same headers take far fewer bytes. That single packet carries 525 requests, using stream IDs up to 1051.
The RST_STREAM for stream 1051 does not fit in packet 15, so in packet 16 the server manages to respond with a 404. Only in packet 17 does the client send the final reset, before continuing with the remaining 475 requests. Notably, both client packets contain far more HEADERS frames than the advertised concurrency limit of 100. The client never waited for any server return traffic; it was limited only by packet size. And no server RST_STREAM frames appear anywhere in the trace, meaning the server never detected a concurrent-stream violation. The concurrency limit was functionally useless against this client.
How customers were affected
Because canceled requests notify upstream services to abort before significant resources are consumed, most malicious requests in these attacks never reached origin servers. Still, the unprecedented scale of incoming requests caused measurable impact in Cloudflare's most affected data centers.
The most visible symptom was an increase in 502 Bad Gateway errors. Cloudflare's infrastructure handles HTTPS traffic through a chain of proxies: a TLS decryption proxy first processes HTTP/1, HTTP/2, or HTTP/3 traffic, then forwards requests to a "business logic" proxy responsible for customer settings, routing, and — critically — Layer 7 security features such as attack mitigation.
With the rapid reset attack, each connection delivered an enormous number of requests that had to reach the business logic proxy before any blocking decision could be made. When request throughput exceeded proxy capacity, the connection between the TLS proxy and its upstream saturated, preventing new connections and producing bare 502 errors for clients.
Notably, these errors are not visible in the Cloudflare dashboard, because HTTP analytics logs are emitted by the same business logic proxy that was overwhelmed. Internal dashboards showed about 1% of requests impacted during the initial attack wave before mitigations were deployed, with peaks near 12% for a few seconds during the most severe attack on August 29th.
Following mitigations and stack changes, that error rate has dropped to effectively zero.
499 errors and stream concurrency limits
Some customers also observed an increase in 499 errors, which stems from HTTP/2 maximum stream concurrency settings rather than infrastructure saturation.
HTTP/2 peers exchange SETTINGS frames at connection start, but clients often don't wait for the server's settings before issuing requests. The default for SETTINGS_MAX_CONCURRENT_STREAMS is effectively unlimited (the 31-bit stream ID space allows 1,073,741,824 concurrent streams). Since the specification recommends servers support at least 100 streams, many clients gamble on that number and begin sending immediately. If the server's actual limit is lower, streams get reset.
Servers reset streams for many reasons beyond concurrency violations — HTTP/2 requires stream closure on parsing or logic errors. In response to earlier HTTP/2 DoS vulnerabilities, Cloudflare deployed mitigation counters that track server resets per connection and close connections with a GOAWAY frame when thresholds are exceeded. This strategy distinguishes occasional client mistakes from broken or malicious behavior.
During the CVE-2023-44487 response, Cloudflare reduced maximum stream concurrency to 64. However, clients that assume the recommended default of 100 immediately issue that many requests — for example, an image gallery page can trigger 100 simultaneous requests. The 36 streams over the limit triggered reset counters, causing legitimate connections to be closed and pages to fail loading entirely. Once this interoperability issue was identified, concurrency was restored to 100.
Cloudflare's mitigations
Cloudflare's 2019 response to HTTP/2 DoS vulnerabilities laid groundwork for addressing CVE-2023-44487. Existing protections were extended to monitor client-sent RST_STREAM frames, closing connections where they were used abusively while leaving legitimate uses untouched.
Beyond the direct fix, Cloudflare improved HTTP/2 frame processing, request dispatch code, and added queuing and scheduling enhancements to the business logic server. These changes reduce unnecessary work, improve cancellation responsiveness, and increase headroom before saturation.
Earlier attack interception
Cloudflare's "IP Jail" system, designed for hyper-volumetric attacks, collects attacking client IPs and blocks them from connecting to the targeted property — either at the IP level or via the TLS proxy. However, this system requires a few seconds to activate, and this botnet ramps up with effectively no delay. During those seconds, origin protection works but the infrastructure must still absorb all HTTP requests.
To close that gap, IP Jail was expanded to protect all of Cloudflare: a jailed IP is not only blocked from the attacked property, but also forbidden from using HTTP/2 to any other domain on the platform for a period of time. Protocol abuse isn't feasible over HTTP/1.x, so attackers lose the ability to run large-scale attacks, while legitimate clients sharing an IP experience minimal performance impact. Because IP-based mitigations are blunt instruments, and botnet IPs have short lifespans, the system is carefully tuned to minimize false positives.
Handling these actions in the TLS proxy at the start of the HTTPS pipeline conserves significant resources compared to regular Layer 7 mitigation. As a result, the botnets' random 502 errors are now zero.
Better observability
Returning errors to clients without those errors appearing in customer analytics was unsatisfactory. A pre-existing project to overhaul logging systems has gained urgency: the goal is for each infrastructure service to log its own data rather than relying on the business logic proxy to consolidate and emit logs. Work is also underway on connection-level logging to spot protocol abuses faster and strengthen DDoS mitigation capabilities.
Looking ahead
Record-breaking attacks will keep coming, and Cloudflare continues to proactively identify threats and deploy countermeasures across its global network so customers receive automatic protection.
