When a mitigation became the incident

Cloudflare's public postmortems usually follow a familiar pattern: an outage, a root cause, a fix. This time the company is writing about something narrower but arguably more serious — a single customer's site, tardis.dev, was made unusable by Cloudflare's own actions. The throttle applied to the customer's traffic was not a policy enforcement action, a plan limitation, or a terms-of-service violation. It was a mistake, and Cloudflare says so plainly.

The affected customer was not banned, suspended, or otherwise locked out of their account. Instead, Cloudflare applied a bandwidth throttle between its network and the customer's origin server. The result was that the site effectively stopped working. The traffic in question was passing through Cloudflare Workers, for which the customer was paying, and the company says there was no need for any plan upgrade. The incident also caused internal confusion: some support staff incorrectly believed the throttle was tied to Section 2.8 of the Cloudflare Self-Serve Subscription Agreement, which prohibits using the self-service CDN to serve excessive non-HTML content like images and video without a paid plan that includes those services. That clause exists to prevent, for example, someone building an image-hosting service on the free tier and consuming outsized bandwidth — but it did not apply here.

What triggered the throttle

The sequence began on February 2, when an on-call network engineer received an alert about a congesting interface with Equinix IX in Cloudflare's Ashburn data center. The alert was unusual for two reasons: it was the second consecutive day it had fired, and the congestion was caused by a sudden, extreme spike in traffic.

The engineer traced the spike to tardis.dev, which was pulling traffic between Cloudflare and the customer's origin network, a storage provider. That congestion was on a physical interface connected to external peers, so the impact was immediate and broad — packet loss, reduced throughput, and elevated latency for many customers and peers. Cloudflare's automatic mitigation for congesting interfaces was in place but could not fully resolve the problem.

The customer's traffic pattern had changed dramatically: requests went from an average of 1,500 per second with a 0.5 MB payload per request to 3,000 requests per second and more than 12 MB per request — a 2x increase in request rate and a 25x increase in payload size. Because the requests were all unique URLs going to the origin, caching could not help; Cloudflare had no ability to serve the content from cache.

At that point, an engineer decided to apply a throttling mechanism to prevent the zone from pulling so much traffic from its origin. That decision is the core of the incident: Cloudflare has no established process for throttling customers who consume large amounts of bandwidth, and says it does not intend to create one. The action was unsanctioned, and the company describes it as a deep regret. The throttle was lifted through internal escalation 12 hours and 53 minutes after it was applied.

Process changes and what comes next

To prevent a repeat, Cloudflare says it is establishing clearer rules for handling such situations. Any action taken against a customer domain — paying or not — will now require multiple levels of approval and clear communication to the customer. The tooling used by engineers will be updated to reflect those requirements. The company notes that it has many traffic-shaping options for situations where a traffic spike affects a link, and that a different mitigation could have been applied here.

Cloudflare is also rewriting its terms of service to better match the kinds of services customers actually run on the platform today, and says it will explain in plain language what is permitted under self-service plans. A follow-up blog post dedicated to those terms changes is planned.