Designing a Billing Pipeline That Handles 100,000 Events Per Second
Usage-based billing (UBB) is becoming the default pricing model for businesses that want to tie cost directly to customer value. Stripe has been expanding its own UBB offering within Stripe Billing, and the engineering behind it touches on problems that go well beyond invoicing: high-throughput event ingestion, exactly-once stream processing, and real-time financial accuracy under failure.
Building a system that delivers all of those at once forced some difficult trade-offs. The architecture Stripe settled on—asynchronous ingestion, active-active stream processing, and a dual-path aggregation layer—reflects a set of choices that are worth examining on their own terms.
Asynchronous Ingest Demands Better Observability
The second generation of Stripe's UBB API set some aggressive targets: 100x throughput over the first version, 99.999% availability, zero data loss, and low latency—all while keeping operational costs reasonable. The standard request/response model used by most Stripe APIs wouldn't get there. Those APIs hold synchronous requests across authentication, validation, routing, and RPC calls before executing business logic. That works for payments, but an event stream involves orders of magnitude more requests, making the synchronous model too slow and expensive.
The UBB API takes a different route. Events hit an edge router that performs stateless authentication and validation, then loads them directly onto an event bus. No synchronous processing is held up waiting for downstream systems. The event becomes available to the rest of the system—dashboards, billing logic, analytics—immediately.
That shift trades one problem for another. With synchronous processing, a failure surfaces right away. With asynchronous processing, a failed event can sit unnoticed. To address this, Stripe built two developer-facing tools: a Dashboard that shows asynchronous event processing in real time, and webhooks that fire when validation fails. The larger lesson: asynchronous APIs enable fast, cheap, reliable event streams, but only if the developer experience includes strong observability into what happens after the request leaves the caller's hands.
Active-Active Processing With a Reconciliation Metadata Layer
Once the pipeline could ingest 100,000 events per second per user, the next problem was processing that data accurately without sacrificing reliability or latency. Stripe chose Apache Flink for its distributed stream processing, low latency, and exactly-once guarantees. But Flink, like any complex system, has occasional downtime. That's tolerable for batch reporting or analytics, not for billing, where customers need precise real-time usage totals and invoices must be exact.
The solution was an active-active Flink deployment: the same event is processed simultaneously in two geographic regions. If one region fails, traffic can be routed automatically to the other. The trade-off is keeping data consistent across regions. Stripe solved that by attaching metadata to each event before it reaches either Flink application—a standardized timestamp plus event type and source. Because the timestamp is identical on both streams, reconciliation is possible when the streams drift apart. A delay in one stream can be resolved by comparing against the other, keeping aggregation and billing calculations accurate even during an outage. That metadata layer is the mechanism that makes active-active processing viable for financial workloads.
Matching Usage to Pricing Without Pausing the Stream
Ingesting and processing usage events is only half the problem. The other half is mapping those events to a customer's pricing structure—plan changes, discounts, credit burndown—and turning that mapping into an invoice. Stripe's approach models pricing as a stream of changes. At any moment, the stream represents the pricing structure currently assigned to a customer; changes to that structure appear as new entries in the stream. The billing system then matches the pricing stream against the customer's usage event stream.
That matching is straightforward when both streams stay aligned. It breaks down when they don't. Consider a retroactive discount applied from the start of the day. The system needs to open a lookback window, revisit past events, and re-rate them under the new pricing. The obvious approach—pausing the event stream while that happens—cancels out all the throughput advantages of the architecture. So Stripe built two pipelines moving at different speeds.
The fast path uses a 30-second tumbling window held in memory. It's responsible for billing alerts—notifying a customer when they're over budget or nearly out of credits—and guarantees events are never lost. The slow path uses a five-minute window and writes streaming events to disk in a transactional ledger. Without real-time pressure, it can handle delayed or out-of-order events and edge cases, while also producing analytics, invoicing data, and financial records for revenue recognition.
The dual-path design lets Stripe keep the event stream at full speed while retaining the ability to retroactively adjust pricing and reconcile differences. Near-real-time alerting and exactly accurate billing don't have to be the same pipeline.
Performance in Practice
The system Stripe ended up with handles 100,000 events per second per user, with a P95 latency under 30 seconds for time-sensitive operations. End-to-end latency—from usage ingestion to rated output—typically runs about five minutes, supporting near-real-time usage insights and timely invoicing.
