From streaming hack to compute overhaul
Fluid compute started out as a workaround for a simple limitation: AWS Lambda doesn't natively support streaming HTTP responses. When the Next.js team began building the App Router on React Server Components, streaming server-rendered UI became a requirement. Lambda's execution model, where a function receives an input blob, runs, and returns an output blob, couldn't express a chunked response.
The solution was a secure TCP-based transport tunneled between Vercel's infrastructure and each Lambda instance. Instead of encoding the entire HTTP response as a single blob, the function sends discrete packet types—ResponseStarted, one or more ResponseBody packets, and a final ResponseEnd—back through the tunnel. The Vercel Function Router reconstructs those packets into a normal HTTP response and streams it to the client.
A Rust-based core runs inside each function as the glue between user code and this transport. It communicates with the language runtime (Node.js or Python) over HTTP and with the Function Router over the TCP protocol. Each response chunk is mapped to its packet type and forwarded. Since the protocol is extensible, features like waitUntil, request metrics, session tracing, and larger logs were all shipped later as new packet types.
Reversing the channel
Once the outbound tunnel was in place, the team realized the same connection could be used to send data the other way. If Vercel could push an additional HTTP request into an already-running Lambda instance, it would break the strict one-invocation-per-instance model that defines Lambda's cost structure.
The traditional model is wasteful for I/O-bound work. Each invocation occupies one instance for its full wall time, even when the function is idle waiting on a database query, an external API, or an LLM response. With longer-running AI workloads becoming common, that idle time became a significant cost. Multiplexing multiple requests onto a single instance would let the same resources serve more traffic, reduce cold starts, and lower costs for any workload with concurrency greater than one, with a single tunnel reused to carry multiple concurrent streams.
Finding the right instance
Reusing a tunnel requires knowing which functions are running and where. Vercel's infrastructure spans 19 regions, each with many replicas of the Proxy and Function Router pods. The larger the region, the lower the chance that an incoming request lands on a router pod that already has a connection open to the target function.
To solve that coordination problem, Vercel built a service called compute-resolver. It acts as a DNS-like lookup for Proxy pods, tracking where previous requests for a given function ID were routed so new requests can be directed to a router pod that may already have a warm connection. It also balances load across router pods to avoid hot spots during traffic spikes. The service handles over 100K RPS at peak with sub-millisecond p99.99 resolution times, and more than 99% of requests are routed to a pod that may already have a live request for the same function.
Load-aware admission control
Sending more traffic to an instance is only safe if the instance can handle it. Each Lambda has fixed vCPUs and memory, so overloading one would degrade latency or cause OOM errors. The system needed a real-time answer to whether an instance can accept more work, which requires knowing its current CPU and memory usage, throttling state, remaining lifetime, file descriptor usage, and whether previous requests on that instance failed.
The heuristics live in the Rust-based core, which continuously sends in-band metrics to the Function Router. The router uses those metrics to pick the instance with the most available resources and fewest concurrent requests, rather than using round-robin, since requests hitting the same route can have very different CPU and memory profiles.
But the router's view of metrics can be stale due to network latency or race conditions. So the Rust core also maintains local metrics and can send a nack for any request it can't handle. On receiving a nack, the router pauses traffic to that instance, updates its internal state, and forwards the request to another available instance or spins up a new one. The heuristics are language-agnostic and apply equally to Python functions, with more runtimes to come.
The pricing model that follows
Fluid compute's multiplexing made concurrent workloads cheaper, but applications with little or no concurrency didn't benefit—there are no idle instances to share work with. And long-running but mostly-idle workloads such as Model Context Protocol (MCP) servers still incurred cost for the full wall time.
The same architecture that enables multiplexing also enables a different billing model. Active CPU pricing charges only for the resources actually consumed: active CPU time measured in milliseconds and provisioned memory measured in GB-hours. This delivers additional savings of over 90% for heavily I/O-bound workloads, on top of what Fluid compute already saves from instance reuse.
Fluid compute now powers more than 45 billion weekly requests and is the default for new projects. Over 75% of all Vercel Function invocations use it, saving customers up to 95% on compute costs. No CPU is billed while a function instance is idle.
