Streaming Responses in Serverless Environments

Vercel has announced stable support for HTTP response streaming across both its Node.js (Lambda) and Edge runtimes, a first among serverless computing providers. This capability lets developers build high-performance web applications that prioritize speed, scalability, and efficient resource usage without being constrained by traditional serverless response models.

HTTP streaming is a data transfer technique where the server sends response data in incremental chunks rather than as a single, fully generated payload. Traditional HTTP responses require the server to compile the entire payload before sending it to the client. While acceptable for smaller datasets, this approach becomes a bottleneck for larger or dynamically generated content, leading to higher latency and slower page loads. Streaming changes this dynamic by delivering data as it becomes available.

Network and main thread activity for a non-streaming HTTP request

Browsers can process and render streamed chunks immediately upon receipt. This significantly reduces latency, notably improving Time To First Byte (TTFB) and First Contentful Paint (FCP) metrics.

Network and main thread activity for a streaming HTTP request

Key Benefits of Streaming

  • Improved performance: Streaming reduces TTFB and accelerates page loads. Sending responses in chunks ensures that slower backend services don't block the entire page from rendering. The browser's ability to incrementally process data on the main thread also allows it to handle user input more effectively, resulting in a smoother, more responsive experience.
  • Large payload handling: Data can be transmitted without waiting for the full payload to be generated, effectively eliminating AWS Lambda response size limitations and boosting rendering performance.
  • Enhanced scalability: By avoiding the need to buffer entire responses in memory before sending, streaming reduces overall memory consumption, improving application scalability.

How Vercel Overcomes Lambda Limitations

Vercel's serverless architecture is primarily powered by AWS Lambda, which does not yet natively support HTTP streaming. To bridge this gap, Vercel's engineering team developed a custom solution that enables streaming in their Serverless Functions.

When a streaming-enabled request reaches a Vercel Serverless Function, it first passes through an Edge Location. The request is then routed to a Serverless Function Invocation Service, which inspects and modifies it to include streaming-specific attributes, such as a callback URL. This modified request is subsequently forwarded to AWS Lambda.

Request flows through Edge Location to Vercel’s Serverless Function Invocation Service. This forwards the request to the Lambda with the callback URL, which the bridge uses to establish a secure socket connection.

Inside Lambda, where native streaming support is absent, Vercel uses special bridge code to facilitate response streaming. This bridge establishes a secure socket connection to the provided callback URL, using it to transmit the response stream back to the invocation service. A receiver within this service reads the incoming data from the socket and forwards it to the client as the response to the original HTTP request. The client can start processing and rendering data as soon as the first chunk arrives, without waiting for the stream to complete.

Vercel stands as the first platform to enable this capability on AWS Lambda, a development that allows developers to create web applications focused on speed, scalability, and efficient resource use. Developers can utilize this feature with frameworks such as Next.js Route Handlers, React Server Components, Remix Streaming SSR, and SvelteKit.