Bringing gRPC to the Edge: A Protocol Translation Story
Cloudflare began supporting gRPC during the 2020 Birthday Week beta. The core challenge wasn't simply flipping a switch: gRPC relies on HTTP/2 features — most notably trailers and bidirectional streaming — that were not fully supported across the company's internal proxy pipeline. The solution involved translating between two related but distinct protocols at different points in the request path.
Why gRPC Is Tricky to Proxy
gRPC is an open-source RPC framework that runs over HTTP/2. It allows one machine to invoke procedures on another, with several defining characteristics:
- It mandates HTTP/2 as the transport, which is now widely available.
- Complete client/server reference implementations, demos, and test suites are available as open source.
- It does not dictate a message format, though Protocol Buffers are the preferred serialization method.
- Both clients and servers can stream data, eliminating the need for polling or repeated connection establishment.
A gRPC request and response look much like a standard HTTP/2 request at the frame level. The unusual part is the use of HTTP trailers. Trailers have existed since the original HTTP/1.1 specification in 1999 — they are headers appended after the message body. In HTTP/1.1, this requires chunked transfer encoding; in HTTP/2, trailers arrive in a HEADER frame following the DATA frames of the body. Despite their long history, trailers are rarely used and many server and client implementations lack full support for them.
Trailers are useful in specific scenarios. For instance, a server streaming a large, variable-size payload cannot attach a checksum in the initial response header, since that header must be sent before the body begins. A trailer allows sending metadata — such as a SHA256 Digest — after all data has been transmitted. gRPC leverages trailers for two critical purposes: communicating the final status code (grpc-status) after content is sent, and conveying post-processing results for long-lived streaming requests, including error codes that arise mid-stream.
The basic frame layout for a gRPC exchange looks like this:
Three Obstacles at the Edge
Since Cloudflare already supported HTTP/2, native gRPC support might seem straightforward. However, several architectural issues stood in the way:
- Trailer support: Cloudflare's edge proxy, which accepts traffic from eyeballs using NGINX, has limited trailer support. Requests also pass through multiple other proxies inside the network.
- HTTP/2 to origin: The edge proxy fetches objects from origins using HTTP/1.1. Proxying gRPC traffic requires HTTP/2 connections to customer origins.
- Bidirectional streaming: gRPC supports both unary (simple request/response) and streaming modes. Streaming allows non-stop data flow in either direction — for example, client streaming keeps sending a request body even after the response headers have been received.
Because of these constraints — particularly the cascade of HTTP/1.1 proxies in the internal pipeline and the nature of the load-balancing architecture — deploying HTTP/2 everywhere internally was not viable. The NGINX gRPC upstream module was considered, but it requires HTTP/2 downstream and operates as a separate module, making it incompatible with the existing multi-proxy setup.
The Translation Approach: gRPC to gRPC-Web
The team found a path forward by converting gRPC messages to a format that works over HTTP/1.1 inside the network, then converting them back to HTTP/2 before sending to the origin. Rather than designing a new format, they leveraged an existing community specification: gRPC-Web.
gRPC-Web was originally created to bring gRPC to web browsers, which cannot directly access HTTP/2 frames. Its key modification is moving the HTTP trailer into the message body, eliminating the need for trailer support within proxies. It also includes streaming support. This means that once converted, gRPC traffic can be inspected by security products — such as WAF and Bot Management — just like any other HTTP request.
The flow works as follows. When an HTTP/2 gRPC message arrives at the edge proxy, it is converted to HTTP/1.1 gRPC-Web format. From there, it travels through the standard pipeline — WAF, Cache, Argo, and other services — as a normal HTTP request. Just before the message leaves the Cloudflare network, it is converted back to HTTP/2 gRPC for delivery to the origin. Converted requests are marked so the system does not mistakenly translate genuine gRPC-Web traffic originating from clients.
Building HTTP/2 Origin Support
Before this project, Cloudflare had no capability to connect to origins via HTTP/2. The team therefore built a standalone origin proxy from scratch that supports HTTP/2 upstream connections. The gRPC conversion logic was implemented on top of this new platform, making gRPC the first feature to use it. Broader HTTP/2-to-origin support is planned for the future.
Handling Streaming
Streaming gRPC — where message blocks can be sent at any time during the request's lifetime — required special handling. The end of a stream is indicated by a HEADER frame with the END_STREAM flag. When translating to gRPC-Web, the body is sent using chunked encoding, and the connection is kept open to accept bidirectional data flow until the end-of-stream marker arrives. This necessitated proxy support for simultaneous bidirectional transfer, particularly for client streaming, where the server may send its response headers and status while the client is still transmitting the request body.
Testing and Interoperability
During initial development, the team used the Envoy proxy with its gRPC-Web filter alongside official gRPC examples. A test environment was set up where requests from a gRPC test client went to the edge (converting to gRPC-Web), then to Envoy, which converted them back to gRPC before forwarding to a gRPC test origin. This validated basic behavior.
Deeper interoperability testing followed, referencing the existing gRPC interoperability test cases. The first iteration ran between the edge proxy and the new origin proxy locally. The second iteration introduced different gRPC implementations to cover variations in how status is returned — some servers send grpc-status in a trailers-only response (a single HEADERS frame with both END_STREAM and END_HEADERS flags) for immediate errors, while others send the final status as a trailer in a separate HEADERS frame. After local verification, the test harness was run against a full development environment mirroring production services, ensuring no unintended side effects.
Dogfooding provided the final validation. The Cloudflare drand randomness beacon was one of the first services deployed on the edge gRPC support, running in production without issue for several weeks.
A Pragmatic Design Choice
The key decision — converting between HTTP/2 gRPC and HTTP/1.1 gRPC-Web within the network — allowed support to be built quickly without disruptive changes to existing systems. It leveraged an existing specification, enabled compatibility with the internal HTTP/1.1 proxy infrastructure, and maintained full visibility for security and performance services. The result was a faster path to production that met the technical constraints of the platform.
