Pingora is now open source

Cloudflare has released Pingora, the Rust framework behind its HTTP proxy services, under the Apache License version 2.0. Pingora is an async, multithreaded framework for building programmable network services, and it has already handled close to a quadrillion requests across Cloudflare's network. The code is available on GitHub.

The move is part of a broader push toward memory-safe internet infrastructure. Cloudflare is collaborating with the Internet Security Research Group's (ISRG) Prossimo project to encourage adoption of Pingora in critical internet systems. Prossimo focuses on replacing vulnerable components with memory-safe alternatives, and Pingora is a prime candidate given its Rust foundation.

What the framework includes

Pingora provides building blocks not just for proxies, but also for clients and servers. It supports HTTP/1 and HTTP/2 end-to-end, along with gRPC and WebSocket proxying; HTTP/3 is on the roadmap. On the transport side, you get TLS, TCP, and Unix domain sockets. Load balancing and failover are customizable, and you can plug in either OpenSSL or BoringSSL for TLS, giving you options for FIPS compliance and post-quantum crypto.

The framework's API surface includes filters and callbacks for shaping how a service processes, transforms, and forwards requests. Users familiar with OpenResty and NGINX will recognize the pattern, as the callbacks map closely to the *_by_lua style of request lifecycle hooks. Utility libraries cover common needs like event counting, error handling, and caching.

Operationally, Pingora supports zero-downtime graceful restarts, so you can upgrade a running service without dropping a single request. Observability integrations for syslog, Prometheus, Sentry, and OpenTelemetry are also included.

Who should consider Pingora

Pingora makes sense in a few scenarios. If security is the top concern, it offers a memory-safe alternative to C/C++ services. In practice, the Rust-based codebase reduces memory safety bugs and frees up time for feature work rather than debugging unsafe code.

Performance-sensitive workloads also stand to benefit. The multithreaded architecture saves CPU and memory compared to earlier designs, which matters for services where speed and cost are tightly coupled. And for teams needing deep customization, the proxy framework's programmable APIs make it possible to build advanced gateways and load balancers.

Building a load balancer with Pingora

To show how the API works, here is a minimal load balancer that round-robins between two upstream HTTPS servers. You start with a blank HTTP proxy:

pub struct LB();

#[async_trait]
impl ProxyHttp for LB {
    async fn upstream_peer(...) -> Result<Box<HttpPeer>> {
        todo!()
    }
}

Any object implementing the ProxyHttp trait is an HTTP proxy. The only required method is upstream_peer(), called for every request. It returns an HttpPeer that specifies the origin IP and connection details. Pingora ships a LoadBalancer with common selection algorithms like round robin and hashing; more sophisticated logic can be implemented directly in upstream_peer().

pub struct LB(Arc<LoadBalancer<RoundRobin>>);

#[async_trait]
impl ProxyHttp for LB {
    async fn upstream_peer(...) -> Result<Box<HttpPeer>> {
        let upstream = self.0
            .select(b"", 256) // hash doesn't matter for round robin
            .unwrap();

        // Set SNI to one.one.one.one
        let peer = Box::new(HttpPeer::new(upstream, true, "one.one.one.one".to_string()));
        Ok(peer)
    }
}

Because we are connecting to an HTTPS server, SNI must be set. Certificates, timeouts, and other options can also be configured on the HttpPeer.

Finally, the load balancing service is told to listen on 127.0.0.1:6188 and handed to a Pingora server that runs the process:

fn main() {
    let mut upstreams = LoadBalancer::try_from_iter(["1.1.1.1:443", "1.0.0.1:443"]).unwrap();

    let mut lb = pingora_proxy::http_proxy_service(&my_server.configuration, LB(upstreams));
    lb.add_tcp("127.0.0.1:6188");

    let mut my_server = Server::new(None).unwrap();
    my_server.add_service(lb);
    my_server.run_forever();
}

Testing it reveals a problem:

curl 127.0.0.1:6188 -svo /dev/null
> GET / HTTP/1.1
> Host: 127.0.0.1:6188
> User-Agent: curl/7.88.1
> Accept: */*
> 
< HTTP/1.1 403 Forbidden

The origin returns a 403 because the proxy forwards the Host header set by curl (i.e., 127.0.0.1:6188), which the origin rejects. The fix is to add an upstream_request_filter, which runs after the origin connection is established and before the request is sent. This filter can add, remove, or rewrite request headers:

async fn upstream_request_filter(…, upstream_request: &mut RequestHeader, …) -> Result<()> {
    upstream_request.insert_header("Host", "one.one.one.one")
}

With that filter in place, the proxy behaves correctly:

curl 127.0.0.1:6188 -svo /dev/null
< HTTP/1.1 200 OK

The full example is available on GitHub. More filters and callbacks exist at different stages of the request lifecycle for modifying, rejecting, routing, and logging requests and responses. Behind the scenes, Pingora handles connection pooling, TLS handshakes, parsing, and other proxy plumbing, letting developers focus on the logic that matters to them.

What to watch out for

Pingora is a library, not a turnkey executable. It is the engine, not the car. Cloudflare and ISRG plan to build a batteries-included application on top of it for lower-code configurations, but that work is still ahead.

Two caveats apply to the current release. First, the API is not yet stable. During this pre-1.0 period, components like request and response filters may change. Second, non-Unix operating systems are not on the roadmap now, though cloudflare says that could change later.

Bugs, documentation issues, and feature requests can be filed via the GitHub issue tracker; the project's contribution guide should be reviewed before opening a pull request.