A proxy framework built for high-load traffic

Cloudflare has introduced Oxy, a proxy framework written in Rust that serves as the foundation for several production systems, including Zero Trust Gateway, the iCloud Private Relay second hop proxy, and an internal egress routing service. The project consolidates years of experience building high-load proxies into a reusable framework designed to handle massive daily traffic volumes while remaining highly customizable.

Oxy is not just a fixed-function proxy server. It is a framework in the sense that every component of the proxying pipeline — protocol decapsulation, traffic analysis, routing, tunneling logic, and DNS resolution — can be programmatically controlled by application code. At the same time, engineers can stand up a production-ready server with minimal code, relying on built-in configuration options and predefined traffic flow scenarios. The design philosophy is iterative: start with a basic deployment and layer in features through Oxy's extensibility points as requirements grow.

For example, building an HTTP firewall with Oxy requires enabling its built-in HTTP(S) proxying and implementing a hook handler for requests and responses. No additional code is needed for production concerns like metrics and logging. Similarly, a layer 4 firewall can be implemented by providing hooks for ingress and egress connections, allowing for nuanced logic such as geographic traffic routing or authentication rather than simple allow/deny decisions. If layer 4 information is insufficient, an application can request that Oxy decapsulate the traffic and process it with the HTTP firewall.

In practice, this foundation has proven effective: several privacy proxy applications built on Oxy serve substantial production traffic with fewer than two hundred lines of application code. Without the framework, these applications would have required orders of magnitude more code and development time. The framework's value comes from absorbing the hard-won lessons of building high-load services, leaving developers to focus on business logic.

Ingress: flexible on-ramps

On-ramps define how Oxy accepts ingress traffic, determined by the combination of transport layer socket type and protocols used by server listeners. Oxy supports a broad range of ingress flows:

  • HTTP 1/2/3, including CONNECT protocols for layer 3 and layer 4 traffic
  • TCP and UDP over Proxy Protocol
  • General-purpose IP traffic, including ICMP

Applications can inspect and manipulate traffic at multiple OSI layers, from layer 3 through layer 7. A key capability is forced decapsulation: an application receiving IP traffic can direct Oxy to upgrade the flow to a UDP tunnel for analysis at a higher level. Going further, the application can ask Oxy to sniff the UDP packets for HTTP/3 traffic; if detected, the flow upgrades to HTTP and HTTP/3 requests are handled natively. This enables simultaneous processing across L3, L4, and L7 perspectives on the same flow, providing a robust toolkit for complex traffic processing.

Egress: off-ramps and beyond

Off-ramps define the egress side — the combination of transport and protocols used by connectors when sending traffic out. Oxy supports HTTP 1 and 2, UDP, TCP, and IP for egress. The framework provides built-in DNS resolution and caching, supports custom resolvers, and includes automatic fallback for reliability. Additional features include happy eyeballs for TCP, advanced tunnel timeout handling, and the ability to route traffic to internal services with associated metadata.

Through integration with an internal egress service — itself an Oxy application — Oxy can provide geographical egress. Applications can route traffic to the public Internet from various cities across Cloudflare's network using configuration settings alone, with no additional development cost.

Tunneling, HTTP, and recursive processing

Between on-ramps and off-ramps, Oxy handles stateful tunneling of TCP, UDP, QUIC, and IP traffic, giving applications full control over blocking and redirection. For HTTP traffic, Oxy provides complete request and response control, enabling direct use as an HTTP or API service. It includes tools for streaming analysis of HTTP bodies, making it straightforward to extract data from uploads and downloads.

The framework goes beyond standard proxying with support for advanced HTTP tunneling methods like CONNECT-UDP and CONNECT-IP, leveraging the latest HTTP/3 and HTTP/2 extensions. Oxy can also process HTTP CONNECT request payloads at layer 4 and recursively process the payload, treating it as HTTP if the encapsulated traffic happens to be HTTP.

TLS and cryptography

Encryption is built on BoringSSL, with two available versions: a FIPS-compliant build offering a limited set of certified features, and a newer build supporting all currently available TLS features. Applications can switch between them in real-time, on a per-request or per-connection basis.

The TLS client component is browser-grade, performing certificate chain reconstruction, certificate revocation checks, and other standard verification steps when making HTTPS requests to upstream servers. Oxy applications themselves can be secured with TLS 1.3 and optionally mTLS, with client authentication information extractable from x509 certificates.

For security products, Oxy supports inspecting and filtering HTTPS traffic, including HTTP/3, and can dynamically generate certificates — a foundation for data loss prevention (DLP) implementations. The non-FIPS version of Cloudflare's BoringSSL fork also supports raw public keys as an alternative to WebPKI, which is well-suited for internal service communication by avoiding the overhead of certificate authority management.

Operational glue and observability

Beyond network primitives, Oxy handles operational concerns that typically require significant engineering effort. The framework manages application bootstrapping, including configuration parsing and application, asynchronous runtime setup, seccomp hardening, and automated graceful restart functionality. This means a few lines of bootstrap code yield a production-ready service with a wide range of startup options.

Observability is built in, with support for panic reporting to Sentry, Prometheus metrics exposed via a Rust-macro API, Kibana logging, distributed tracing, and memory and runtime profiling. Detailed audit logs for layer 4 traffic can be generated for billing and network analysis purposes. An integration testing framework uses TypeScript-based tests to verify application interactions.

A two-tier extension model

Oxy configurations are written in YAML, with numerous options available for each feature. Application developers can extend the configuration space using framework-provided macros. Defining a settings structure and annotating it with #[oxy_app_settings] is sufficient to add a custom configuration section:

///Application’s key-value (KV) database settings
#[oxy_app_settings]
pub struct MyAppKVSettings {
    /// Key prefix.
    pub prefix: Option<String>,
    /// Path to the UNIX domain socket for the appropriate KV 
    /// server instance.
    pub socket: Option<String>,
}

From this, Oxy can generate a default YAML configuration file that lists all available options along with their defaults, including any application-specific extensions. Options are auto-documented in the generated file using Rust doc comments.

Multi-tenancy is supported: a single application instance can expose multiple on-ramp endpoints, each with unique configuration. When YAML configuration reaches its limits, Oxy's hooks provide comprehensive Rust-level extension points covering nearly all aspects of traffic processing. This two-tier model — YAML for easy configuration, hooks for deep customization — accommodates a wide spectrum of application requirements with minimal friction. A basic Oxy application requires only a small amount of bootstrap code:

struct MyApp;

// Defines types for various application extensions to Oxy's
// data types. Contexts provide information and control knobs for
// the different parts of the traffic flow and applications can extend // all of them with their custom data. As was mentioned before,
// applications could also define their custom configuration.
// It’s just a matter of defining a configuration object with
// `#[oxy_app_settings]` attribute and providing the object type here.
impl OxyExt for MyApp {
    type AppSettings = MyAppKVSettings;
    type EndpointAppSettings = ();
    type EndpointContext = ();
    type IngressConnectionContext = MyAppIngressConnectionContext;
    type RequestContext = ();
    type IpTunnelContext = ();
    type DnsCacheItem = ();

}
   
#[async_trait]
impl OxyApp for MyApp {
    fn name() -> &'static str {
        "My app"
    }

    fn version() -> &'static str {
        env!("CARGO_PKG_VERSION")
    }

    fn description() -> &'static str {
        "This is an example of Oxy application"
    }

    async fn start(
        settings: ServerSettings<MyAppSettings, ()>
    ) -> anyhow::Result<Hooks<Self>> {
        // Here the application initializes various hooks, with each
        // hook being a trait implementation containing multiple
        // optional callbacks invoked during the lifecycle of the
        // traffic processing.
        let ingress_hook = create_ingress_hook(&settings);
        let egress_hook = create_egress_hook(&settings);
        let tunnel_hook = create_tunnel_hook(&settings);
        let http_request_hook = create_http_request_hook(&settings);
        let ip_flow_hook = create_ip_flow_hook(&settings);

        Ok(Hooks {
            ingress: Some(ingress_hook),
            egress: Some(egress_hook),
            tunnel: Some(tunnel_hook),
            http_request: Some(http_request_hook),
            ip_flow: Some(ip_flow_hook),
            ..Default::default()
        })
    }
}

// The entry point of the application
fn main() -> OxyResult<()> {
    oxy::bootstrap::<MyApp>()
}

Why Rust?

Oxy is written in Rust, a language Cloudflare has increasingly adopted for new products. The choice comes down to Rust's ownership and borrowing model, which eliminates entire classes of bugs such as null pointer dereferences and data races at compile time, while still offering the low-level control and minimal runtime overhead needed for performance-critical proxy workloads.

The project deliberately avoids reinventing the wheel. Its backbone is built on the widely adopted hyper and tokio open-source libraries. The engineering philosophy is to pull from proven, battle-tested solutions wherever possible to speed up iteration. When existing code doesn't fit, the team collaborates with upstream maintainers to contribute fixes rather than forking quietly — two Oxy team members are now core contributors to tokio and hyper.

Although Oxy itself is proprietary, Cloudflare gives back to the ecosystem by open-sourcing key building blocks, including boring and quiche, which the framework depends on.

From Proof-of-Concept to Framework

Oxy's origins trace back to a proof-of-concept HTTP firewall built in Rust, initially housed inside the WARP service repository. That PoC quickly outgrew its home, and it was spun out into a dedicated Gateway proxy for both technical and operational reasons.

Shortly after, the team needed a relay proxy for iCloud Private Relay. Rather than starting from scratch, early iterations of the relay service were forks of the Gateway server. This also let the Gateway project benefit from HTTP/3 support that was being added for Private Relay.

Seeing the overlap between the two codebases, engineers extracted their common elements into a standalone framework: Oxy. The project's commit history can be traced back through both predecessor projects up to its separation. This hands-on approach — building real products first, then identifying reusable components — has shaped Oxy's development ever since. A small core team stewards the framework, while internal contributors from across the company are brought in as subject-matter experts, helping shape the API to be both functional and ergonomic for its users.

Oxy vs. Pingora

Cloudflare also maintains Pingora, another Rust-based proxy, and it's easy to confuse the two. They were, in fact, conceived around the same time, and merging them was considered. That idea was quickly abandoned when it became clear their objectives were fundamentally different.

Pingora is built for a very specific job: handling traffic between Cloudflare and its clients' upstream servers across the broader Internet. That means accommodating notoriously difficult and unusual configurations — non-UTF-8 URLs, and TLS settings that most libraries won't support. It exists to establish connectivity at the edge, even in the most obscure technical corners.

Oxy, in contrast, is a general-purpose, multipurpose platform. It supports a wider range of communication protocols and is designed to make it straightforward to build high-performance proxy applications with custom business logic.

Looking Ahead

Oxy is a foundational piece of Cloudflare's ongoing effort to modernize its architecture. Because it emerged from real, in-production use cases, it's designed to be flexible and scalable enough to adapt to a wide variety of needs.

The development model remains iterative and collaborative. The team continues to seek out opportunities to consolidate code, reuse existing solutions, and contribute improvements upstream. This approach has already produced tangible results and will drive Oxy's evolution as the demands on Cloudflare's infrastructure continue to grow.