Extending Oxy through the connection lifecycle
Oxy, our Rust framework for building proxies, handles the common plumbing of proxy infrastructure through a YAML configuration file — listeners, telemetry, and transport protocols. But application logic is a different matter. For that, Oxy exposes a set of hook traits that are invoked throughout the lifecycle of a connection, not just individual requests. This lets applications control nearly every layer of the OSI model: packet I/O, tunneling protocols, HTTP versions, and even DNS resolution.
The hooks work as a form of dependency injection, similar in spirit to middleware patterns in Express or the Lua-based extension model we used in OpenResty/NGINX deployments. The key advantage over embedded scripting languages is Rust itself: hook implementations get full compile-time type checking, and objects passed between callbacks retain their mutability guarantees.
Writing hook implementations
From the application developer's perspective, the model is straightforward. An application implements a start method that returns a struct containing its hook implementations:
async fn start(
_settings: ServerSettings<(), ()>,
_parent_state: Metadata,
) -> anyhow::Result<Hooks<Self>> {
Ok(Hooks {
..Default::default()
})
}A basic egress hook, EgressHook::handle_connection, forwards every connection to the upstream requested by the client. Oxy invokes it before establishing the upstream connection. The default behavior would allow proxying to any address, but real deployments often need restrictions — for instance, preventing clients from reaching internal services through the proxy.
#[async_trait]
impl<Ext> EgressHook<Ext> for MyEgressHook
where
Ext: OxyExt,
{
async fn handle_connection(
&self,
upstream_addr: SocketAddr,
_egress_ctx: EgressConnectionContext<Ext>,
) -> ProxyResult<EgressDecision> {
Ok(EgressDecision::ExternalDirect(upstream_addr))
}
}
async fn start(
_settings: ServerSettings<(), ()>,
_parent_state: Metadata,
) -> anyhow::Result<Hooks<Self>> {
Ok(Hooks {
egress: Some(Arc::new(MyEgressHook)),
..Default::default()
})
}A more selective approach is to authorize connections based on client identity. Consider a Pre-Shared Key (PSK) scheme: if a client sends the header Proxy-Authorization: Preshared oxy-is-a-proxy, they are allowed to reach private addresses. This is easy to express with Oxy's Opaque Extensions, which attach arbitrary, fully typed context data to a connection. The data is mutable and shared across hooks: headers are read and the PSK state is recorded during the HTTP CONNECT phase, then the egress hook checks that same state later when the upstream connection is attempted.
#[derive(Default)]
struct AuthorizationResult {
can_access_private_cidrs: Arc<AtomicBool>,
}
#[async_trait]
impl<Ext> HttpRequestHook<Ext> for MyHttpHook
where
Ext: OxyExt<IngressConnectionContext = AuthorizationResult>,
{
async fn handle_proxy_connect_request(
self: Arc<Self>,
connect_req_head: &Parts,
req_ctx: RequestContext<Ext>,
) -> ConnectDirective {
const PSK_HEADER: &str = "Preshared oxy-is-a-proxy";
// Grab the authorization header and update
// the ingress_ctx if the preshared key matches.
if let Some(authorization_header) =
connect_req_head.headers.get("Proxy-Authorization") {
if authorization_header.to_str().unwrap() == PSK_HEADER {
req_ctx
.ingress_ctx()
.ext()
.can_access_private_cidrs
.store(true, Ordering::SeqCst);
}
}
ConnectDirective::Allow
}
}With the extension data in place, any hook in the pipeline can access it. The egress callback can be updated to check whether a valid PSK was presented before allowing the connection to proceed:
#[async_trait]
impl<Ext> EgressHook<Ext> for MyEgressHook
where
Ext: OxyExt<IngressConnectionContext = AuthorizationResult>,
{
async fn handle_connection(
&self,
upstream_addr: SocketAddr,
egress_ctx: EgressConnectionContext<Ext>,
) -> ProxyResult<EgressDecision> {
if self.private_cidrs.find(upstream_addr).is_some() {
if !egress_ctx
.ingress_ctx()
.ext()
.can_access_private_cidrs
.load(Ordering::SeqCst)
{
return Ok(EgressDecision::Block);
}
}
Ok(EgressDecision::ExternalDirect(upstream_addr))
}
}This pattern scales well beyond toy examples. A practical case is implementing the Proxy-Status header from RFC 9209, which requires including the upstream IP address chosen on the client's behalf. Two pre-existing callbacks and a small piece of state are enough: EgressHook::handle_connection_established stores the upstream address, then HttpRequestHook::handle_proxy_connect_response reads it back to set the CONNECT response header.
#[derive(Default)]
struct ConnectProxyConnectionContext {
upstream_addr: OnceCell<SocketAddr>,
}
#[async_trait]
impl<Ext> EgressHook<Ext> for MyEgressHook
where
Ext: OxyExt<IngressConnectionContext = ConnectProxyConnectionContext>,
{
fn handle_connection_established(
&self,
upstream_addr: SocketAddr,
egress_ctx: EgressConnectionContext<Ext>,
) {
egress_ctx
.ingress_ctx()
.ext()
.upstream_addr
.set(upstream_addr);
}
}
#[async_trait]
impl<Ext> HttpRequestHook<Ext> for MyHttpRequestHook
where
Ext: OxyExt<IngressConnectionContext = ConnectProxyConnectionContext>,
{
async fn handle_proxy_connect_response(
self: Arc<Self>,
mut res: Response<OxyBody>,
req_ctx: RequestContext<Ext>,
) -> ProxyConnectResponseHandlingOutcome {
let ingress = req_ctx.ingress_ctx();
let ingress_ext = ingress.ext();
if let Some(upstream_addr) = ingress_ext.upstream_addr.get() {
res.headers_mut().insert(
"Proxy-Status",
HeaderValue::from_str(&format!("next-hop=\"{upstream_addr}\"")).unwrap(),
);
}
res.into()
}
}Beyond HTTP CONNECT
These examples walk through part of the HTTP CONNECT pipeline, but that is only one path through the framework. Many Oxy applications do not terminate L7 traffic at all — they operate on raw connections or tunneling protocols. The hook traits cover these lower-level layers as well, which we'll examine in the implementation details ahead.
How hooks shape a proxy framework
Oxy is built to serve multiple teams with divergent needs, which calls for a pragmatic extensibility model. Hooks combined with Opaque Extensions give applications nearly unlimited customization through a clean, strongly typed interface. The implementation is intentionally straightforward — hook callbacks are invoked throughout the codebase:
if let Some(ref hook) = self.hook {
hook.handle_connection_established(upstream_addr, &egress_ctx)
.await;
}When a user-provided hook exists, it runs. Some hooks act as pure events, like handle_connection_established, while others return values that Oxy matches on for control flow, such as handle_connection. If no callback is implemented, the default trait implementation takes over, and if the hook itself isn't implemented, Oxy's core logic simply runs its default behavior. These layers of defaults keep the minimal example working out of the box.
Hooks solve integration of app logic into the framework, but custom state also needs to travel alongside. Oxy manages this state and passes it to every hook invocation. Because that state is generic over the application-defined type, the design gets more interesting.
Erasing types behind the public API
Each team using Oxy brings unique business logic, so one team's changes shouldn't force refactoring cascades on others. Context fields are user-defined types, which might suggest heavy generic use. Oxy instead presents a generic interface to application developers while erasing the type internally. Keeping generics out of internal code makes adding new extension types painless.
This approach relies on the Any trait. Internally, the framework treats state as an opaque blob; when crossing the public API boundary, the wrapped Any object is downcast to the user's concrete type. The public API requires the user type to implement Default, so Oxy fully owns instance creation and management. Users mutate state through interior mutability, typically with atomics and locks. Similar extension mechanisms exist in crates like reqwest_middleware, tracing, and http.
Productivity of Oxy application developers is a core concern, and the many injection points let users layer features without wrestling with unrelated proxy logic. Sensible defaults balance customizability against complexity. Not every callback fires for every packet — an L3-only application sees a different set than an L7 one. The set is also adjustable: Oxy's design allows connections to be upgraded or downgraded, changing which callbacks get invoked.
The ingress phase hosts hooks that control L3 upgrades and decapsulation of specific L4 protocols. For L3 IP tunnels, IpFlowHook::handle_flow lets applications drop, upgrade, or redirect flows, while IpFlowHook::handle_packet offers the same control per packet, including modification of the byte array in transit.
In the H2 Proxy Protocol example from the diagram, once Oxy accepts the connection it fires ProxyProtocolConnectionHook::handle_connection with the parsed header, letting applications process any TLVs of interest. This pattern recurs throughout: Oxy does the heavy lifting, then hands useful information to the application. L4 connections then pass through IngressHook, which includes the IngressHook::handle_connection callback from the opening example — applications can Allow or Block a connection as it enters. The counterpart IngressHook::handle_connection_close reports ingress statistics like loss, retransmissions, and bytes transferred.
The transformation phase reveals more powerful hooks. TunnelHook::should_intercept_https receives the SNI along with the standard connection context, allowing hostname-based HTTPS interception configuration with custom context data such as ACLs. By default Oxy splices ingress and egress sockets, but TunnelHook::get_app_tunnel_pipeline hands the two sockets directly to applications for complete tunneling control. For L7 firewall builders, HttpRequestHookPipeline offers handle_request and handle_response — both with a high-level interface for streaming rewrites or scanning of HTTP bodies.
The EgressHook carries the most callbacks, including the most potent ones. When hostnames are involved, DNS resolution must happen somewhere. Oxy can simply use application-specified nameservers, or EgressHook::handle_upstream_ips lets applications mutate resolved IP addresses before the connection is made. For absolute control, EgressHook::dns_resolve_override takes a hostname and expects a Vec<IpAddr> in return.
Like its ingress counterpart, EgressHook::handle_connection governs connection egress, but with more options than just Allow or Block: applications can send traffic externally, internally within Cloudflare, or downgrade to IP packets. Most applications defer connection establishment to the framework, but override callbacks such as tcp_connect_override and udp_connect_override exist for those who need them — mainly the egress service, though available to any Oxy application.
One of the newest additions is AppLifecycleHook, which should see far fewer invocations than the network hooks. Its AppLifecycleHook::state_for_restart callback runs during graceful shutdown, giving applications a chance to serialize state for the child process. Graceful restarts have their own nuances, but this hook cleanly solves passing application state between releases.
Evolving the hook set
Oxy currently has around 64 public-facing hooks, with more added regularly. The diagram above is largely accurate at the time of writing, but if a team needs a hook and a sensible default exists, it might as well join the framework. A major driver of the hook architecture is that teams can independently develop the hooks they need, keeping business logic outside the core so others can build on that work.
Discoverability is a known friction point, especially for application developers tracing when and where specific callbacks fire. Understanding the invocation order is harder still, since many hooks materially alter control flow, and a change in Oxy could shift application semantics. To address this, the team is experimenting with recording hook execution orders during integration tests, possibly via a proc-macro or compiler tooling.
The hooks in Oxy are, at heart, dependency injection — but applied at every layer of the networking stack, from IP packets and tunnels up to proxied UDP streams over QUIC. In the earlier example, two hooks and a few lines of code produced a forward proxy with metrics, tracing, graceful restarts, and more. This shared-core-plus-hooks approach has proven effective across Cloudflare, powering everything from iCloud Private Relay to Cloudflare Zero Trust, with generic capabilities available to all teams at little to no cost when unused.
