Driving Content Delivery Efficiency Through Classifying Cache Misses

Classifying What a CDN Cache Miss Actually Means

Open Connect, Netflix’s dedicated content delivery network, is built around a simple premise: keep content as close to the member as possible by placing Open Connect Appliances (OCAs) at internet exchange points and ISP sites. That localization cuts latency, reduces buffering, and improves video start times. But the CDN only works well if the right bytes are on the right servers at the right time. When they are not, Open Connect logs what it calls a cache miss.

Netflix’s definition of a cache miss is worth unpacking, because it differs from the traditional proxy-cache view. The company measures misses from the client’s perspective, not the server’s. A miss is recorded when a member’s stream is not served from the most proximal OCA site for that client’s IP address — regardless of whether the OCA itself was functioning normally or held the content. The key metric is whether delivery happened from the best possible location for the end user.

How a Playback Request Is Routed

To understand the miss classification, it helps to trace a playback request. Every OCA periodically reports its health, learned BGP routes, and the list of files it stores to the Cache Control Service (CCS). When a member presses play, the request goes to Netflix’s AWS-based Playback Apps service, which determines which files correspond to the requested title. Playback Apps then asks the Steering Service to pick an OCA.

The Steering Service consults CCS data plus client information such as geo-location, and returns a rank-ordered list of OCA URLs to the client device. That ranking is called the “proximity rank,” and it is computed purely from network proximity: the BGP prefixes announced by ISP partners. The client connects to the top-ranked URL and starts streaming.

A cache miss is logged whenever the client streams from any site other than the first one on the proximity list. Each miss is attributed to one of three logical categories, each of which drives a different remediation strategy.

The Three Miss Categories

Content Miss: The files were not present on any OCA at the local site. This points to forecasting and prepositioning problems: how accurately Netflix predicts content popularity, how quickly it can push that content out to OCAs, how well OCA hardware is designed for the expected workload, and whether storage capacity at a given location is sufficient.

Health Miss: The local OCA hardware was saturated and could not handle additional traffic, so the client was directed to another site with available capacity. Each OCA runs a control loop that watches bottleneck metrics such as CPU and disk usage to assess whether it can serve more streams. Health misses inform load-balancing across heterogeneous hardware, provisioning enough copies of popular content to spread the load, and prepositioning content onto hardware that can actually serve it efficiently.

A third category exists but is described only as a logical grouping in the source material; the two detailed above are the ones discussed in depth.

Logging the Misses in Real Time

Computing cache-miss metrics requires joining two log streams for every playback request.

The first is the Steering Playback Manifest Log. Inside the Steering Service, Netflix logs the full proximity rank for each client request, along with the logical decisions and filters applied to that ranking given the state of the system at that moment. Because the logs preserve the decision inputs, they can be replayed to simulate hypothetical scenarios — for example, what would happen to second-rank sites if every site in the first rank went down.

The second is the OCA Server Log. Once a client connects and streams, the OCA logs session data: which files were served and how many bytes went out. These logs are consolidated to identify which OCA actually served a given client, and how much content was streamed.

Joining these two logs per playback request yields miss metrics in bytes and hours streamed, aggregated by OCA, movie, file, encode type, country, or any other dimension needed.

Architecture for Near-Real-Time Metrics

The system that computes these metrics is built around Kafka for event streaming and a multi-stage pipeline designed to minimize cross-region data movement.

Log emission. Both log types are emitted to Kafka clusters in the AWS region where they are generated. The Steering Service logs its manifest at playback time; OCAs log their stream data after serving bytes. Kafka was chosen for high throughput, low latency, and reliability.

Consolidation and enrichment. Because a single playback can produce logs in different AWS regions — the steering manifest in one region’s Kafka cluster, the OCA logs in another — the first processing step consolidates all logs into a single region with one cross-region transfer. This avoids complex many-to-many streaming joins and redundant data copies. During streaming joins, the logs are enriched with metadata from slow-changing dimension tables to capture OCA details and content attributes.

Windowed join and calculation. The enriched logs are merged with a streaming window-based join. The final computation checks whether the client streamed from an OCA in the first site of the proximity rank or from a higher-ranked site. Streaming from a higher rank triggers the cache-miss record.

The result is a real-time view of where content delivery is falling short of the ideal, split into categories that each point toward a distinct fix: improve popularity prediction and content placement for content misses, or improve load balancing and capacity planning for health misses.

Turning Cache Miss Logs into Actionable Metrics

A key benefit of the unified logging pipeline is the ability to replay steering decisions offline. Using logs from both the steering playback manifest and the OCA, we can simulate production conditions under varying parameters, testing new features and hypothetical scenarios without affecting live Netflix traffic. This requires the underlying Steering Service to be modular so that each logical step can append its decision and rationale to a shared state that is logged without adding noticeable latency to playback requests. The aggregated data also needs to support near-real-time monitoring.

The joined data model captures the decisions and byte counts needed to evaluate cache misses. Table 1 lists the main fields after merging the manifest and server logs.

Table 1: Unified Data Model after joining steering playback manifest and OCA server logs.

Computing and Categorizing Cache Misses

A cache miss is identified for a given play request when the client streams from an OCA that is not ranked as the most proximal. The data model links each file to routable OCAs and their sites with a zero-based proximity rank, where rank zero indicates the preferred site. If the client connects at rank zero, no miss occurred; higher ranks represent a miss. Summing all bytes and hours streamed from non-preferred sites gives the total missed opportunity.

For each file needed during a streaming session, the bytes streamed by the client fall into one of three categories:

• No Miss: proximity rank is zero, so bytes streamed from the optimal OCA.
• Health Miss (“H”): miss caused by high utilization at the OCA.
• Content Miss (“C”): miss caused by the content not being available locally.

These labels come directly from the steering playback manifest logs, which record the reason an OCA was not selected for playback.

Tracking Efficiency with a Content Shed Ratio

The team monitors total streaming traffic alongside the share of traffic served from less preferred locations. The ratio of content shed traffic to total streamed traffic yields a metric called the content shed ratio:

content shed ratio = content shed traffic total streamed traffic

This ratio is a primary signal for operational feedback. Content shedding can stem from multiple causes, so having clear indicators of when it occurs and automated remediation steps—such as dynamically pushing mispredicted content onto OCAs—helps maintain streaming quality. Cases needing manual intervention are treated as opportunities to improve system comprehensiveness.

Ongoing Refinement of Miss Metrics

Reducing cache misses directly improves quality of experience by keeping members on the most proximal servers. The team uses granular miss metrics and alerts to detect when members are served from suboptimal locations across a global footprint of tens of thousands of servers. As new streaming formats like Live and Ads introduce different access patterns, the classification logic continues to evolve to handle these emerging traffic shapes.