“Look, Ma, no probes!” — Characterizing CDNs’ latencies with passive measurement

Why direct measurement falls short

Service providers constantly need to assess their own performance, but comparing that performance against other CDNs presents a fundamental problem: the data required for such comparisons often sits outside a provider’s own network, and collecting certain measurements might intrude on user privacy. Two questions recur: how much better (or worse) is one CDN’s location strategy than another’s, and can we predict in advance where new data centers will yield the greatest improvement?

The second question demands more than raw measurement — it requires inference. To answer it meaningfully, network measurements must be distilled in a way that exposes the relationships among network structure, content distribution practices and routing policies, and how those factors jointly affect performance.

Active measurements are the standard tool for observing other networks, since no CDN shares its internal performance data with competitors for business and privacy reasons. Direct probing from Cloudflare’s own data centers to other CDNs, however, reveals little about actual client experience. The only useful active measurements must originate from third-party vantage points: ping probes from RIPE Atlas clients or JavaScript-based Real User Measurement (RUM) services embedded in pages worldwide.

Active measurements are valuable, but they carry inherent biases. RIPE Atlas probes cluster heavily in Europe and North America, leaving sparse coverage across Asia, Africa and South America. Moreover, the distribution of probes across autonomous systems (ASes) does not reflect real user populations — university networks, hosting providers and enterprises are overrepresented. RUM data suffers from a different problem: it cannot be validated by outside parties, an essential property of sound measurement. The disparity between two leading RUM platforms measuring the same CDN, in the same location and time period, is striking enough to call both datasets into question.

Active measurements are ultimately constrained by what they can directly observe. On their own, they do not yield predictive models for assessing how infrastructure or policy changes might affect performance. But when their biases are well understood, active measurements serve two purposes: they inform understanding, and they help validate models of the world. Both roles come into play in a passive approach to evaluating CDN latencies.

Inferring CDN RTTs from passive data

The passive method uses Cloudflare’s own RTT measurements as a proxy for estimating latency between clients and other CDNs. This technique estimates latency both where CDNs have deployed infrastructure and where they have not, revealing performance advantages of one CDN over another in particular regions. The underlying assumptions have been validated through large-scale traceroute and ping campaigns, and the method is designed to be reproducible by others.

Step 1: predicting anycast catchments

The first step is predicting which data center a given client IP will reach — the anycast catchment. This requires compiling the network footprint of each CDN whose performance we want to predict. The necessary datasets include:

• ISPs hosting off-net server caches of CDNs, identified using methodology from a study by Gigis et al.
• On-net city-level data centers from PeeringDB, CDN network maps and IP geolocation measurements.
• IXP connections for each CDN, cross-referenced with other member ASes, from PeeringDB, Euro-IX IXP-DB and Packet Clearing House.
• CDN interconnections to other ASes extracted from BGP data in RouteViews and RIPE RIS.

With this data, possible paths between a client AS and a CDN’s data centers can be computed, and anycast catchments inferred by reproducing Internet inter-domain routing policies, following techniques from recent papers by Zhang et al. and Sermpezis and Kotronis. For BGP-based anycast CDNs, the predicted catchment is the data center selected by routing policy. For DNS-based CDNs, the method first predicts latency to each data center and then assumes the operator routes to the lowest-latency path.

The challenge is that routing policies of individual ASes — both client networks and intermediate transit networks — are not fully observable. However, with Cloudflare’s extensive data center footprint, the IP address space can be partitioned into catchment regions, allowing routing decisions of client ASes visible to Cloudflare to be reverse engineered.

Consider an example: an ISP has two paths to a CDN — one over a transit provider and one through a direct IXP peering link — each terminating at a different data center. Because transit incurs exchange costs while IXP peering does not, routing policy should favor the IXP path.

Estimating CDN Latency from Passive Measurements

To predict the RTT between a client AS and a given CDN location, we turn to passive TCP RTT data gathered on Cloudflare's own infrastructure. For every data center, we compute the median TCP RTT for each IP /24 subnet that sends us HTTP requests. We then apply that measured RTT as an estimate for the latency between the same subnet and a colocated data center belonging to another CDN. The underlying assumption is that distance between endpoints dominates latency, so host-software differences matter less than the network path itself.

We intentionally use the median RTT as our estimator of expected client performance. The minimum RTT, by contrast, reflects how close clients are to a server, not what they will typically experience. This approach is comparable to prior path-prediction work by Madhyastha et al., which combined median RTT measurements with topology-informed path inference. That work reported 65% accuracy for arbitrary ASes. Our focus on CDNs makes the problem easier: most clients are within one AS hop of a CDN point of presence, as noted by Chiu et al. and by Singh and Gill. For RTT estimation, what matters is predicting which CDN data center a client IP will be routed to, not enumerating the intermediate hops.

A Worked Example

Consider an IP subnet belonging to AS3379, a Greek ISP. Cloudflare receives requests from that prefix at several data centers. Most traffic lands at the nearest location — Athens — but load balancing and service-tier routing can direct some requests to other sites.

Now suppose a different CDN, call it B, operates no infrastructure in Athens or Sofia but does have data centers in Milan, Frankfurt, and Amsterdam. Using topology and colocation data for B, we predict its anycast catchment for this prefix and determine that Frankfurt will serve AS3379. The median RTT we measured from that subnet to Frankfurt then becomes our estimate for the median latency between CDN B and that prefix.

The method relies on Cloudflare's breadth of network coverage. We currently hold passive measurements between 63,832 ASes — effectively every AS that hosts clients — and 300 cities across 115 countries where Cloudflare infrastructure sits. That footprint covers the vast majority of regions where other CDNs have deployed capacity.

Validating the Predictions

To test the accuracy of our estimates, we ran a global measurement campaign using 9,990 RIPE Atlas probes in 161 countries. For each CDN target, we picked an anycasted hostname and configured DNS resolution on the probes so that the returned IP corresponded to the probe's nearest data center.

We first checked our anycast catchment predictions, geolocating the destination IPs of each completed traceroute and comparing them to the data center we had predicted. Nearly 90% of the predicted data centers matched what we actually measured.

We then examined RTT prediction error. The histogram below shows the absolute difference between measured and predicted RTT in milliseconds across all data centers. Over half of our predictions came within 3 ms of the measured value, and nearly 95% were within 10 ms.

Applying the Methodology

In September 2021, we measured nine major CDNs, including Cloudflare, using this approach. Cloudflare showed the lowest median RTT across all observed clients, at approximately 10 ms.

Methodology Caveats

Because this is an estimation technique, millisecond-accurate measurements are out of reach. That precision is largely unattainable even with live user measurements: network conditions shift constantly, so RTT values vary between measurement runs.

This approach also cannot serve as a real-time network hygiene monitor for catching transient performance issues outside Cloudflare's network. Its value lies in characterizing the expected performance of our topology and connectivity. It allows us to run what-if analyses — for example, predicting how a new data center deployment, or the loss of connectivity to an ISP or IXP, would affect end-user latency.

A final caveat concerns geographic coverage. Cloudflare operates the most extensive network of data centers and IXPs among CDNs, but in some countries we have no presence where competitors do. In others, our infrastructure sits in partner facilities rather than carrier-neutral data centers, which can limit direct peering with regional ISPs. In those cases, BGP's cost-over-proximity routing may send client IPs to a data center outside their country. For roughly 7% of client /24 prefixes, we therefore lack a measured RTT between an IP and a within-country data center. We are addressing this gap with active traceroute measurements and plan to report on those results later.

Ongoing Work

Being able to predict and compare CDN performance opens the door to evaluating peering strategies and data center placement policies, and to spotting weaknesses in our anycast catchments and traffic engineering rules. Current efforts focus on quantifying how IXP peering affects end-to-end latency, and on identifying regional Internet ecosystems where open peering policies might actually increase latency. The eventual goal is to tailor infrastructure placement and control-plane choices to the topological character of each region, minimizing user-visible latency.