Performance measurements… and the people who love them

Why “fast” needs a definition before it needs a timer

“Performance” is a word we throw around as if it means one thing. It doesn’t. Latency is a step closer to measurable, but even that needs a precise frame: someone starts something, and it finishes — the elapsed time in between is the latency. That definition is deliberately simple, and it yields a useful insight: latency is always measured from a client’s perspective. At Cloudflare, we exist only on the server side, so every internal measurement of request latency is an estimation. DNS lookups, local TLS computation, and resource contention on the client’s machine all contribute to the user’s experience but only add noise when we’re evaluating our own performance.

Time to first byte (TTFB) sounds like the obvious answer — until you ask what it actually measures. The name implies the timer ends when the client receives the first byte of the response; it says nothing about when it starts. That ambiguity is why TTFB means different things on different platforms. We’ve written before about how TTFB “isn’t what it used to be” after the introduction of early hints in June 2022 changed one widely used browser definition. TTFB has real value, but to use it unambiguously we have to pick a definition that’s already in production. For us, that meant aligning with the terminology our customers use in Catchpoint, the global probe network they lean on to compare our performance with our competitors’.

This breakdown shows how Catchpoint computes the metrics our customers watch every day. Some are direct measurements; others, like TTFB, are sums of direct measurements — in this case, DNS, Connect, TLS, and Wait times. Browsers report the stages of a request similarly, though they use timestamps rather than durations, leaving gaps between phases for work the client’s machine does outside the request lifecycle. Line those timestamps up with Catchpoint’s phases and the two models align closely enough that we can adopt one vocabulary for internal dashboards, customer reports, and browser-based RUM data alike.

Aggregation hides the tail

Once we agree on what to measure, we face a second problem: nobody records raw samples anymore. Tools like Prometheus are built to store pre-aggregated data, not individual measurements, and for good reason — storing every sample at full fidelity would drown any system. Even if it didn’t, older data degrades in value; the most recent measurements are the most actionable. So every latency visualization you’ve ever seen depends on some aggregation step, usually windowed percentiles: median, p75, p90, and p99.

That internal dashboard shows a terrifying spike at 14:40 UTC — p99 jumps from 500 ms to 6500 ms, and p50 goes from 4.4 ms to 600 ms. Something clearly happened. But the chart is compressing the entire latency distribution into four percentile bands, and because we only look up to the 99th percentile, a full 1% of samples are invisible to us. This is where statistical modeling behaves like lossy compression: it simplifies a complex system well, but it also hides subtlety.

Consider www.cloudflare.com — roughly 30,000 requests per second, an Edge TTFB p99 of 500 ms. That means 300 req/s take longer than half a second just to reach the first byte from our edge, and we have no idea how much longer. Plot the p100, the max, and the spike that looked so exceptional becomes background noise.

Seen that way, the incident at 14:40 UTC is no longer a dramatic anomaly — it’s one point on a chart that was already messy. The percentiles we picked told us something was wrong, but the same statistics kept us from seeing how wrong things were the rest of the time. The measurement tools we trust can lie to us, not by fabricating data, but by compressing it until we only see the shape we expect.

Why Median Latency Misleads

Focusing on the median request latency feels intuitive. Surely, the experience of the typical user matters more than the experience of one in a hundred? That logic holds for people, but not for requests. A user rarely interacts with a website one request at a time.

Consider that loading www.cloudflare.com triggers more than 70 requests. That number is modest compared to www.amazon.com, which requires over 400 requests before the page is fully useful. While not all requests must finish before a page is interactive, the sheer volume changes the probability math governing the user's perceived latency.

Combining probabilities requires two simple principles:

• The probability of two independent events occurring is the product of their individual probabilities: P(X ∩ Y) = P(X) × P(Y).
• The probability of an event at the Xth percentile is X%: P(pX) = X%.

Let P(pX_N) represent the probability that a visitor on a page requiring N requests avoids any latency above the Xth percentile. For a two-request page, the chance of avoiding an above-median latency for both requests is the probability of each request individually being below the median, multiplied together:

P(p50_2) = 50% × 50% = 25%

This generalizes neatly: P(pX_N) = X%^N. Calculating this for cloudflare.com's ~70 requests reveals why chasing the median is futile:

P(p50_70) = 50%^70 ≈ 0.000000000000000000001%

The chance that a single user completes the full page load without experiencing a single request above the median is effectively zero. Medians describe single requests, not people. The better question is: what request latency percentile actually characterizes what the median user will experience?

Setting the probability of the entire page load being below a given percentile to 50% and solving for X with 70 requests yields:

X = e^(ln(50%) / 70) ≈ 99%

The math works out elegantly. To capture the experience of the median visitor to cloudflare.com, you need the p99 request latency. Extend that logic: if you want to represent what 99% of users experience on that page, you must examine the 99.99th percentile.

Comparing Real-World Performance

Single-system monitoring gives absolute numbers for, say, p50 or p95. That is useful for alerting and internal dashboards. But external communication about performance often involves comparisons to other providers, and those comparisons are usually made by plotting request latency over time and visually assessing the gap.

The Limitations of Raw Comparison

Simple line charts of median latency miss the full picture. Using the concepts above, a more accurate comparison method is possible. We demonstrated how exposing the extreme percentile spectrum reveals details that the median hides. Combining that broader view with the probabilistic impact of multiple requests yields surprising conclusions.

The example data below compares the latency (defined as expected total time before the browser can begin rendering, consisting of TLS connection setup, connection, and wait phases) for the customer, measured through Cloudflare and through a competing provider. The dataset contains 90,000 samples for each provider.

A CDF plot is one visualization that exposes the entire percentile range simultaneously.

Reading this chart is straightforward: to find the latency for a given percentile, move up to that percentile and then across to the curve. The provider whose curve sits farther left has the lower latency at that percentile. The competing provider shows lower latency by up to 30ms for nearly the entire percentile distribution. This advantage disappears at the very top of the chart, where a small segment of the Cloudflare curve sits above and to the left of the competitor's curve.

To inspect the precise difference throughout the range, a QQ-Plot (quantile-quantile plot) is better suited. This chart preserves the same information but plots the latency values of both providers at each quantile.

On this chart, the blue diagonal line marks where the providers have equal latency. Points below the line show percentiles where the competitor is faster; points above the line show percentiles where Cloudflare is faster. With almost all points below the line, the competitor dominates the majority of the request latency distribution. Only at the upper tail, roughly above the 99th percentile, does Cloudflare measure significantly faster.

That fact alone is not especially compelling. The view changes when the data is interpreted through the number of requests a page actually makes. Considering that this page requires over 180 requests to load fully, and recalculating with only half of those requests needed for usable page load, the effective comparison shifts dramatically.

The effective picture is far more balanced. The median latency of the two providers is nearly identical. The divergence appears in the extremes: a user's experience carries roughly an even chance of landing in cases where Cloudflare is substantially faster versus cases where the competitor is slightly faster. The impact of this shift in perspective is best shown by the expected value of latency for a single request versus the overall experience.

Measuring the user's real experience instead of isolated requests, Cloudflare's measured latency is 70 ms faster than the competitor. This advantage stems entirely from Cloudflare's focus on tail latency and reliability, but it remains completely invisible without examining a wide percentile range and accounting for how many requests constitute a real page load. To make this analysis more broadly accessible, Cloudflare plans to release these tools to customers later in 2025.