Measuring network quality to better understand the end-user experience

What a Speed Test Actually Measures

Most speed tests return four numbers: bandwidth, latency, jitter, and packet loss. Despite two decades of near-unchanged methodology, the relationship between those figures and day-to-day internet use remains murky. The gap matters more as video calls, gaming, and streaming dominate traffic, and the IETF has taken notice.

At the network layer, only three properties are directly observable: bandwidth (throughput), packet loss, and latency (round-trip time). They are interdependent, and throughput is especially sensitive to the other two. Loss and latency govern when your device sends packets; either in excess and bandwidth collapses. The numbers alone do not tell the whole story, but they are all we have to work with.

Bandwidth

Bandwidth is the maximum capacity of a link. The highway analogy holds: more lanes (bandwidth) let more cars (data) pass. ISPs historically frame it as “speed” since it dictates how quickly large files download.

Packet Loss

Packet loss occurs when packets sent from a source never reach their destination. The impact is severe for real-time and interactive applications, because missing data means the receiver must piece together an incomplete message or wait for retransmission. Loss at low levels is normal and even necessary; too much is debilitating.

Latency and Jitter

Latency is the time for a packet to travel from point A to point B, fundamentally limited by the speed of light through cables. But idle latency differs from latency under load. Routers and switches queue packets by design, and when queues grow too large — a phenomenon known as bufferbloat — the connection feels slower than its raw potential. AIM tests measure both idle latency (what your connection could be) and loaded latency (what it typically is during real use).

Jitter is variance in latency. Zero jitter means a stable commute; high jitter means arrival times swing unpredictably. For real-time scenarios like voice calls, that variance breaks the experience even when average latency looks fine.

Understanding what these numbers mean is only half the battle. They reveal that something is wrong but not where.

Local Network or ISP Problem?

A speed test connects not just to your ISP but also to your local network. If packet loss and jitter are high, the culprit could be your WiFi. Radio signals weaken through brick, plaster, and concrete, and distance from the access point matters. Mesh systems like Nest WiFi and Eero already run periodic speed tests from their main node specifically to catch such issues. Users need clear guidance up front — like “move closer to the router” — to rule out their own setup before calling the ISP.

But individual fixes are not enough. When results are aggregated and visible to network operators, problems in the last mile can be detected proactively, before user complaints and expensive truck rolls. That is the core purpose of AIM.

Introducing AIM

To address the three major challenges in network measurement — accurately translating quality into user experience, isolating where problems occur, and contextualizing individual results against the broader population — Cloudflare has launched the Aggregated Internet Measurement (AIM) initiative. AIM is an open format for expressing internet quality around specific use cases that demand particular network characteristics, while preserving the raw data engineers need for analysis.

Cloudflare is partnering with Measurement Lab on the project, storing results in a publicly accessible repository for independent analysis. The speed test source code and AIM score calculations are now open-sourced as well.

For a concrete example: a test revealing high packet loss and jitter points toward wireless interference. That information, delivered to the user as actionable tips, solves the problem before anyone picks up a phone. Network operators, seeing the same data in aggregate, can spot neighborhood-level issues and dispatch help only when necessary. Both sides gain: users get concise guidance on what their connection can and cannot handle, and providers save on support costs by addressing real infrastructure faults rather than user-configuration issues.

Making sense of raw speed test numbers

Raw speed test results can be difficult to interpret without networking expertise. Consider a typical office scenario: an employee working in the room farthest from the wireless access point experiences constant dropped video calls. A standard speed test from that location might report high jitter and packet loss — metrics a network engineer would immediately recognize as symptoms of poor signal strength. But for most users, those numbers are just abstract values that don't clearly point to an obvious fix.

That ambiguity often leads to unnecessary calls to an ISP or networking consultants when the real solution is as simple as relocating the access point. To address this, Cloudflare's Aggregated Internet Measurement (AIM) approach translates raw performance metrics into qualitative assessments based on what you're actually trying to do online.

In the office scenario described above, a speed test from the distant office produces results like this:

Scenario-specific quality scores

AIM computes separate quality scores for three scenarios: Streaming, Gaming, and WebChat/RTC. Each metric — download bandwidth, unloaded latency, loaded latency, packet loss, and jitter — is weighted differently depending on which application experience you're evaluating:

• Streaming: download bandwidth + unloaded latency + packet loss + (loaded latency − unloaded latency difference)
• Gaming: packet loss + unloaded latency + (loaded latency − unloaded latency difference)
• RTC/video: packet loss + jitter + unloaded latency + (loaded latency − unloaded latency difference)

Each scenario is assigned a point value rubric that converts test measurements into a single score, placing your connection on a scale from "Bad" to "Great" for that specific use case. The weighting isn't meant to be static — the intent is for the rubric to evolve as application developers, network operators, and the broader Internet community learn more about how performance characteristics affect real-world application experience. Publishing the underlying data to M-Lab is part of that collaborative refinement process.

Applying the scoring to the distant-office speed test example yields these AIM results:

Instead of saying "your bandwidth is X and your jitter is Y," AIM can report that the connection is "okay for Netflix, but poor for gaming, and only average for Zoom." In the office example, moving the access point to a central location turned those scores around:

These scenario scores are available today as a Network Quality Score on the Cloudflare speed test:

Measuring under working conditions

A persistent problem with traditional speed tests is that they run against dedicated test servers that aren't the same servers you actually interact with day to day. Network paths optimized to serve speed tests may differ significantly from those that carry your normal application traffic, meaning the test doesn't measure the path you actually depend on.

Real-world usage also involves reusing established connections — browsers and applications deliberately avoid the latency of setting up new encrypted connections for every request. AIM addresses this by running tests the way applications would, under what Cloudflare calls "working conditions," while your network is experiencing real concurrent load:

The measurements also collect results against endpoints that users actually frequent — Cloudflare and other common websites — rather than isolated test infrastructure, providing a clearer picture of typical network performance.

Public data and collaboration

AIM data is now publicly available through a partnership with Measurement Lab (M-Lab), an open Internet measurement repository that provides free access to network data for researchers, policy makers, journalists, and digital inclusion advocates. Both M-Lab and Cloudflare calculate AIM scores from their respective speed tests and pool them into a shared database. M-Lab's existing Network Diagnostic Test (NDT), which powers the "speed test" result shown in Google searches, will also contribute to the aggregated dataset.

A sample of Cloudflare-only data for Tokyo over the first week of October illustrates what the dataset reveals:

Of 5,814 speed tests in that city, 50.7% of users saw good streaming quality while 48.2% were rated average. Gaming proved harder — 39% of Tokyo users had a poor gaming experience — while most registered an average-to-decent RTC experience. Cross-city comparisons show generally solid video streaming in most locations (Mumbai lags slightly), variable gaming performance due to the strong influence of latency, and mostly average RTC quality everywhere sampled.

Open source and integration

Cloudflare has also open-sourced its speed test client, making the measurement tool embeddable as a JavaScript application for any website. This allows applications to run network quality assessments and calculate AIM scores without requiring users to navigate to a separate speed test page. The open-source repository documents how AIM scores are calculated and how those calculations evolve over time.

For ISPs, the aggregated and anonymized data can translate user complaints into actionable network engineering work. Policy makers and researchers can use it to understand what communities are experiencing and advocate for better connectivity. Other speed test operators interested in contributing data are invited to participate, as are developers of applications needing built-in network quality measurement.