Feedback wanted: An experimental responsiveness metric | Blog

An experimental responsiveness metric: measuring true interaction latency

The Chrome Speed Metrics Team is continuing its work on a new responsiveness metric that captures the full end-to-end latency of user interactions. Following up on earlier proposals, the team has settled on a measurement approach and is now evaluating several ways to aggregate per-interaction data into a single page-level score. Developer feedback is requested to help determine which aggregation strategy best reflects real-world input responsiveness.

From input delay to full event duration

The existing First Input Delay (FID) metric only covers one part of the latency story: the gap between a user's input and the moment event handlers can run. The proposed metric widens that scope to measure the complete duration of an interaction, from the initial input through the execution of all event handlers until the next frame is painted.

Measurement will be organized around interactions, which group events dispatched as part of the same logical gesture (for instance, pointerdown, click, and pointerup together). To combine multiple event durations into a single interaction latency, two options are under consideration:

• Maximum event duration: The interaction's latency is the largest of any individual event duration in the group.
• Total event duration: All event durations are summed, subtracting any overlap between them.

The distinction matters in practice. In a key press interaction, keydown and keyup events may overlap, so the interaction latency could be either the larger of the two or the combined sum minus the shared time. More real-world data is needed before a final definition is chosen.

Aggregating interactions into a page score

With a latency definition in place, the challenge becomes summarizing the responsiveness of a whole page visit, which may contain many interactions. Chrome is currently collecting field data on several candidate aggregation strategies, and site owners are being asked which approach most faithfully represents the interaction patterns on their pages.

To illustrate the options, consider a page visit with four interactions having the following latencies:

Worst interaction latency

The simplest strategy is to take the single largest interaction latency on the page. For the example above, that score would be 120 ms.

Over-budget strategies

User experience research indicates that latencies below a certain threshold may not be perceived negatively by users. Based on that work, established thresholds exist per event type:

The over-budget strategies consider only the amount by which each interaction exceeds its threshold. The example interactions would produce the following over-budget values:

Worst interaction latency over budget

Take the largest single over-budget interaction:

max(20, 0, 10, 30) = 30 ms

Total interaction latency over budget

Sum all over-budget amounts:

(20 + 0 + 10 + 30) = 60 ms

Average interaction latency over budget

Divide the total over-budget amount by the number of interactions:

(20 + 0 + 10 + 30) / 4 = 15 ms

High quantile approximations

Using the strict maximum can penalize pages with many interactions, since more opportunities mean more chances for rare outliers. As an alternative, two high-quantile approximation strategies are being explored:

• Option 1: Track the largest and second-largest over-budget interactions. After each group of 50 new interactions, discard the largest from the previous group and add the largest from the current group. The remaining largest value becomes the score.
• Option 2: Keep the 10 largest over-budget interactions. Given N total interactions, the score is the (N / 50 + 1)-th largest value, capped at the 10th value for pages exceeding 500 interactions.

Measuring the strategies in JavaScript

The first three aggregation options (worst interaction latency, worst over budget, and total over budget) can be computed with a relatively small amount of page instrumentation. Some constraints remain: the total number of interactions per page is not yet measurable from JavaScript, so the average-based and high-quantile strategies are not computable without additional support.

const interactionMap = new Map();

let worstLatency = 0;
let worstLatencyOverBudget = 0;
let totalLatencyOverBudget = 0;

new PerformanceObserver((entries) => {
  for (const entry of entries.getEntries()) {
    // Ignore entries without an interaction ID.
    if (entry.interactionId > 0) {
      // Get the interaction for this entry, or create one if it doesn't exist.
      let interaction = interactionMap.get(entry.interactionId);
      if (!interaction) {
        interaction = {latency: 0, entries: []};
        interactionMap.set(entry.interactionId, interaction);
      }
      interaction.entries.push(entry);

      const latency = Math.max(entry.duration, interaction.latency);
      worstLatency = Math.max(worstLatency, latency);

      const budget = entry.name.includes('key') ? 50 : 100;
      const latencyOverBudget = Math.max(latency - budget, 0);
      worstLatencyOverBudget = Math.max(
        latencyOverBudget,
        worstLatencyOverBudget,
      );

      if (latencyOverBudget) {
        const oldLatencyOverBudget = Math.max(interaction.latency - budget, 0);
        totalLatencyOverBudget += latencyOverBudget - oldLatencyOverBudget;
      }

      // Set the latency on the interaction so future events can reference.
      interaction.latency = latency;

      // Log the updated metric values.
      console.log({
        worstLatency,
        worstLatencyOverBudget,
        totalLatencyOverBudget,
      });
    }
  }
  // Set the `durationThreshold` to 50 to capture keyboard interactions
  // that are over-budget (the default `durationThreshold` is 100).
}).observe({type: 'event', buffered: true, durationThreshold: 50});

Providing feedback

Developers are encouraged to test these experimental metrics on their own sites and report any issues or concerns. Feedback on the proposed approaches can be sent to the web-vitals-feedback Google group with "[Responsiveness Metrics]" in the subject line.