Why Performance Scores Differ Across Tools
Running a performance check is straightforward. Lighthouse in Chrome DevTools offers free testing and produces polished reports, making it the go-to option for many developers. But if two tools appear to use the same underlying technology, why do their scores sometimes disagree so dramatically?
The short answer: different tools make different assumptions using different data. Whether you're using Lighthouse, PageSpeed Insights (PSI), or commercial services, each tool evaluates performance through its own lens. Let's break down the common questions that explain these divergences.
What PageSpeed Insights Means by "Real-User Experience Data"
When PSI references real-user data, it's talking about real-user monitoring (RUM). There are two types, and they serve distinct purposes.
Chrome User Experience Report (CrUX)
PSI evaluates your core web vitals against the Chrome User Experience Report (CrUX)—anonymized data from Chrome users who have consented to share it. This data matters because core web vitals factor into Google's search ranking.
Google reports on the 75th percentile of users in CrUX data. This approach represents the majority of users while minimizing the impact of outlier experiences. But CrUX has limitations: the data refreshes every 28 days, so it isn't real-time. For historical analysis, you'd need the CrUX History API or BigQuery.
Full Real-User Monitoring
The second type of RUM is collected directly by site owners via an analytics snippet. Unlike CrUX, full RUM captures data from all browsers, not just Chrome, and does so continuously. There's no 28-day wait to see the impact of a change.
The difference in data sources alone explains why scores vary: CrUX-based results give a useful high-level view, but full RUM provides more immediate feedback and greater detail about individual user experiences.
Lighthouse Uses Lab Data Instead
Lighthouse doesn't draw on RUM data at all. It relies on lab data, which comes in two types.
Observed Data
Observed data measures performance as it happens in a test environment with predefined conditions. You might apply throttling to simulate a slower connection—much like deciding race conditions in advance rather than adapting to a live track.
Simulated Data
Simulated data works differently. Lighthouse first collects data on a fast connection, then estimates how the page would perform on a slower one. This approach is significantly faster than actually running tests under throttled conditions, which is why Lighthouse uses it by default—and why PSI, being powered by Lighthouse, does too.
How the Two Types Appear in Lighthouse
While simulated data is the default, Lighthouse can also collect observed data. Observed data tends to be more reliable since it doesn't depend on incomplete simulations of Chrome internals. Its accuracy hinges on test setup: throttling applied at the operating system level produces metrics closer to real user experiences, while DevTools throttling is easier to configure but less accurate.
Why Lab Data Has Limits
Lab data examines a single experience in a fixed environment that may not match what your typical user encounters. Real-user monitoring tells you about actual experiences; lab data can't do that on its own.
The trade-off is depth: lab reports include detailed analysis and improvement recommendations, while CrUX data only reports metric values without guidance on how to fix them.
Why Lighthouse LCP Looks Worse Than Real-User Data
The gap between Lighthouse and PSI scores for Largest Contentful Paint (LCP) becomes clearer once you understand the data types involved.
CrUX reports the 75th percentile of real users, which naturally filters out most outliers. As Chrome's Brian McQuade explains:
By using the 75th percentile, we know that most visits to the site (3 of 4) experienced the target level of performance or better. Additionally, the 75th percentile value is less likely to be affected by outliers.
Lighthouse's simulated data, on the other hand, doesn't account for real users or discard outliers. Heavy throttling applied by default in Lighthouse effectively represents a slow outlier experience—the very kind of data CrUX might exclude. That's why you'll often see a worse LCP score from Lighthouse than from PSI, even though both use the same core tooling.
Why Lighthouse Often Reports a Better CLS Than Real Users See
Cumulative Layout Shift (CLS) measures the visible stability of a page as it loads. If you have ever started reading an article, scrolled down, and then had the text jump because a late-loading image or ad shifted the layout, you have felt CLS firsthand.
The key difference between lab and field data for CLS comes down to interaction. Real users can and do scroll and click before a page has fully finished loading. Layout shifts frequently occur further down the page, so those shifts are far more likely to be observed by someone who has already scrolled. CrUX data captures this because it comes from real browsing sessions where such interactions happen.
Lighthouse, however, simulates a patient visitor. It waits for the full page load and never scrolls, clicks, hovers, or interacts in any way. Because of that, it is far less likely to trigger the shifts that occur only after user interaction. This is why your PSI report—which pulls from CrUX—often shows a worse CLS than a local Lighthouse run. It is not that PSI is stricter, only that actual users are more representative of how a page is really used.
Why INP Is Missing From Lighthouse Reports
Interaction to Next Paint (INP) is entirely about how quickly a page responds to user input. Since Lighthouse does not interact with the page during its simulated load, it has no real interactions to measure. INP is based on the latency of all interactions on a page, with the slowest (or near-slowest) response defining the metric. For example, if a visitor clicks an accordion and the content takes longer to render than any other interaction, that click determines the INP score.
When INP officially becomes a core web vital in March 2024, you may notice it does not appear in Lighthouse reports. That absence is expected: lab tools without scripted interaction are not capable of producing a meaningful INP value.
Note: Lighthouse does support scripted user flows, including in DevTools, but that is beyond the scope of this discussion.
Why TTFB Can Look Worse for Real Users
Time to First Byte (TTFB) measures the delay between establishing a connection and receiving the first byte of the response. Although TTFB is not a core web vital, it matters because it precedes all other metrics—nothing can render until the server responds.
TTFB is essentially a measure of server responsiveness and a signal for how quickly users can navigate. But the way lab and field tools measure it differs significantly, which leads to different results.
As Jeremy Wagner and Barry Pollard point out, TTFB thresholds are a rough guide. A low TTFB is critical for getting markup to the client quickly, but if that markup is just a shell that requires JavaScript to populate meaningful content, then a low TTFB becomes even more important so client-side rendering can begin sooner. The thresholds should always be weighed against how your site actually serves its core content.
If your real-user TTFB is higher than Lighthouse’s lab number, the cause is often environmental. A field user might hit a redirect from a shortened URL before reaching your server, connect from a location far from your origin without a CDN in place, or your tests may have benefited from cached responses. The discrepancy usually comes down to user location and your caching/edge setup.
Which Tool Should You Trust for Core Web Vitals?
The reason different tools disagree is not that one is broken—it is that every measurement is a snapshot with its own variables. Test location, CPU speed, Chrome version, and whether data is observed or simulated all shape the result. There is no singular correct value for a metric.
When using lab data, prefer tools that use observed data over fully simulated approaches. But even high-quality lab tools will differ from each other because no two tests run in the exact same environment. Lab data is best used to identify optimizations and to track changes over time with a consistent setup.
For understanding how real visitors experience your site, the 28-day Google CrUX data is the gold standard for SEO. It does have gaps, however. If you have shipped performance fixes in the last few weeks, CrUX will not reflect them yet. Additionally, Google does not report CrUX data for some high-traffic pages when visitors are not signed in to a Google account.
Building your own real-user monitoring (RUM) can close those gaps, but your numbers still will not match CrUX exactly. Custom RUM tracks visitors using browsers other than Chrome and includes users who have disabled Chrome analytics reporting, which expands the pool in ways Google does not.
Finally, keep in mind that Google’s thresholds are based on the fastest 75% of experiences. That does not mean the 75th percentile number is the only one that matters. Even when your core web vitals pass, a quarter of your visitors may still be having a distinctly slow experience.
Reading the Discrepancies
Performance reports are not wrong just because they disagree. The differences between lab data, CrUX, and full RUM are the most informative part of the analysis. A CLS score that is much better in Lighthouse than in the field is not a flattery—it is a signal that real users are interacting with your page in ways that cause layout shifts, and that is where you should focus your optimization effort.
