Lighthouse CI as a performance monitoring layer

Lighthouse CI is a suite of free tools that make it possible to run Lighthouse as part of a continuous integration pipeline. A single Lighthouse report captures a page's performance at one moment in time; Lighthouse CI adds the dimension of history, making it possible to trace how performance has shifted across commits and releases. That history can pinpoint the effect of a specific code change or enforce performance budgets before code is merged. Though performance is the most frequent use case, the same infrastructure also tracks SEO, accessibility, and other categories covered by Lighthouse.

The core of Lighthouse CI is its command line interface, a separate tool from the standard Lighthouse CLI. The CLI provides a set of commands — including autorun, which runs Lighthouse multiple times, selects the median report, and uploads it for storage. Behavior is configured through a lighthouserc.js file in the root of the repository, or by passing additional flags.

In practice, teams typically interact with Lighthouse CI in one of several ways:

  • Running it as a step within an existing CI pipeline
  • Using a GitHub Action that executes Lighthouse on every pull request and leaves a comment with results
  • Tracking performance over time through a dashboard provided by Lighthouse Server

All of these approaches sit on top of the Lighthouse CI CLI.

Alternative approaches include third-party performance monitoring services, which may be preferable if you'd rather outsource server management and test device infrastructure, or if you want built-in alerting (email or Slack, for example) without building those integrations yourself.

Running Lighthouse CI locally first

Before wiring Lighthouse CI into a provider like GitHub Actions, run it locally to verify your lighthouserc.js configuration behaves as expected. Start by installing the CLI:

npm install -g @lhci/cli

Lighthouse CI requires a lighthouserc.js file at the root of your repo. This file contains all Lighthouse CI configuration. The instructions here assume your project uses git. Next, create the file and add an empty configuration as a starting point:

touch lighthouserc.js

module.exports = {
  ci: {
    collect: {
      /* Add configuration here */
    },
    upload: {
      /* Add configuration here */
    },
  },
};

Lighthouse CI starts a local server each time it runs so that Lighthouse can load your site even with no other server active. The server is shut down automatically when the run completes. You must configure how that server should serve your content via one of two properties in the ci.collect object:

  • staticDistDir — for static sites, points to the directory containing static files that Lighthouse CI should serve during testing.
  • startServerCommand — for dynamic sites, the command used to start your own server, which Lighthouse CI launches as a subprocess during testing and terminates afterward.
// Static site example
collect: {
  staticDistDir: './public',
}

// Dynamic site example
collect: {
  startServerCommand: 'npm run start',
}

Next, set the URLs to audit with the url property in ci.collect. The value is an array of one or more URLs. By default, Lighthouse CI runs Lighthouse three times against each URL.

collect: {
  // ...
  url: ['http://localhost:8080']
}

Configure report storage by adding the target property to ci.upload with a value of 'temporary-public-storage'. Reports uploaded to temporary public storage are kept for seven days and then automatically deleted. This option is the fastest way to get started; other storage targets are documented in the Lighthouse CI configuration reference.

upload: {
  target: 'temporary-public-storage',
}

The report location will resemble the following URL pattern:

https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1580152437799-46441.report.html

Run the CLI with the autorun command to trigger three Lighthouse runs and upload the median report:

lhci autorun

Successful output looks like this:

✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
⚠️   GitHub token not set
Healthcheck passed!

Started a web server on port 65324...
Running Lighthouse 3 time(s) on http://localhost:65324/index.html
Run #1...done.
Run #2...done.
Run #3...done.
Done running Lighthouse!

Uploading median LHR of http://localhost:65324/index.html...success!
Open the report at https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1591720514021-82403.report.html
No GitHub token set, skipping GitHub status check.

Done running autorun.

You can ignore the GitHub token not set message in the console warning — that token is only required for the GitHub Action integration covered later in this guide. Clicking the https://storage.googleapis.com... link in the output opens the median Lighthouse report in a browser.

The autorun defaults are configurable both via CLI flags and through the configuration file. For example, to increase the number of runs per URL to five, set the numberOfRuns property in ci.collect:

module.exports = {
    // ...
    collect: {
      numberOfRuns: 5
    },
  // ...
  },
};

Then re-run autorun:

lhci autorun

Terminal output should confirm five runs were executed:

✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
⚠️   GitHub token not set
Healthcheck passed!

Automatically determined ./dist as `staticDistDir`.
Set it explicitly in lighthouserc.json if incorrect.

Started a web server on port 64444...
Running Lighthouse 5 time(s) on http://localhost:64444/index.html
Run #1...done.
Run #2...done.
Run #3...done.
Run #4...done.
Run #5...done.
Done running Lighthouse!

Uploading median LHR of http://localhost:64444/index.html...success!
Open the report at https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1591716944028-6048.report.html
No GitHub token set, skipping GitHub status check.

Done running autorun.

Further configuration options are described in the official Lighthouse CI configuration documentation.

Integrating Lighthouse CI with your CI provider

Lighthouse CI works with any common CI tool. The "Configure Your CI Provider" section of the Lighthouse CI getting-started guide contains configuration file examples for popular CI systems, showing how to run Lighthouse CI and collect performance measurements during the build.

Collecting measurements is a reasonable starting point, but advanced setups often go further: making the CI process fail when predefined criteria aren't met. The assert property in lighthouserc.js controls this behavior by evaluating assertions against the Lighthouse results.

Assertions support three severity levels:

  • off: ignore the assertion entirely
  • warn: print failures to stderr but do not exit with an error
  • error: print failures to stderr and exit Lighthouse CI with a non-zero exit code, which fails the build step

The following lighthouserc.js configuration sets assertions on the performance and accessibility category scores:

module.exports = {
  ci: {
    collect: {
      // ...
    },
    assert: {
      assertions: {
        'categories:performance': ['warn', {minScore: 1}],
        'categories:accessibility': ['error', {minScore: 1}]
      }
    },
    upload: {
      // ...
    },
  },
};

When the assertion fails, console output looks like this:

Screenshot of a warning message generated by Lighthouse CI

Full details on available assertion options are in the Lighthouse CI configuration reference.

Run Lighthouse checks from a GitHub Action

A GitHub Action can generate a fresh Lighthouse report on every push to any branch. Combined with a GitHub status check, those results can be displayed directly on each pull request.

Screenshot of a GitHub status check

  1. Create a directory named .github/workflows in the root of your repository. GitHub reads workflow configuration from this directory.

    mkdir .github
mkdir .github/workflows

  2. Inside .github/workflows, create a file called lighthouse-ci.yaml.

    touch lighthouse-ci.yaml

  3. Populate lighthouse-ci.yaml with the workflow definition:

    name: Build project and run Lighthouse CI
on: [push]
jobs:
  lhci:
    name: Lighthouse CI
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Use Node.js 10.x
        uses: actions/setup-node@v1
        with:
          node-version: 10.x
      - name: npm install
        run: |
          npm install
      - name: run Lighthouse CI
        run: |
          npm install -g @lhci/[email protected]
          lhci autorun --upload.target=temporary-public-storage || echo "LHCI failed!"

    This defines a workflow with a single job that runs on every push. The job has four steps:

    • Check out the repository
    • Install and configure Node
    • Install the required npm packages
    • Run Lighthouse CI and upload results to temporary public storage.

  4. Commit the file and push it to GitHub. The workflow will trigger on the push.

  5. Open the Actions tab of your project to confirm the run. The Build project and Run Lighthouse CI workflow should appear under the latest commit.

Screenshot of the GitHub 'Settings' tab

To inspect a report, open the Actions tab, click the relevant commit, open the Lighthouse CI workflow step, and expand the output of the run Lighthouse CI step.

Screenshot of the GitHub 'Settings' tab

Add a GitHub status check

A status check displays a message on every pull request, typically reporting whether tests passed or a build succeeded.

Screenshot of the GitHub 'Settings' tab

  1. Visit the Lighthouse CI GitHub App page and click Configure.

  2. If you are in multiple GitHub organizations, select the one that owns the repository.

  3. Choose All repositories to enable Lighthouse CI everywhere, or Only select repositories to limit it, then pick the repositories. Click Install & Authorize.

  4. Copy the displayed token; you will add it to your repository secrets in the next step.

  5. Open your repository's Settings page, click Secrets, then Add a new secret.

Screenshot of the GitHub 'Settings' tab

  1. Set the Name field to LHCI_GITHUB_APP_TOKEN and paste the copied token into Value. Click Add secret.

  2. Go back to lighthouse-ci.yaml and reference the new secret in the "run Lighthouse CI" step.

-           name: run Lighthouse CI
            run: |
              npm install -g @lhci/[email protected]
              lhci autorun --upload.target=temporary-public-storage || echo "LHCI failed!"
+            env:
+              LHCI_GITHUB_APP_TOKEN: $

  1. The status check is ready. Test it by creating a pull request or pushing a commit to an existing one.

Host the Lighthouse CI server for historical reports

The Lighthouse CI server offers a dashboard for browsing historical Lighthouse reports and works as a private, long-term store for them.

Screenshot of the Lighthouse CI Server dashboard
Screenshot of comparing two Lighthouse reports in Lighthouse CI Server

The comparison view shows:

  1. The two commits being compared.
  2. How much the Lighthouse score changed between them.
  3. Only metrics that changed; everything else is hidden.
  4. Regressions in pink.
  5. Improvements in blue.

The server is intended for teams comfortable deploying and managing their own infrastructure. Setup instructions, including Heroku and Docker deployment recipes, are available in the Lighthouse CI server documentation.

More resources