Automating performance checks with Lighthouse Bot

Performance work doesn't end when you ship an optimization. Every subsequent code change is a chance to regress. Rather than relying on manual DevTools or CLI audits, you can wire performance testing into your continuous integration pipeline so every pull request is checked automatically.

Lighthouse already grades your app across five categories, including performance. Lighthouse Bot plugs into Travis CI to run those audits in the cloud on each push. With a required status check configured on your repository, pull requests that drop your Lighthouse scores below a threshold you define won't merge.

Localhost audits are useful, but your app often behaves differently on a live server. For realistic results, deploy to a staging environment before running the bot. Any host works; this walkthrough uses Firebase hosting with a starter app from GitHub that sorts three numbers.

Deploy the starter app to Firebase

First, create a project in the Firebase console. Then make sure the Firebase CLI is current:

npm install -g firebase-tools

Authorize the CLI and initialize the project:

firebase login

firebase init

During setup you'll be asked:

  • Select Hosting as the feature.
  • Pick the Firebase project you created.
  • Use public as the public directory.
  • Answer N to the single-page app question.

That generates a firebase.json file in the project root. Deploy with:

firebase deploy

You'll get a live URL for your staging app.

Set up Travis CI

Register for Travis and activate GitHub Apps integration in your profile settings. Sync your account so the project repo appears in your Travis settings.

Continuous integration needs two pieces in place: a .travis.yml file in the repo root and a git push to trigger the first build. The starter repo already includes a .travis.yml that installs dependencies and builds the app:

language: node_js
node_js:
  - "8.1.3"
install:
  - npm install
before_script:
  - npm install -g firebase-tools
script:
  - webpack

Push the example app to your own GitHub repository:

git push origin main

In the Travis dashboard for your repo, the build should go from yellow to green within a couple of minutes.

Let Travis deploy for you

Travis can't authenticate to Firebase on its own. Generate a token from your local CLI:

firebase login:ci

After the browser verification, copy the token and go to your project's Travis dashboard. Under More options > Settings > Environment variables, add it as FIREBASE_TOKEN.

Then append these lines to .travis.yml so every successful build deploys:

after_success:
  - firebase deploy --token $FIREBASE_TOKEN --non-interactive

Push the change and watch the Travis log for a Deploy complete message. From now on, every committed change reaches your Firebase staging site automatically.

Invite Lighthouse Bot

Lighthouse Bot posts its audit results as a GitHub collaborator. Add lighthousebot as a collaborator via your repo's Settings > Collaborators page.

Lighthouse bot collaborator status

Collaborator approvals are handled manually, so this can take a little time. While waiting, request a Lighthouse Bot API key and add it to Travis as another environment variable, named LIGHTHOUSE_API_KEY:

Install the bot in your project:

npm i --save-dev https://github.com/ebidel/lighthousebot

Then add this to package.json:

"scripts": {
  "lh": "lighthousebot"
}

Enforce a budget on pull requests

The final step is telling Travis to run Lighthouse Bot after every successful build. In .travis.yml, add another step under after_success:

after_success:
  - firebase deploy --token $FIREBASE_TOKEN --non-interactive
  - npm run lh -- https://staging.example.com

Replace https://staging.example.com with your actual Firebase URL, like your-app-123.firebaseapp.com. To reject any change that brings the performance score below 95, set the budget like this:

- npm run lh -- --perf=95 https://staging.example.com

Lighthouse Bot only responds to pull requests; pushes to the main branch will log "This script can only be run on Travis PR requests." To run the test, push a new branch and open a pull request. On the PR page you'll see Lighthouse Bot report the scores and whether the budget check passed.

Passing Lighthouse scores Passing GitHub checks

Adjusting the audit thresholds

Performance is only one of the five categories Lighthouse tests. The bot accepts flags to enforce scores for any of them:

--perf  # performance
--pwa   # progressive web app score
--a11y  # accessibility score
--bp    # best practices score
--seo   # SEO score

For example, this fails a pull request if performance drops below 93 or SEO drops below 100:

npm run lh -- --perf=93 --seo=100 https://staging.example.com

If you'd rather keep the PR page clean, use the --no-comment option to suppress Lighthouse Bot's comments.