• This repository has been archived on 09/Jul/2020
  • Stars
    star
    2,235
  • Rank 20,610 (Top 0.5 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created over 7 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Run Lighthouse in CI, as a web service, using Docker. Pass/Fail GH pull requests.

Lighthouse Bot (deprecated)

Update: LighthouseBot has been deprecated and we now recommend using the official Lighthouse CI project to automate running Lighthouse for every commit, view the changes, and prevent regressions

Historical README below

This repo contained the frontend and backend for running Lighthouse in CI and integration with Github Pull Requests. An example web service is hosted for demo purposes.

Auditing GitHub Pull Requests

Please note: This drop in service is considered Beta. There are no SLAs or uptime guarantees. If you're interested in running your own CI server in a Docker container, check out Running your own CI server.

Lighthouse can be setup as part of your CI on Travis only. As new pull requests come in, the Lighthouse Bot tests the changes and reports back the new score.

Run Lighthouse on Github PRs

To audit pull requests, do the following:

1. Initial setup

Add the lighthousebot to your repo

First, add lighthousebot as a collaborator on your repo. Lighthouse CI uses an OAuth token scoped to the repo permission in order to update the status of your PRs and post comments on the issue as the little Lighthouse icon.

* Until Lighthousebot accepts your invitation to collaborate, which is currently a lengthy manual process, it does not have permission to update the status of your PRs. However, it will post a comment on your PR.

Get an API Key

Request an API Key. API keys will eventually be enforced and are necessary so we can contact you when there are changes to the CI system.

Once you have a key, update Travis settings by adding an LIGHTHOUSE_API_KEY environment variables with your key:

Travis LIGHTHOUSE_API_KEY env variable

The lighthousebot script will include your key in requests made to the CI server.

2. Deploy the PR

We recommend deploying your PR to a real staging server instead of running a local server on Travis. A staging environment will produce realistic performance numbers that are more representative of your production setup. The Lighthouse report will be more accurate.

In .travis.yml, add an after_success that deploys the PR's changes to a staging server.

after_success:
  - ./deploy.sh # TODO(you): deploy the PR changes to your staging server.

Since every hosting environment has different deployment setups, the implementation of deploy.sh is left to the reader.

Tip: Using Google App Engine? Check out deploy_pr_gae.sh which shows how to install the GAE SDK and deploy PR changes programmatically.

3. Call lighthousebot

Install the script:

npm i --save-dev https://github.com/GoogleChromeLabs/lighthousebot

Add an NPM script to your package.json:

"scripts": {
  "lh": "lighthousebot"
}

Next, in .travis.yml call npm run lh as the last step in after_success:

install:
  - npm install # make sure to install the deps when Travis runs.
after_success:
  - ./deploy.sh # TODO(you): deploy the PR changes to your staging server.
  - npm run lh -- https://staging.example.com

When Lighthouse is done auditing the URL, the bot will post a comment to the pull request containing the updated scores:

Lighthouse Github comment

You can also opt-out of the comment by using the --no-comment flag.

Failing a PR when it drops your Lighthouse score

Lighthouse CI can prevent PRs from being merged when one of the scores falls below a specified value. Just include one or more of --pwa, --perf, --seo, --a11y, or --bp:

after_success:
  - ./deploy.sh # TODO(you): deploy the PR changes to your staging server.
  - npm run lh -- --perf=96 --pwa=100 https://staging.example.com

Options

$ lighthouse-ci -h

Usage:
runlighthouse.js [--perf,pwa,seo,a11y,bp=<score>] [--no-comment] [--runner=chrome,wpt] <url>

Options:
  Minimum score values can be passed per category as a way to fail the PR if
  the thresholds are not met. If you don't provide thresholds, the PR will
  be mergeable no matter what the scores.

  --pwa        Minimum PWA score for the PR to be considered "passing". [Number]
  --perf       Minimum performance score for the PR to be considered "passing". [Number]
  --seo        Minimum seo score for the PR to be considered "passing". [Number]
  --a11y       Minimum accessibility score for the PR to be considered "passing". [Number]
  --bp         Minimum best practices score for the PR to be considered "passing". [Number]

  --no-comment Doesn't post a comment to the PR issue summarizing the Lighthouse results. [Boolean]

  --runner     Selects Lighthouse running on Chrome or WebPageTest. [--runner=chrome,wpt]

  --help       Prints help.

Examples:

  Runs Lighthouse and posts a summary of the results.
    runlighthouse.js https://example.com

  Fails the PR if the performance score drops below 93. Posts the summary comment.
    runlighthouse.js --perf=93 https://example.com

  Fails the PR if perf score drops below 93 or the PWA score drops below 100. Posts the summary comment.
    runlighthouse.js --perf=93 --pwa=100 https://example.com

  Runs Lighthouse on WebPageTest. Fails the PR if the perf score drops below 93.
    runlighthouse.js --perf=93 --runner=wpt --no-comment https://example.com

Running on WebPageTest instead of Chrome

By default, lighthousebot runs your PRs through Lighthouse hosted in the cloud. As an alternative, you can test on real devices using the WebPageTest integration:

lighthousebot --perf=96 --runner=wpt https://staging.example.com

At the end of testing, your PR will be updated with a link to the WebPageTest results containing the Lighthouse report!

Running your own CI server

Want to setup your own Lighthouse instance in a Docker container?

The good news is Docker does most of the work for us! The bulk of getting started is in Development. That will take you through initial setup and show how to run the CI frontend.

For the backend, see builder/README.md for building and running the Docker container.

Other changes, to the "Development" section:

  • Create a personal OAuth token in https://github.com/settings/tokens. Drop it in frontend/.oauth_token.
  • Add a LIGHTHOUSE_CI_HOST env variable to Travis settings that points to your own URL. The one where you deploy the Docker container.

Development

Initial setup:

  1. Ask an existing dev for the oauth2 token. If you need to regenerate one, see below.
  • Create frontend/.oauth_token and copy in the token value.

Run the dev server:

cd frontend
npm run start

This will start a web server and use the token in .oauth_token. The token is used to update PR status in Github.

In your test repo:

  • Run npm i --save-dev https://github.com/GoogleChromeLabs/lighthousebot
  • Follow the steps in Auditing Github Pull Requests for setting up your repo.

Notes:

  • If you want to make changes to the builder, you'll need Docker and the GAE Node SDK.
  • To make changes to the CI server, you'll probably want to run ngrok so you can test against a local server instead of deploying for each change. In Travis settings, add a LIGHTHOUSE_CI_HOST env variable that points to your ngrok instance.
Generating a new OAuth2 token

If you need to generate a new OAuth token:

  1. Sign in to the lighthousebot Github account. (Admins: the credentials are in the usual password tool).
  2. Visit personal access tokens: https://github.com/settings/tokens.
  3. Regenerate the token. Important: this invalidates the existing token so other developers will need to be informed.
  4. Update token in frontend/.oauth_token.

Deploy

By default, these scripts deploy to Google App Engine Flexible containers (Node). If you're running your own CI server, use your own setup :)

Deploy the frontend:

npm run deploy YYYY-MM-DD frontend

Deploy the CI builder backend:

npm run deploy YYYY-MM-DD builder

Source & Components

This repo contains several different pieces for the Lighthouse Bot: a backend, frontend, and frontend UI.

UI Frontend

Quick way to try Lighthouse: https://lighthouse-ci.appspot.com/try

Relevant source:

Bot CI server (frontend)

Server that responds to requests from Travis.

REST endpoints:

  • https://lighthouse-ci.appspot.com/run_on_chrome
  • https://lighthouse-ci.appspot.com/run_on_wpt

Example

Note: lighthousebot does this for you.

POST https://lighthouse-ci.appspot.com/run_on_chrome
Content-Type: application/json
X-API-KEY: <YOUR_LIGHTHOUSE_API_KEY>

{
  testUrl: "https://staging.example.com",
  thresholds: {
    pwa: 100,
    perf: 96,
  },
  addComment: true,
  repo: {
    owner: "<REPO_OWNER>",
    name: "<REPO_NAME>"
  },
  pr: {
    number: <PR_NUMBER>,
    sha: "<PR_SHA>"
  }
}

Relevant source:

  • frontend/server.js - server which accepts Github pull requests and updates the status of your PR.

CI backend (builder)

Server that runs Lighthouse against a URL, using Chrome.

REST endpoints:

  • https://lighthouse-ci.appspot.com/ci

Example

Note: lighthousebot does this for you.

curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-API-KEY: <YOUR_LIGHTHOUSE_API_KEY>" \
  --data '{"output": "json", "url": "https://staging.example.com"}' \
  https://builder-dot-lighthouse-ci.appspot.com/ci

FAQ

Why not deployment events?

Github's Deployment API would be ideal, but it has some downsides:

  • Github Deployments happen after a pull is merged. We want to support blocking PR merges based on a LH score.
  • We want to be able to audit changes as they're add to the PR. pull_request/push events are more appropriate for that.
Why not a Github Webhook?

The main downside of a Github webhook is that there's no way to include custom data in the payload Github sends to the webhook handler. For example, how would Lighthouse know what url to test? With a webhook, the user also has to setup it up and configure it properly.

Future work: Lighthouse Bot could define a file that developer includes in their repo. The bot's endpoint could pull a .lighthouse_ci file that includes meta data {minLighthouseScore: 96, testUrl: 'https://staging.example.com'}. However, this requires work from the developer.

More Repositories

1

squoosh

Make images smaller using best-in-class codecs, right in the browser.
TypeScript
21,605
star
2

comlink

Comlink makes WebWorkers enjoyable.
TypeScript
11,198
star
3

quicklink

⚡️Faster subsequent page-loads by prefetching in-viewport links during idle time
JavaScript
11,024
star
4

ndb

ndb is an improved debugging experience for Node.js, enabled by Chrome DevTools
JavaScript
10,913
star
5

carlo

Web rendering surface for Node applications
JavaScript
9,313
star
6

ProjectVisBug

FireBug for designers › Edit any webpage, in any state https://a.nerdy.dev/gimme-visbug
JavaScript
5,452
star
7

sw-precache

[Deprecated] A node module to generate service worker code that will precache specific resources so they work offline.
JavaScript
5,236
star
8

react-adaptive-hooks

Deliver experiences best suited to a user's device and network constraints
JavaScript
5,097
star
9

ui-element-samples

A collection of prototyped UI elements
JavaScript
4,101
star
10

sw-toolbox

[Deprecated] A collection of service worker tools for offlining runtime requests
JavaScript
3,621
star
11

webpack-libs-optimizations

Using a library in your webpack project? Here’s how to optimize it
3,365
star
12

critters

🦔 A Webpack plugin to inline your critical CSS and lazy-load the rest.
JavaScript
3,330
star
13

psi

PageSpeed Insights Reporting for Node
JavaScript
3,100
star
14

bubblewrap

Bubblewrap is a Command Line Interface (CLI) that helps developers to create a Project for an Android application that launches an existing Progressive Web App (PWAs) using a Trusted Web Activity.
TypeScript
2,375
star
15

preload-webpack-plugin

Please use https://github.com/vuejs/preload-webpack-plugin instead.
JavaScript
2,162
star
16

worker-plugin

👩‍🏭 Adds native Web Worker bundling support to Webpack.
JavaScript
1,914
star
17

prerender-loader

📰 Painless universal pre-rendering for Webpack.
JavaScript
1,912
star
18

simplehttp2server

A simple HTTP/2 server for development
Go
1,735
star
19

jsvu

JavaScript (engine) Version Updater
JavaScript
1,698
star
20

size-plugin

Track compressed Webpack asset sizes over time.
JavaScript
1,672
star
21

clooney

Clooney is an actor library for the web. Use workers without thinking about workers.
JavaScript
1,419
star
22

browser-fs-access

File System Access API with legacy fallback in the browser
JavaScript
1,383
star
23

proxx

A game of proximity
TypeScript
1,315
star
24

dark-mode-toggle

A custom element that allows you to easily put a Dark Mode 🌒 toggle or switch on your site:
JavaScript
1,174
star
25

application-shell

Service Worker Application Shell Architecture
JavaScript
1,169
star
26

pwacompat

PWACompat to bring Web App Manifest to older browsers
JavaScript
1,130
star
27

container-query-polyfill

A polyfill for CSS Container Queries
TypeScript
1,127
star
28

idlize

Helper classes and methods for implementing the idle-until-urgent pattern
JavaScript
1,061
star
29

houdini-samples

Demos for different Houdini APIs
JavaScript
974
star
30

jsbi

JSBI is a pure-JavaScript implementation of the official ECMAScript BigInt proposal.
JavaScript
908
star
31

css-triggers

A reference for the render impact of mutating CSS properties.
JavaScript
893
star
32

howto-components

Literate code examples for common UI patterns.
JavaScript
851
star
33

tooling.report

tooling.report a quick way to determine the best build tool for your next web project, or if tooling migration is worth it, or how to adopt a tool's best practice into your existing configuration and code base.
JavaScript
848
star
34

page-lifecycle

PageLifecycle.js is a tiny JavaScript library that allows developers to easily observe Page Lifecycle API state changes cross browser
JavaScript
803
star
35

css-paint-polyfill

CSS Custom Paint / Paint Worklet polyfill with special browser optimizations.
JavaScript
709
star
36

web-audio-samples

Web Audio API samples by Chrome Web Audio Team
JavaScript
686
star
37

chrome-for-testing

JavaScript
667
star
38

picture-in-picture-chrome-extension

JavaScript
667
star
39

estimator.dev

🧮 Calculate the size and performance impact of switching to modern JavaScript syntax.
JavaScript
665
star
40

comlink-loader

Webpack loader to offload modules to Worker threads seamlessly using Comlink.
JavaScript
616
star
41

pwa-wp

WordPress feature plugin to bring Progressive Web Apps (PWA) to Core
PHP
615
star
42

ProgressiveWordPress

A Sample WordPress-based Progressive Web App
JavaScript
570
star
43

web-push-codelab

JavaScript
557
star
44

text-app

A text editor for ChromeOS and Chrome
JavaScript
552
star
45

gulliver

A PWA directory, focusing on collecting PWA best practices and examples.
JavaScript
549
star
46

wasm-feature-detect

A small library to detect which features of WebAssembly are supported.
JavaScript
546
star
47

progressive-tooling

A list of community-built, third-party tools that can be used to improve page performance
JavaScript
546
star
48

svgomg-twa

A sample that project Trusted Web Activities technology to wrap SVGOMG in an Android Application
Shell
519
star
49

web-vitals-report

Measure and report on your Web Vitals data in Google Analytics
JavaScript
503
star
50

text-editor

A text editor build on the Native File System APIs
JavaScript
501
star
51

pptraas.com

Puppeteer as a service
JavaScript
455
star
52

progressive-rendering-frameworks-samples

Samples and demos from the Progressive Rendering I/O talk
JavaScript
411
star
53

wasm-bindgen-rayon

An adapter for enabling Rayon-based concurrency on the Web with WebAssembly.
JavaScript
403
star
54

MiniMobileDeviceLab

A mini mobile web device lab
Objective-C
396
star
55

webm-wasm

webm-wasm lets you create webm videos in JavaScript via WebAssembly.
C++
394
star
56

link-to-text-fragment

Browser extension that allows for linking to arbitrary text fragments.
JavaScript
390
star
57

cronet-sample

A sample for the Cronet library
Java
390
star
58

buffer-backed-object

Buffer-backed objects in JavaScript.
JavaScript
372
star
59

pinch-zoom

TypeScript
370
star
60

webpack-training-project

A training project for learning Webpack optimizations
JavaScript
368
star
61

samesite-examples

Examples of using the SameSite cookie attribute in a variety of language, libraries, and frameworks.
HTML
366
star
62

airhorn

Air horn
JavaScript
361
star
63

AutoWebPerf

AutoWebPerf provides a flexible and scalable framework for running web performance audits with arbitrary audit tools including PageSpeedInsights, WebPageTest and more.
JavaScript
353
star
64

first-input-delay

A JavaScript library for measuring First Input Delay (FID) in the browser.
JavaScript
347
star
65

tti-polyfill

Time-to-interactive polyfill
JavaScript
333
star
66

react-shrine

"Shrine" Progressive Web App sample built with React
JavaScript
330
star
67

dynamic-import-polyfill

A fast, tiny polyfill for dynamic import() that works in all module-supporting browsers
JavaScript
320
star
68

wasi-fs-access

This is a demo shell powered by WebAssembly, WASI, Asyncify and File System Access API.
TypeScript
312
star
69

native-url

Node's url module implemented using the built-in URL API.
JavaScript
284
star
70

two-up

TypeScript
281
star
71

adaptive-loading

Demos for Adaptive Loading - differentially deliver fast, lighter experiences for users on slow networks & devices
JavaScript
264
star
72

so-pwa

A progressive web app to read Stack Overflow content.
JavaScript
255
star
73

import-from-worker

It’s like import(), but runs the module in a worker.
JavaScript
249
star
74

sample-pie-shop

Example e-commerce site to explore PWA (Progressive Web App) use cases.
JavaScript
237
star
75

file-drop

A simple file drag and drop custom-element
TypeScript
232
star
76

serial-terminal

Demo application for the Web Serial API.
TypeScript
226
star
77

form-troubleshooter

TypeScript
215
star
78

postcss-jit-props

A CSS custom property helper based on PostCSS. Supply a pool of variables and this plugin will add them to the stylesheet as they are used.
JavaScript
215
star
79

audioworklet-polyfill

🔊 Polyfill AudioWorklet using the legacy ScriptProcessor API.
JavaScript
190
star
80

http2-push-manifest

Generate a list of static resources for http2 push.
JavaScript
187
star
81

pointer-tracker

Track mouse/touch/pointer events for a given element.
TypeScript
184
star
82

pr-bot

🤖 Compare your base branch to a pull request and run plugins over it to view differences
JavaScript
179
star
83

discovery

Discoveries on Sustainable Loading research
177
star
84

imagecapture-polyfill

MediaStream ImageCapture polyfill. Take photos from the browser as easy as .takePhoto().then(processPhoto)
JavaScript
176
star
85

tasklets

JavaScript
176
star
86

telnet-client

TypeScript
166
star
87

extension-manifest-converter

Python
164
star
88

credential-management-sample

Credential Management Sample
HTML
157
star
89

wadb

A TypeScript implementation of the Android Debug Bridge(ADB) protocol over WebUSB
TypeScript
156
star
90

extension-update-testing-tool

Tool for testing the update flow for Chrome Extensions during development, including permission grants.
JavaScript
154
star
91

wasm-av1

Port of the AV1 Video codec to WebAssembly
C
154
star
92

houdini.how

A community-driven gathering place for CSS Houdini worklets and resources.
JavaScript
152
star
93

pwa-workshop-codelab

JavaScript
152
star
94

perf-track

Tracking framework performance and usage at scale
Svelte
150
star
95

kv-storage-polyfill

A polyfill for the kv-storage built-in module.
JavaScript
146
star
96

http2push-gae

Drop-in HTTP2 push on App Engine
HTML
140
star
97

chromeos_smart_card_connector

Smart Card Connector App for Chrome OS
C++
135
star
98

snapshot

TypeScript
133
star
99

devwebfeed

Firehose of team++ resources
JavaScript
130
star
100

sample-currency-converter

A sample currency conversion Progressive Web App
JavaScript
129
star