ToBeReviewed Bot
The automation in this repository supports a To-Be-Reviewed Pull Request workflow:
- Allows a repository to enable branch protection and require pull requests, but have flexibility in submission of pull requests in case of urgent need by not mandating an approver before submission.
- If a PR is submitted without an Approver, the bot will notice within a few minutes and file a GitHub issue requiring followup.
- The bot notes cases where intent is clear and does not intervene. Merging someone else's PR constitutes Approval. A comment containing "LGTM" constitutes Approval.
- The issues requiring followup carry a distinctive title allowing for easy generation of the full population during a Compliance-related periodic audit, and to demonstrate that all such issues did get a followup review within a reasonable amount of time.
Configuration variables
The bot expects to run continuously on a production system, and supports the following environment variables:
TBRBOT_ORG: The GitHub organization to use, like "tailscale".TBRBOT_BUGREPO: The name of the repository to file followup issues in, like "private".TBRBOT_APPNAME: The name of the GitHub app account, like "tbr-bot", to use as the author in issues filed.TBRBOT_REPOS: A comma-separated list of GitHub repositories to check for to-be-reviewed PRs, like "opensource,private".TBRBOT_APP_ID: The GitHub App ID, found in https://github.com/organizations/*ORGNAME*/settings/apps/*APPNAME*TBRBOT_APP_INSTALL: The GitHub App Install ID for this installation, found in https://github.com/organizations/*ORGNAME*/settings/installations
Additionally, the bot supports the following environment variables which should ideally be handled by secrets management infrastructure in cloud providers:
TBRBOT_APP_PRIVATE_KEY: a PEM encoded private key, which can be generated in https://github.com/organizations/*ORGNAME*/settings/apps/*APPNAME*
Reducing latency using GitHub webhooks
Normally the bot wakes up every hour to check for recently submitted PRs needing followup. Its reaction to a submitted PR can be hastened by setting up a GitHub webhook for "Pull requests" events (which send on a merged PR).
GitHub should be configured to deliver webhook events to https://Public-DNS-name/webhook
The bot expects to find the shared secret for validating webhook payloads in a WEBHOOK_SECRET
environment variable. The shared secret is configured in the webhook in
https://github.com/organizations/*ORGNAME*/settings/hooks/*WEBHOOK_ID*?tab=settings
Monitoring
When used as part of the controls for Compliance requirements, it is important to to monitor whether the bot is working. Finding out on the eve of an audit that the bot has been offline for an extended period would be ruinous.
In addition to /webhook the bot also exports metrics:
https://Tailscale-MagicDNS-name/debug/varsin JSON formathttps://Tailscale-MagicDNS-name/debug/varzin Prometheus metric format
The /debug endpoints can only be reached from a local Tailscale
tailnet. It is reasonable to allow public Internet access to https://Public-DNS-name/
for GitHub to be able to deliver webhooks, TBR-bot will restrict the other endpoints to
only be accessible via a private tailnet connection.
A metric of interest for monitoring is tbrbot_repos_checked, which counts the number of times
the bot has checked a repository for submitted PRs. This is expected to increment at least once
per hour. An alert when tbrbot_repos_checked goes N hours with no change is a reasonable way
to monitor TBR-bot's operation. An example alerting rule for Grafana in a panel for the
tbrbot_repos_checked metric is: WHEN diff_abs() OF query (A, 12h, now) IS BELOW 1
Hosting
The included Dockerfile and example fly.toml are suitable to run the tbr-bot hosted on fly.io.
We recommend forking this repository and making local modifications to the supplied fly.toml to set it to the name of your instance and update the environment variables to correspond to the GitHub repositories you want it to watch.
The bot needs a small amount of persistent storage for its Tailscale state, plus the various configuration and secrets described above.
$ flyctl volumes create tbrbot_data --region sjc --size 1
$ flyctl scale count 1
$ flyctl secrets set TS_AUTHKEY=... TBRBOT_APP_ID=... TBRBOT_APP_INSTALL=...
$ flyctl secrets set TBRBOT_WEBHOOK_SECRET=...
$ flyctl secrets set TBRBOT_APP_PRIVATE_KEY=- < pem
$ flyctl ips allocate-v6
$ flyctl ips allocate-v4
We recommend using a one-time authkey with Tags set to
authorize the bot to join the tailnet. Once the bot has run once and written its state
to persistent storage, the TS_AUTHKEY secret should be removed.
Contributing
PRs welcome! But please file bugs. Commit messages should reference bugs.
We require Developer Certificate of Origin
Signed-off-by lines in commits.