Magic Nix Cache

Save 30-50%+ of CI time without any effort or cost. Use Magic Nix Cache, a totally free and zero-configuration binary cache for Nix on GitHub Actions.

Add our GitHub Action after installing Nix, in your workflow, like this:

- uses: DeterminateSystems/magic-nix-cache-action@main

See Usage for a detailed example.

Why use the Magic Nix Cache?

Magic Nix Cache uses the GitHub Actions built-in cache to share builds between Workflow runs, and has many advantages over alternatives.

1. Totally free: backed by GitHub Actions' cache, there is no additional service to pay for.
2. Zero configuration: add our action to your workflow. That's it. Everything built in your workflow will be cached.
3. No secrets: Forks and pull requests benefit from the cache, too.
4. Secure: Magic Nix Cache follows the same semantics as the GitHub Actions cache, and malicious pull requests cannot pollute your project.
5. Private: The cache is stored in the GitHub Actions cache, not with an additional third party.

Note: the Magic Nix Cache doesn't offer a publicly available cache. This means the cache is only usable in CI. Zero to Nix has an article on binary caching if you want to share Nix builds with users outside of CI.

Usage

Add it to your Linux and macOS GitHub Actions workflows, like this:

name: CI

on:
  push:
  pull_request:

jobs:
  check:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v3
      - uses: DeterminateSystems/nix-installer-action@main
      - uses: DeterminateSystems/magic-nix-cache-action@main
      - run: nix flake check

That's it. Everything built in your workflow will be cached.

Usage Notes

The GitHub Actions Cache has a rate limit on reads and writes. Occasionally, large projects or large rebuilds may exceed those rate-limits, and you'll see evidence of that in your logs. The error looks like this:

error: unable to download 'http://127.0.0.1:37515/<...>': HTTP error 418
       response body:
       GitHub API error: API error (429 Too Many Requests): StructuredApiError { message: "Request was blocked due to exceeding usage of resource 'Count' in namespace ''." }

The caching daemon and Nix both handle this gracefully, and won't cause your CI to fail. When the rate limit is exceeded while pulling dependencies, your workflow may perform more builds than usual. When the rate limit is exceeded while uploading to the cache, the remainder of those store paths will be uploaded on the next run of the workflow.

Development

This project depends on the GitHub Actions Cache API. For local development, see gha-cache/README.md for more details on how to obtain the required tokens.

cargo run -- -c creds.json --upstream https://cache.nixos.org
cargo build --release --target x86_64-unknown-linux-musl
cargo build --release --target aarch64-unknown-linux-musl
nix copy --to 'http://127.0.0.1:3000' $(which bash)
nix-store --store $PWD/test-root --extra-substituters 'http://localhost:3000' --option require-sigs false -r $(which bash)

Acknowledgement

Magic Nix Cache is a collaboration with Zhaofeng Li. Zhaofeng is a major contributor to the Nix community, having authored Attic and Colmena. We'd like to express our deep gratitude to Zhaofeng for his tremendous work on this project.

Telemetry

The goal of Magic Nix Cache is to help teams save time in CI. The cache daemon collects a little bit of telemetry information to help us make that true.

Here is a table of the telemetry data we collect:

To disable diagnostic reporting, set the diagnostics URL to an empty string by passing --diagnostic-endpoint="".

You can read the full privacy policy for Determinate Systems, the creators of this tool and the Determinate Nix Installer, here.