• Stars
    star
    243
  • Rank 165,150 (Top 4 %)
  • Language
    Nix
  • License
    The Unlicense
  • Created over 5 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Nix functions for filtering local git sources

Make Nix precisely emulate gitignore

This goal of this project lets you include local sources in your Nix projects, while taking gitignore files into account.

Note

Although this project does a good job at emulating git's behavior, it is not the same implementation! It is unaware of the git "index" and committed files, and therefore unaware of files that are in the index despite gitignore.

Installation

Recommended with Niv

nix-env -iA niv -f https://github.com/nmattia/niv/tarball/master
niv init
niv add hercules-ci/gitignore.nix

With Flakes

Although Flakes usually process sources within the flake using the git fetcher, which takes care of ignoring in its own peculiar way, you can use gitignore.nix to filter sources outside of a flake. You can load flake-based expressions via builtins.getFlake (toString ./.) for example or via flake-compat.

# // flake.nix
{
  inputs.gitignore = {
    url = "github:hercules-ci/gitignore.nix";
    # Use the same nixpkgs
    inputs.nixpkgs.follows = "nixpkgs";
  };
  outputs = { self, /* other, inputs, */ gitignore }:
  let
    inherit (gitignore.lib) gitignoreSource;
  in {
    packages.x86_64.hello = mkDerivation {
      name = "hello";
      src = gitignoreSource ./vendored/hello;
    };
  };
}

Plain Nix way

let
  gitignoreSrc = pkgs.fetchFromGitHub { 
    owner = "hercules-ci";
    repo = "gitignore.nix";
    # put the latest commit sha of gitignore Nix library here:
    rev = "";
    # use what nix suggests in the mismatch message here:
    sha256 = "sha256:0000000000000000000000000000000000000000000000000000";
  };
  inherit (import gitignoreSrc { inherit (pkgs) lib; }) gitignoreSource;
in
  <your nix expression>

Or using only nixpkgs/lib and only evaluation-time fetching:

import (builtins.fetchTarball "https://github.com/hercules-ci/gitignore.nix/archive/000000000000000000000000000000000000000000000000000".tar.gz") {
  lib = import (builtins.fetchTarball "https://github.com/NixOS/nixpkgs/archive/000000000000000000000000000000000000000000000000000".tar.gz" + "/lib");
}

Usage

mkDerivation {
  name = "hello";
  src = gitignoreSource ./vendored/hello;
}
gitignoreSource :: path -> path

Returns result of cleanSourceWith, usable as a path but also composable.

gitignoreFilter :: path -> (path -> type -> bool)

f = gitignoreFilter path

Parentheses in the type emphasize that a partial application memoizes the git metadata. You can use Nixpkgs' cleanSourceWith to compose with other filters (by logical and) or to set a name.

path: a path being the root or a subdirectory of a local git repository

f: a function that returns false for files that should be ignored according to gitignore rules, but only for paths at or below path.

See gitignoreFilter for an example.

Features

  • Reads parent gitignores even if only pointed at a subdirectory
  • Source hashes only change when output changes
  • Not impacted by large or inaccessible ignored directories
  • Composes with cleanSourceWith
  • Reads user git configuration; no need to bother your team with your tool config.
  • Also works with restrict-eval enabled (if avoiding fetchFromGitHub)
  • No import from derivation ("IFD")
  • Name and hash are not sensitive to checkout location

Comparison

Feature \ Implementation cleanSource fetchGit/fetchTree siers siers recursive icetan Profpatsch numtide this project
Reproducible ✔️ ! ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Includes added but ignored files ✔️ ✔️
Uses user tooling rules from dotfiles ? ✔️
Ignores .git ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
No special Nix configuration ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
No import from derivation ✔️ ! ✔️ ✔️ ✔️ ✔️ ✔️
Uses subdirectory gitignores ✔️ ✔️ ✔️ ✔️
Uses parent gitignores ✔️ ✔️ ? ✔️
Uses user gitignores ✔️ ? ✔️ ✔️
Has a test suite ✔️ ? ✔️ ✔️ ✔️ ? ✔️
Works with restrict-eval / Hydra ✔️ ? ✔️ ✔️ ✔️ ✔️
Descends into submodule correctly ✔️ ? ? ? ? ✔️ ? ? #8
Included in nixpkgs ✔️ ✔️ ✔️ ✔️
No traversal of ignored dirs
(perf on large repos)
- ✔️ ✔️ ? ✔️ ✔️ ? ✔️ ? ✔️ ?
Legend
✔️ Supported
✔️ ? Probably supported
Not supported
? Probably not supported
- Not applicable or depends
! Caveats

Caveats:

  • fetchGit is not reproducible. It has at least one serious reproducibility problem that requires a breaking change to fix. Unlike fixed-output derivations, a built-in fetcher does not have a pinned implementation!
  • fetchGit blocks the evaluator, just like import from derivation

Please open a PR if you've found another feature, determined any of the '?' or if you've found an inaccuracy.

Security

Files not matched by gitignore rules will end up in the Nix store, which is readable by any process.

gitignore.nix does not yet understand git-crypt's metadata, so don't call gitignoreSource on directories containing such secrets or their parent directories. This also applies to many Nix functions (path, filterSource, cleanSource, etc) and Nix string interpolation.

Contributing

This project isn't perfect (yet) so please submit test cases and fixes as pull requests. Before doing anything drastic, it's a good idea to open an issue first to discuss and optimize the approach.

Thanks

A great shoutout to @siers for writing the intial test suite and rule translation code!

More Repositories

1

flake-parts

❄️ Simplify Nix Flakes with the module system
Nix
703
star
2

arion

Run docker-compose with help from Nix/NixOS
Nix
613
star
3

legacy-old-hercules

Abandoned
Haskell
168
star
4

hercules-ci-agent

https://hercules-ci.com build and deployment agent
Haskell
97
star
5

canonix

Experiment in Nix formatting
Nix
21
star
6

warp-systemd

Socket activation and other systemd integration for the Warp web server (WAI)
Haskell
20
star
7

hercules-ci-effects

Expressions to change the world (just a tiny bit)
Nix
16
star
8

project.nix

A configuration manager for your projects
Nix
13
star
9

flake.parts-website

Deployment of the https://flake.parts website
Nix
9
star
10

hage

A simple, modern and secure file encryption tool, format, and library, based on https://github.com/FiloSottile/age
Haskell
7
star
11

flake-compat-ci

Stop-gap for traversing flakes with nix-build and Hercules CI
Nix
5
star
12

support

User feedback, questions and our public roadmap. [email protected]
5
star
13

nixops2-flake

Example continuous deployment with NixOps 2 and a Nix Flake
Nix
3
star
14

nixops-example

Nix
3
star
15

gh-pages-example

An example of deploying to GitHub pages with hercules-ci-effects
Nix
3
star
16

nixops-hercules-ci

Use Hercules CI as a NixOps backend
Python
2
star
17

docs.hercules-ci.com

https://docs.hercules-ci.com
Handlebars
2
star
18

haskell-language-server.nix

Nix wrapper providing builds of https://github.com/haskell/ide
Nix
2
star
19

haskell-releaser

Automation of Haskell package release process.
Haskell
2
star
20

terraform-hercules-ci

Terraform modules for Hercules CI
Nix
1
star
21

agent-in-arion

Experimental module to run hercules-ci-agent in podman via arion
Nix
1
star
22

hercules-ci-enterprise

Binaries for Hercules CI Enterprise
Nix
1
star
23

devshell-example

Nix
1
star
24

elm-hercules-extras

Convenience functions not found in core elm packages
Elm
1
star
25

miniherc

A GitHub Action that runs Hercules CI jobs. Good enough for some use cases.
1
star
26

nix-ci-minimal-repo

Nix
1
star