• Stars
    star
    334
  • Rank 126,264 (Top 3 %)
  • Language
    Nix
  • Created almost 7 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Nix-based incremental build tool for Haskell projects

Build Status built with nix

Snack

SNACK IS UNMAINTAINED!!

It was a fun proof of concept but I don't have the time to take it further.

snack is a build tool that uses the power of Nix to build Haskell projects.

Snack requires Nix >= 2.0

It will

  • use your existing Hpack file or a Nix-based config (described below).
  • build your project incrementally: running snack build will only rebuild the modules that have been modified since the previous build.
  • work in the Nix sandbox.
  • give you lots of cool Nix features for free: strong reproducibility guarantees, remote caching, remote builds, and more.
  • improve build performance in some cases, for instance:
    • all Haskell modules are built in parallel.
    • there is a single linking step performed (typically) on a fast tmpfs.

Excited? Check out the install and usage sections. Make sure to also check out the Caveat Emptor section.

Install

See the Hacking section if you want to hack on snack

Make sure you have Nix installed.

1. With niv

  1. If Niv is not already configured in your project:
$ niv init
  1. Add the nmattia/snack dependency with Niv (this will add an entry in your nix/sources.json with the latest revision of Snack):
$ niv add nmattia/snack
  1. Then to use it you will need to import the source description from nix/sources.nix and then import the source (which downloads and evaluates its default.nix) so you can access the snack-exe attribute:
# nix/default.nix
let
  sources = import ./sources.nix;
in
  {
    snack = (import sources.snack).snack-exe;
  }

An example that combines this into an overlay over the nixpkgs also exported by nix/sources.nix:

# nix/default.nix
{ sources ? import ./sources.nix }:
with
  { overlay = _: _:
      {
        niv = (import sources.niv {}).niv; # this is how you import niv into the overlay as well
        snack = (import sources.snack).snack-exe;
      };
  };
import sources.nixpkgs
  { overlays = [ overlay ] ; config = {}; }
  1. Then you can have a shell.nix import it as such:
# shell.nix
with { pkgs = import ./nix {}; };
pkgs.mkShell {
  buildInputs = [ pkgs.niv pkgs.nix pkgs.snack ];
}

2. Globally

Run this command:

$ nix-env -iA snack-exe -f https://github.com/nmattia/snack/tarball/master

The snack executable is now in your PATH:

$ snack --help
Usage: <interactive> [-l|--lib DIR] ([-s|--snack-nix PATH] | [--no-snack-nix])
                     [-j|--jobs INT] [-p|--package-file PATH] (COMMAND |
                     COMMAND)

Available options:
  -l,--lib DIR             Path to the directory to use as the Nix library
                           instead of the default one bundled with the snack
                           executable.
  -s,--snack-nix PATH      Use the specified environment (snack.nix) file. When
                           none is provided ./snack.nix is used (if it exists).
                           (Use --no-snack-nix to disable this behavior)
  --no-snack-nix           Don't use ./snack.nix as the environment (snack.nix)
                           file.
  -j,--jobs INT            How many jobs to run concurrently
  -p,--package-file PATH   Specifies a YAML or Nix file to use as package
                           description. If not provided, snack looks for either
                           'package.yaml' or 'package.nix' in the current
                           directory.
  -h,--help                Show this help text

Available commands:
  build
  run
  ghci

Unavailable commands:
  test                     Use build, run or ghci commands with test suites.

Snack can be used to build, run and interact with packages. There is no test command as we treat test suites as we do executables, giving each test suite its own package description.

Usage

There are two ways to describe a package:

The next two sections show an example config for each option. They use the following example project which displays the title of the top-rated post on the haskell subreddit (you can also find the code here):

.
β”œβ”€β”€ app
β”‚Β Β  └── Main.hs
└── src
    └── Lib.hs

src/Lib.hs :

module Lib where

import Control.Lens
import Network.Wreq
import Data.Aeson.Lens
import Data.Text (Text)

topReddit :: IO Text
topReddit =
    getWith opts url
      <&> (^. responseBody
      . key "data"
      . key "children"
      . nth 0
      . key "data"
      . key "title"
      . _String)
  where
    url = "https://www.reddit.com/r/haskell/top.json"
    opts = defaults
      & param "limit" .~ ["1"]
      & param "t" .~ ["all"]

app/Main.hs :

module Main where

import Lib

main :: IO ()
main = topReddit >>= print

Hpack

The project can have this minimal package.yaml:

name: snack-readme

dependencies:
    - lens
    - wreq

library:
    source-dirs: ./src

executable:
    main: Main.hs
    source-dirs: ./app
    dependencies:
        - snack-readme

default-extensions:
    - OverloadedStrings

This command will build the project and display the top-rated post's title:

$ snack run

You can also build without executing:

$ snack build

Alternatively you can load up the project in ghci:

$ snack ghci
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Lib              ( /home/nicolas/projects/nmattia/snack/tests/readme/src/Lib.hs, interpreted )
[2 of 2] Compiling Main             ( /home/nicolas/projects/nmattia/snack/tests/readme/app/Main.hs, interpreted )
Ok, two modules loaded.
*Main>

Nix

To build the project the following Nix config is sufficient:

let
  lib =
    { src = ./src;
      dependencies = [ "wreq" "lens" ];
      extensions = [ "OverloadedStrings"];
    };
in
  { main = "Main";
    src = ./app;
    packages = [ lib ];
    dependencies = [ "wreq" "lens" ];
  }

Building and running the project is as simple as

$ snack run

Alternatively, use $ snack build or $ snack ghci if you only want to build, or fire up ghci, respectively.

Using other versions of GHC and nixpkgs

The snack executable comes with a bundled version of nixpkgs and uses the GHC executable provided by haskell.packages.ghc864. You may override those defaults by providing a snack.nix:

# By default ./snack.nix is used if detected
$ snack build
# But you can also explicitly pass the snack file
$ snack --snack-nix my-snack.nix build

Customize the GHC version

# snack.nix
rec {
  # If you only wish to change the version of GHC being used, set
  # `ghc-version`. The following versions are currently available:
  #  * ghc822
  #  * ghc822Binary
  #  * ghc844
  #  * ghc863Binary
  #  * ghc864
  #  * ghc865
  #  * ghcHEAD
  # NOTE: not all versions have been tested with snack.
  ghc-version = "ghc802";
}

Customize the haskellPackages

# snack.nix
rec {
  # Alternatively you can provide your own `haskellPackages`, which should have
  # the same structure as that provided by
  # `pkgs.haskell.packages.<version>:
  haskellPackages = pkgs.haskell.packages.ghc822;
}

Customize the pkgs

# snack.nix
rec {
  # Finally you can customize the whole nixpkgs used
  pkgs = import ./nix {};
}

Advanced Nix Example

You may want custom builds that involve things such as archiving and base64 encoding entire directories.

snack builds itself, so its package.nix is a good example of an advanced configuration. You can also check out the test folder.

Why should I use Snack?

There are plenty of Haskell build tools around (Cabal, Stack, Bazel, ...). Unfortunately none of these allow what I consider to be an ideal workflow:

  1. The same build tool is used by developers and on CI.
  2. The build tool guarantees that builds are reproducible.
  3. The builds are incremental, i.e. if a library contains 300 modules and I modify the main function, only the Main module will be rebuilt.

Using Cabal inside of Nix solves (2); however this means that the builds are not incremental anymore (3). This may not be a problem on CI but definitely is when developing locally. The way to work around that is to use Cabal inside a nix-shell locally and call cabal2nix on CI. This means that developers use a different tool locally than on CI (1). Moreover, a lot of projects nowadays use Stack and, somewhat more importantly, Stackage LTSs. This makes local builds quite easy (in spite of the occasional rebuild when changing flags) but in order to perform a Nix build one has to generate some Nix boilerplate through tools like stackage2nix or stack2nix (which do not always work on CI).

In comparison, snack performs the exact same build on the developer's machine as on CI. The builds are incremental, maybe more so than Cabal builds: if you depend on a snack package foo from package bar, and modify a module Foo from foo which isn't used in bar, no recompilation will occur. Moreover, you benefit from your CI's cache. Finally, because snack is just Nix (and works with the Nix sandbox) you have pretty good guarantees that your builds are reproducible.

Caveat Emptor

The snack library and executable are in their very early stages. They need a lot of testing and massaging. The main (advertised) features are there, but (1) may break for your particular project and (2) may break more in the future.

Now that this is out of the way, install snack, break it, and help me improve it!

Hacking

There are two different components you can hack:

Make sure you have a working version of snack installed, e.g.

$ git co master
$ nix-env -f ./default.nix -iA snack-exe

If you are hacking on the snack executable, just start snack in a GHCi session:

$ snack ghci -p ./bin/package.nix
Temporarily symlinking /nix/store/j1x5vkxjr2ibabddfkdih4sm4kwinfda-spec-json/spec.json to spec.json...
done.
Temporarily symlinking /nix/store/w42y6dzgfmli9r8kmgh8akqk6kyda31x-lib64/lib.tar.gz.b64 to lib.tar.gz.b64...
done.
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Main             ( /home/nicolas/projects/nmattia/snack/bin/Snack.hs, interpreted )
Ok, one module loaded.
*Main>

If you are hacking on the library, specify -l/--lib when running snack (this works in GHCi too):

*Main> :main ghci -l ./snack-lib/ -p ./tests/readme/package.nix
GHCi, version 8.2.2: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Lib              ( /home/nicolas/projects/nmattia/snack/tests/readme/src/Lib.hs, interpreted )
[2 of 2] Compiling Main             ( /home/nicolas/projects/nmattia/snack/tests/readme/app/Main.hs, interpreted )
Ok, two modules loaded.
*Main> :main
"\"Category Theory for Programmers\" has been finished!"

Mustn't be afraid to dream a little bigger, darling.

Thanks

Big thanks to

  • zimbatm for brainstorming with me and improving the Nix code.
  • 2mol for showing me how to write understandable READMEs.
  • quite a few people at ZuriHac for giving me ideas and feedback.
  • whomever is willing to help, in advance.

More Repositories

1

niv

Easy dependency management for Nix projects
Haskell
1,529
star
2

homies

linux package management
Lua
108
star
3

wai-lambda

Haskell Webapps on AWS Lambda
Haskell
43
star
4

sorri

Just Add Water direnv support for the nix-shell
Shell
39
star
5

stutter

CLI string generator based on regex-like operations
Haskell
36
star
6

haskell-hot-swap

Hot swapping compiled code while keeping a websocket connection open
Haskell
26
star
7

rulex

A ruby environment for Latex files
Ruby
16
star
8

makefile

Haskell Makefile parser
Haskell
13
star
9

2048-lens

Lens-based 2048-like game
Haskell
10
star
10

haskell-invoice

Haskell generated PDF invoices
Haskell
7
star
11

countries

Haskell contributions by country
Jupyter Notebook
6
star
12

Minotaur

C++ ECS Game Engine using OSG and CEGUI
C++
6
star
13

websters

Fully functional, fully functional webapp for the WebZuri September session
HTML
5
star
14

highcharts-hs

Highcharts, for Haskell
Haskell
5
star
15

nmattia.com

Personal website
Jupyter Notebook
4
star
16

cio

cio: cached HTTP requests for a smooth Jupyter experience!
Haskell
4
star
17

onyx

Rust
3
star
18

pg-shell

PostgreSQL, in a shell.
Nix
2
star
19

pictionary

JavaScript
2
star
20

autoupdate

Update all the things
Shell
2
star
21

halytics

Haskell analytics and report library
Haskell
2
star
22

butler

Butler-suite for typed protocols and state machines
Haskell
2
star
23

soldat-lemmy

soldat lemmy website
CSS
1
star
24

snipcheck

Check embedded snippets
Haskell
1
star
25

giak

Fuzzy finder for cabal executables
Haskell
1
star
26

genesix

Generate files and directory trees with Nix!
Nix
1
star
27

BRDS

Barroom datascience - Haskellers by country
Jupyter Notebook
1
star
28

dune

Nix
1
star
29

nix-reloaded

somewhat typed nix
Haskell
1
star
30

resume

My resume
Nix
1
star
31

svelte-rust-wasm

An example app that puts Rust in the browser
Svelte
1
star
32

smice

Oh that smice
Nix
1
star