• Stars
    star
    1,358
  • Rank 34,598 (Top 0.7 %)
  • Language
    Rust
  • License
    Other
  • Created about 4 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple multi-profile Nix-flake deploy tool.

deploy-rs logo


A Simple, multi-profile Nix-flake deploy tool.

Questions? Need help? Join us on Matrix: #deploy-rs:matrix.org

Usage

Basic usage: deploy [options] <flake>.

Using this method all profiles specified in the given <flake> will be deployed (taking into account the profilesOrder).

Optionally the flake can be constrained to deploy just a single node (my-flake#my-node) or a profile (my-flake#my-node.my-profile).

If your profile or node name has a . in it, simply wrap it in quotes, and the flake path in quotes (to avoid shell escaping), for example 'my-flake#"myserver.com".system'.

Any "extra" arguments will be passed into the Nix calls, so for instance to deploy an impure profile, you may use deploy . -- --impure (note the explicit flake path is necessary for doing this).

You can try out this tool easily with nix run:

  • nix run github:serokell/deploy-rs your-flake

If you want to deploy multiple flakes or a subset of profiles with one invocation, instead of calling deploy <flake> you can issue deploy --targets <flake> [<flake> ...] where <flake> is supposed to take the same format as discussed before.

Running in this mode, if any of the deploys fails, the deploy will be aborted and all successful deploys rolled back. --rollback-succeeded false can be used to override this behavior, otherwise the auto-rollback argument takes precedent.

If you require a signing key to push closures to your server, specify the path to it in the LOCAL_KEY environment variable.

Check out deploy --help for CLI flags! Remember to check there before making one-time changes to things like hostnames.

There is also an activate binary though this should be ignored, it is only used internally (on the deployed system) and for testing/hacking purposes.

Ideas

deploy-rs is a simple Rust program that will take a Nix flake and use it to deploy any of your defined profiles to your nodes. This is strongly based off of serokell/deploy, designed to replace it and expand upon it.

Multi-profile

This type of design (as opposed to more traditional tools like NixOps or morph) allows for lesser-privileged deployments, and the ability to update different things independently of each other. You can deploy any type of profile to any user, not just a NixOS profile to root.

Magic Rollback

There is a built-in feature to prevent you making changes that might render your machine unconnectable or unusuable, which works by connecting to the machine after profile activation to confirm the machine is still available, and instructing the target node to automatically roll back if it is not confirmed. If you do not disable magicRollback in your configuration (see later sections) or with the CLI flag, you will be unable to make changes to the system which will affect you connecting to it (changing SSH port, changing your IP, etc).

API

Overall usage

deploy-rs is designed to be used with Nix flakes. There is a Flake-less mode of operation which will automatically be used if your available Nix version does not support flakes, however you will likely want to use a flake anyway, just with flake-compat (see this wiki page for usage).

deploy-rs also outputs a lib attribute, with tools used to make your definitions simpler and safer, including deploy-rs.lib.${system}.activate (see later section "Profile"), and deploy-rs.lib.${system}.deployChecks which will let nix flake check ensure your deployment is defined correctly.

There are full working deploy-rs Nix expressions in the examples folder, and there is a JSON schema here which is used internally by the deployChecks mentioned above to validate your expressions.

A basic example of a flake that works with deploy-rs and deploys a simple NixOS configuration could look like this

{
  description = "Deployment for my server cluster";

  # For accessing `deploy-rs`'s utility Nix functions
  inputs.deploy-rs.url = "github:serokell/deploy-rs";

  outputs = { self, nixpkgs, deploy-rs }: {
    nixosConfigurations.some-random-system = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [ ./some-random-system/configuration.nix ];
    };

    deploy.nodes.some-random-system.profiles.system = {
        user = "root";
        path = deploy-rs.lib.x86_64-linux.activate.nixos self.nixosConfigurations.some-random-system;
    };

    # This is highly advised, and will prevent many possible mistakes
    checks = builtins.mapAttrs (system: deployLib: deployLib.deployChecks self.deploy) deploy-rs.lib;
  };
}

In the above configuration, deploy-rs is built from the flake, not from nixpkgs. To take advantage of the nixpkgs binary cache, the deploy-rs package can be overwritten in an overlay:

{
  # ...
  outputs = { self, nixpkgs, deploy-rs }: let
    system = "x86_64-linux";
    # Unmodified nixpkgs
    pkgs = import nixpkgs { inherit system; };
    # nixpkgs with deploy-rs overlay but force the nixpkgs package
    deployPkgs = import nixpkgs {
      inherit system;
      overlays = [
        deploy-rs.overlay # or deploy-rs.overlays.default
        (self: super: { deploy-rs = { inherit (pkgs) deploy-rs; lib = super.deploy-rs.lib; }; })
      ];
    };
  in {
    # ...
    deploy.nodes.some-random-system.profiles.system = {
        user = "root";
        path = deployPkgs.deploy-rs.lib.activate.nixos self.nixosConfigurations.some-random-system;
    };
  };
}

Profile

This is the core of how deploy-rs was designed, any number of these can run on a node, as any user (see further down for specifying user information). If you want to mimic the behaviour of traditional tools like NixOps or Morph, try just defining one profile called system, as root, containing a nixosSystem, and you can even similarly use home-manager on any non-privileged user.

{
  # A derivation containing your required software, and a script to activate it in `${path}/deploy-rs-activate`
  # For ease of use, `deploy-rs` provides a function to easily add the required activation script to any derivation
  # Both the working directory and `$PROFILE` will point to `profilePath`
  path = deploy-rs.lib.x86_64-linux.activate.custom pkgs.hello "./bin/hello";

  # An optional path to where your profile should be installed to, this is useful if you want to use a common profile name across multiple users, but would have conflicts in your node's profile list.
  # This will default to `"/nix/var/nix/profiles/system` if `user` is `root` and profile name is `system`,
  # `/nix/var/nix/profiles/per-user/root/$PROFILE_NAME` if profile name is different.
  # For non-root profiles will default to /nix/var/nix/profiles/per-user/$USER/$PROFILE_NAME if `/nix/var/nix/profiles/per-user/$USER` already exists,
  # and `${XDG_STATE_HOME:-$HOME/.local/state}/nix/profiles/$PROFILE_NAME` otherwise.
  profilePath = "/home/someuser/.local/state/nix/profiles/someprofile";

  # ...generic options... (see lower section)
}

Node

This defines a single node/server, and the profiles you intend it to run.

{
  # The hostname of your server. Can be overridden at invocation time with a flag.
  hostname = "my.server.gov";

  # An optional list containing the order you want profiles to be deployed.
  # This will take effect whenever you run `deploy` without specifying a profile, causing it to deploy every profile automatically.
  # Any profiles not in this list will still be deployed (in an arbitrary order) after those which are listed
  profilesOrder = [ "something" "system" ];

  profiles = {
    # Definition format shown above
    system = {};
    something = {};
  };

  # ...generic options... (see lower section)
}

Deploy

This is the top level attribute containing all of the options for this tool

{
  nodes = {
    # Definition format shown above
    my-node = {};
    another-node = {};
  };

  # ...generic options... (see lower section)
}

Generic options

This is a set of options that can be put in any of the above definitions, with the priority being profile > node > deploy

{
  # This is the user that deploy-rs will use when connecting.
  # This will default to your own username if not specified anywhere
  sshUser = "admin";

  # This is the user that the profile will be deployed to (will use sudo if not the same as above).
  # If `sshUser` is specified, this will be the default (though it will _not_ default to your own username)
  user = "root";

  # Which sudo command to use. Must accept at least two arguments:
  # the user name to execute commands as and the rest is the command to execute
  # This will default to "sudo -u" if not specified anywhere.
  sudo = "doas -u";

  # This is an optional list of arguments that will be passed to SSH.
  sshOpts = [ "-p" "2121" ];

  # Fast connection to the node. If this is true, copy the whole closure instead of letting the node substitute.
  # This defaults to `false`
  fastConnection = false;

  # If the previous profile should be re-activated if activation fails.
  # This defaults to `true`
  autoRollback = true;

  # See the earlier section about Magic Rollback for more information.
  # This defaults to `true`
  magicRollback = true;

  # The path which deploy-rs will use for temporary files, this is currently only used by `magicRollback` to create an inotify watcher in for confirmations
  # If not specified, this will default to `/tmp`
  # (if `magicRollback` is in use, this _must_ be writable by `user`)
  tempPath = "/home/someuser/.deploy-rs";

  # Build the derivation on the target system. 
  # Will also fetch all external dependencies from the target system's substituters.
  # This default to `false`
  remoteBuild = true;

  # Timeout for profile activation.
  # This defaults to 240 seconds.
  activationTimeout = 600;

  # Timeout for profile activation confirmation.
  # This defaults to 30 seconds.
  confirmTimeout = 60;
}

Some of these options can be provided during deploy invocation to override default values or values provided in your flake, see deploy --help.

About Serokell

deploy-rs is maintained and funded with ā¤ļø by Serokell. The names and logo for Serokell are trademark of Serokell OƜ.

We love open source software! See our other projects or hire us to design, develop and grow your idea!

More Repositories

1

universum

šŸŒŒ Prelude written in @Serokell
Haskell
176
star
2

importify

šŸ‘¾ Importi.fy ā€” it's like Uber, but for Haskell modules.
Haskell
105
star
3

nix-npm-buildpackage

Build nix packages that use npm/yarn
Nix
101
star
4

vault-secrets

NixOS tooling for Hashicorp Vault
Nix
71
star
5

tezos-packaging

Various forms of Tezos software distribution for Linux and macOS.
Python
60
star
6

haskell-with-utf8

Get your IO right on the first try
Haskell
52
star
7

xrefcheck

Check cross-references in repository documents
Haskell
51
star
8

o-clock

āŒ› Type-safe time units in Haskell
Haskell
49
star
9

nix-templates

Nix Flake templates for various languages
Nix
48
star
10

serokell-util

Various functions which are used across multiple @serokell packages
Haskell
44
star
11

ariadne

An open source Ada wallet for users who need more
Haskell
40
star
12

systemd-nix

Generate systemd units from NixOS-style descriptions
Nix
38
star
13

servant-util

Servant servers utilities
Haskell
34
star
14

foundry

Morte IDE
Haskell
28
star
15

update-daemon

Nix Flake Update daemon
Rust
25
star
16

serokell.nix

Serokell Nix infrastructure library
Nix
24
star
17

hackage-search

An application that lets you search for anything on Hackage
Haskell
24
star
18

ghc.dev

The https://ghc.dev Website Generator
Haskell
21
star
19

hse-haskell-course-src

Haskell course in the Higher School of Economics
TeX
21
star
20

log-warper

Logging library to provide more convenient, extremely configurable but simple monadic interface with pretty output
Haskell
19
star
21

nixcon2020-talk

NixCon 2020 talk about Nix flakes
Lua
18
star
22

time-warp

Distributed systems execution emulation
Haskell
18
star
23

notion-to-markdown

Export your Notion.so page as markdown
Python
17
star
24

blog-posts

The place for submitting new articles to the Serokell blog.
Haskell
16
star
25

gemini-infra

Internal Services Cluster
Nix
16
star
26

nix-pandoc

An expression to build Pandoc documents with Nix
Nix
16
star
27

haskell-crypto

Haskell cryptography done right
Haskell
15
star
28

fift-asm-dsl

A Haskell eDSL for the TON VM Assembler
Haskell
14
star
29

deploy

(WIP) Simple deploy of multi-profile systems using flakes
Shell
13
star
30

edna

An open-source tool for data analysis aimed to help researchers with their experiments.
Haskell
13
star
31

avl-plus

AVL+ tree
Haskell
12
star
32

style

Collection of programming style guides used in Serokell
12
star
33

cardano-sl

Cryptographic currency implementing Ouroboros PoS protocol
Haskell
12
star
34

upload-daemon

A daemon that asynchronously copies paths to a remote store.
Haskell
11
star
35

blockchain-util

Haskell
10
star
36

private-tezos-blockchain

Scripts for running private Tezos blockchain
Shell
10
star
37

qtah

Qt bindings for Haskell.
Haskell
9
star
38

rscoin

Moved to https://github.com/input-output-hk/rscoin-haskell
9
star
39

mix-to-nix

Generate sandbox buildable Nix expression from Mix lock file
Nix
9
star
40

tzbot

Timezone bot for Slack
Haskell
7
star
41

derivery

Continuous integration and delivery hook for Nix projects
Erlang
7
star
42

tztime

Safe timezone-aware handling of time
Haskell
6
star
43

github-app

Haskell API for authenticating as a GitHub App
Haskell
6
star
44

lootbox

Toolbox for your cool project
Haskell
6
star
45

xrefcheck-action

GitHub action for xrefcheck
Shell
6
star
46

opam-nix

A handy nix library to package OCaml software from OPAM repositories
Haskell
5
star
47

stack-to-nix

Generate sandbox buildable Nix expression from Stack file
Nix
5
star
48

GPLVMHaskell

Gaussian Process Latent Variable Models implemented in Haskell
Haskell
4
star
49

hermetic

Slack bot that links to YouTrack issues
Elixir
4
star
50

coffer

Multi-backend password store with multiple frontends
Haskell
4
star
51

time-warp-nt

Moved to https://github.com/input-output-hk/cardano-sl/tree/master/networking
4
star
52

nyan-interpolation

Flexible and extensible interpolation for Haskell
Haskell
4
star
53

gui-haskell-app

An example of making a GUI app in Haskell
Haskell
4
star
54

pegasus-infra

Web App Cluster
Nix
4
star
55

aeson-options

Various options to configure field and constructor names in JSON
Haskell
4
star
56

srk-nixpkgs

Moved to https://github.com/input-output-hk/iohk-ops
3
star
57

lightning-network

A Haskell interface for the Lightning Network
Haskell
3
star
58

cobwebhook

Set of webhook Plug middleware
Elixir
3
star
59

ton-paychan

Fast and cheap off-chain micro-transactions for the Telegram Open Network
Makefile
3
star
60

metatemplates

A template repo with common files used in Serokell
Makefile
2
star
61

haskell-algorand-sdk

A Haskell SDK for the Algorand blockchain
Haskell
2
star
62

vendix

libc shim that proxies /nix/store FS calls to a vendored location
C
2
star
63

ton.nix

A Nix overlay for building the Telegram Open Network tools
Nix
2
star
64

nixage

Dismantle stack because who needs it when we have Nix and Stackage
Haskell
2
star
65

serokell-closure

Overlaid Nixpkgs closure for Serokell build systems
Nix
2
star
66

co-log-sys

Syslog implementation on top of co-log-core
Haskell
2
star
67

overcight

A tool that will tell you how well your CI pipelines exercise the files in the repository
Haskell
2
star
68

serokell-stackage

Stackage for your Nix
HTML
2
star
69

github-ops-access

Make sure a certain team always has admin access to all repositories in the organisation
Haskell
1
star
70

rebar3-to-nix

Generate buildable Nix expression from Rebar3 lock file
Erlang
1
star
71

junkscraper

GitHub pull request branch garbage collector
Erlang
1
star
72

avl-example

An example implementation of blockchain using AVL-plus
Haskell
1
star
73

serokell.github.io

Moved to https://serokell.io
1
star
74

tezos-globacap

Haskell
1
star
75

frontend-task

Description of the old test task for frontend developers
1
star
76

async-combinators

Async combinators
Haskell
1
star
77

weird-ghc

Reproduction of a bug in GHC: https://gitlab.haskell.org/ghc/ghc/-/issues/16608
Haskell
1
star
78

hsblst

Haskell bindings and high-level interface to BLST -- a BLS12-381 cryptography library
Haskell
1
star
79

lambda_cache

Polling zero-arity function cache
Elixir
1
star
80

mon

Serokell monitoring library
Haskell
1
star
81

nixos-packet

iPXE images and configuration tools for NixOS deployments on Packet
Nix
1
star
82

stakerdao-infra

Machines and resources for the part of the StakerDAO infrastructure operated by Serokell OƜ
HCL
1
star
83

zipp

Universal zipper (iterator)
Haskell
1
star
84

beamer-theme-serokell

a beamer theme that tries to conform to Serokell brand identity guide
TeX
1
star
85

zeromq-haskell

Fork of the https://gitlab.com/twittner/zeromq-haskell
Haskell
1
star
86

config_macro

Macro that defines Mix.Config.config/3 compatible getters
Elixir
1
star
87

haskell-nix-weeder

Nix library to run weeder on haskell projects built with haskell.nix
Nix
1
star
88

trac11042

Trac #11042 GHC bug reproduction
Nix
1
star
89

morley-zurihac

Hacking on developer tools for the smart contract language of Tezos with Serokell and TQ
CSS
1
star