nix-serve-ng
nix-serve-ng
is a faster, more reliable, drop-in replacement for nix-serve
.
Quick start
There are two main approaches you can use to upgrade a NixOS system to replace
the old nix-serve
with nix-serve-ng
.
If you specify your desired NixOS system within flake.nix
then you can do
something like this:
{ inputs = {
nixpkgs.url = github:NixOS/nixpkgs;
nix-serve-ng.url = github:aristanetworks/nix-serve-ng;
};
outputs = { nixpkgs, nix-serve-ng, ... }: {
nixosConfigurations.default = nixpkgs.lib.nixosSystem {
modules = [
nix-serve-ng.nixosModules.default
{ services.nix-serve.enable = true;
…
}
];
system = "x86_64-linux";
};
};
}
If you don't use flake.nix
then you can instead define your NixOS module:
like this:
let
nix-serve-ng-src = builtins.fetchTarball {
# Replace the URL and hash with whatever you actually need
url = "https://github.com/aristanetworks/nix-serve-ng/archive/1937593598bb1285b41804f25cd6f9ddd4d5f1cb.tar.gz";
sha256 = "1lqd207gbx1wjbhky33d2r8xi6avfbx4v0kpsvn84zaanifdgz2g";
};
nix-serve-ng = import nix-serve-ng-src;
in
{ ... }: {
imports = [ nix-serve-ng.nixosModules.default ];
…
}
Motivation
Our requirements for this project were:
-
Improve reliability
… since
nix-serve
would intermittently hang and require restarts -
Improve efficiency
… since
nix-serve
was doing some obviously inefficient things which we felt we could improve upon -
Be backwards-compatible
Our replacement would need to be a drop-in replacement for the original
nix-serve
, supporting the same command-line options and even sharing the same executable nameThe only exception is logging: we provide more detailed logging than before
Did we satisfy those requirements?
Results
-
Reliability
We have test-driven this internally under heavy load with stable memory usage and without any failures but it's probably premature to declare victory.
In particular, we have not done the following things:
-
Memory leak detection
In other words, we haven't put our
nix-serve
through, say,valgrind
-
Exploit detection
In other words, we haven't attempted to crash or sabotage the service with maliciously-crafted payload
-
-
Performance
We have improved significantly on efficiency, not only compared to
nix-serve
but also compared to othernix-serve
rewrites. We are more efficient than:See the Benchmarks section below for more details
-
Backwards-compatibility
We have excellent backwards-compatibility, so in the vast majority of cases, you can simply replace
pkgs.nix-serve
withpkgs.nix-serve-ng
and make no other changes.-
Our executable shares the same name (
nix-serve
) as the original program -
We support most the original command-line options
The options that we're aware of that we do not currently support fall into two categories:
-
Useless options which are only relevant to
starman
:Upon request, we can still parse and ignore the following irrelevant options for extra backwards compatibility:
-
--workers
We do not use worker subprocess like
starman
does. Instead we usewarp
which internally uses Haskell green threads to service a much larger number of requests with less overhead and lower footprint when idle. -
--preload-app
This optimization is meaningless for a compiled Haskell executable.
-
--disable-proctitle
-
-
Useful options
We might accept requests to support the following options, but we might explore other alternatives first before supporting them:
-
--max-requests
warp
itself is unlikely to be a bottleneck to servicing a large number of requests but there may still be Nix-specific or disk-specific reasons to cap the number of requests. -
--disable-keepalive
-
--keepalive-timeout
-
--read-timeout
-
--user
-
--group
-
--pid
-
--error-log
-
-
Because of this backwards-compatibility you only need to replace the old
nix-serve
executable with thenix-serve
executable built by this package (which is what the included NixOS module does).You don't need to define or use any new NixOS options. You continue to use the old
services.nix-serve
options hierarchy to configure the upgraded service. -
Benchmarks
The test environment is a large server machine:
- CPU: 24 × Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
- RAM: 384 GB (24 × 16 GB @ 2133 MT/s)
- Disk (
/nix/store
): ≈4 TB SSD
Legend:
- Fetch present NAR info ×10: Time to fetch the NAR info for 10 different files that are present
- Fetch absent NAR info ×1: Time to fetch the NAR info a single file that is absent
- Fetch empty NAR ×10: Time to fetch the NAR for the same empty file 10 times
- Fetch 10 MB NAR ×10: Time to fetch the NAR for the same 10 MB file 10 times
Raw numbers:
Benchmark | nix-serve |
eris |
harmonia |
nix-serve-ng |
---|---|---|---|---|
Fetch present NAR info ×10 | 2.09 ms ± 66 μs | 41.5 ms ± 426 μs | 1.57 ms ± 91 μs | 1.32 ms ± 33 μs |
Fetch absent NAR info ×1 | 212 μs ± 18 μs | 3.42 ms ± 113 μs | 139 μs ± 11 μs | 115 μs ± 6.2 μs |
Fetch empty NAR ×10 | 164 ms ± 8.5 ms | 246 ms ± 20 ms | 279 ms ± 10 ms | 5.16 ms ± 368 μs |
Fetch 10 MB NAR ×10 | 291 ms ± 8.7 ms | 453 ms ± 19 ms | 487 ms ± 41 ms | 86.9 ms ± 3.0 ms |
Speedups (compared to nix-serve
):
Benchmark | nix-serve |
eris |
harmonia |
nix-serve-ng |
---|---|---|---|---|
Fetch present NAR info ×10 | 1.0 | 0.05 | 1.33 | 1.58 |
Fetch absent NAR info ×1 | 1.0 | 0.06 | 1.53 | 1.84 |
Fetch empty NAR ×10 | 1.0 | 0.67 | 0.59 | 31.80 |
Fetch 10 MB NAR ×10 | 1.0 | 0.64 | 0.60 | 3.35 |
We can summarize nix-serve-ng
's performance like this:
- Time to handle a NAR info request: ≈ 100 μs
- Time to serve a NAR: ≈ 500 μs + 800 μs / MB
You can reproduce these benchmarks using the benchmark suite. See the
instructions in ./benchmark/Main.hs
for running your
own benchmarks.
Caveats:
- We haven't used any of these services' tuning options, including:
- Tuning garbage collection (for
nix-serve-ng
) - Tuning concurrency/parallelism/workers
- Tuning garbage collection (for
- We haven't benchmarked memory utilization