• Stars
    star
    327
  • Rank 125,528 (Top 3 %)
  • Language
    Rust
  • License
    GNU General Publi...
  • Created over 4 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Independent verification of binary packages - reproducible builds

rebuilderd(1) crates.io cncf slack irc.libera.chat:6697/#archlinux-reproducible

Independent verification system of binary packages.

rebuildctl pkgs ls example output

rebuilderd monitors the package repository of a linux distribution and uses rebuilder backends like archlinux-repro to verify the provided binary packages can be reproduced from the given source code.

It tracks the state of successfully verified packages and optionally generates a report of differences with diffoscope for debugging. Note that due to the early state of this technology a failed rebuild is more likely due to an undeterministic build process instead of a supply chain compromise, but if multiple rebuilders you trust report 100% reproducible for the set of packages you use you can be confident that the binaries on your system haven't been tampered with. People are encouraged to run their own rebuilders if they can afford to.

Status

Status Docker Doesn't need --privileged Doesn't need /dev/kvm Backend
Arch Linux βœ”οΈ supported ❌ - βœ”οΈ archlinux-repro
Debian πŸš€ experimental βœ”οΈ ❌ βœ”οΈ debrebuild.py
Tails πŸš€ experimental ❌ - ❌ docs (script)
Alpine ✨ planned - - - -

Docker: There's a docker-compose example setup in this repository, but not all rebuilder backends support running inside of a docker container (for example because it's creating containers itself).

Doesn't need --privileged: Some rebuilder backends create containers in a way that works inside of a docker container, if they're granted the required kernel capabilities to do so. This may have security implications for other containers running on that system or the code running inside the container may reconfigure the system outside of the docker container.

Doesn't need /dev/kvm: Some build tools may need to start a virtual machine and depend on /dev/kvm to be available. This is a special requirement for the hosting environment, you either need a VPS with Nested KVM or dedicated non-virtualized hardware.

Accessing a rebuilderd instance in your browser

Many instance run a web frontend to display their results. rebuilderd-website is a very good choice and the software powering the Arch Linux rebuilderd instance:

https://reproducible.archlinux.org/

Loading the index of all packages may take a short time.

Scripting access to a rebuilderd instance

Packaging status

It's also possible to query and manage a rebuilderd instance in a scriptable way. It's recommended to install the rebuildctl commandline util to do this (instructions for your system may vary, see packaging status to the right):

pacman -S rebuilderd-tools

You can then query a rebuilderd instance for the status of a specific package:

rebuildctl -H https://reproducible.archlinux.org pkgs ls --name rebuilderd

You have to specify which instance you want to query because there's no definite truthβ„’. You could ask multiple instances though, including one you operate yourself.

If the rebuilder seems to have outdated data or lists a package as unknown the update may still be in the build queue. You can query the build queue of an instance like this:

rebuildctl -H https://reproducible.archlinux.org queue ls --head

If there's no output that means the build queue is empty.

If you're the administrator of this instance you can also run commands like:

rebuildctl status

Or immediately retry all failed rebuild attempts (there's an automatic retry on by default):

rebuildctl pkgs requeue --status BAD --reset

Running a rebuilderd instance yourself

journalctl output of a rebuilderd-worker

"I compile everything from source" - a significant amount of real world binary packages can already be reproduced today. The more people run rebuilders, the harder it is to compromise all of them.

At the current stage of the project we're interested in every rebuilder there is! Most rebuilderd discussion currently happens in #archlinux-reproducible on libera, feel free to drop by if you're running a instance or considering setting one up. Having a few unreproducible packages is normal (even if it's slightly more than the official rebuilder), but having additional people confirm successful rebuilds is very helpful.

Rebuilding Arch Linux

Please see the setup instructions in the Arch Linux Wiki.

Development with docker

There is a docker-compose setup in the repo, to start a basic stack simply clone the repository and run:

DOCKER_BUILDKIT=1 docker-compose up

The initial build is going to take some time.

To recompile your changes (you can optionally specify a specific image to build):

DOCKER_BUILDKIT=1 docker-compose build

The auth cookie has strict permissions, for development simply change them with:

sudo chmod 0644 secret/auth

Check you can successfully run administrative tasks, use this command to compile and run the rebuildctl binary:

REBUILDERD_COOKIE_PATH=secret/auth cargo run -p rebuildctl -- -v status

There are no packages in the database yet, there's an example profile that we can load. It only contains one lightweight package and should successfully rebuild out-of-the-box in our docker-compose setup.

REBUILDERD_COOKIE_PATH=secret/auth cargo run -p rebuildctl -- pkgs sync-profile --sync-config contrib/confs/rebuilderd-sync.conf debian-anarchism

Check the package was successfully added to the database with status UNKWN:

REBUILDERD_COOKIE_PATH=secret/auth cargo run -p rebuildctl -- pkgs ls

You can display the build queue with this command, it's also going to display a timer for jobs that are currently in progress:

REBUILDERD_COOKIE_PATH=secret/auth cargo run -p rebuildctl -- queue ls --head

You can use a combination of the commands mentioned to monitor your rebuilder. The packages should eventually show up as GOOD in rebuildctl pkgs ls.

Development

If you want to build from source or you want to run rebuilderd built from a specific commit this section contains instructions for that.

A rebuilder consists of the rebuilderd daemon and >= 1 workers:

First we switch into the daemon/ folder and run our rebuilderd daemon:

cd daemon; cargo run

This takes a moment but the api should now be available at https://127.0.0.1:8484/api/v0/dashboard.

This daemon needs to run in the background, so we're starting a new terminal to continue with the next steps.

Next we're going to build the rebuilctl binary and confirm it's able to connect to the api. If we don't get an error message this means it's working.

cd tools; cargo run -- status

We didn't connect any workers yet so this output is empty.

Next we want to connect a rebuilder. rebuilderd only does the scheduling for you, so you need to install additional software here (called a rebuilder backend):

  • Arch Linux: pacman -S archlinux-repro or git clone https://github.com/archlinux/archlinux-repro && archlinux-repro/ && make && sudo make install. Note that on debian buster you need to install systemd from buster-backports.

With a rebuilder backend installed we're now going to run our first worker:

cd worker; cargo run -- connect http://127.0.0.1:8484

This rebuilder should now show up in our rebuildctl status output:

cd tools; cargo run -- status

Next we're going to import some packages:

cd tools; cargo run -- pkgs sync archlinux community \
    'https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch' \
    --architecture x86_64 --maintainer kpcyrd

The --maintainer option is optional and allows you to rebuild packages by a specific maintainer only.

To show the current status of our imported packages run:

cd tools; cargo run -- pkgs ls

To monitor your workers are picking up tasks:

cd tools; cargo build && CLICOLOR_FORCE=1 watch -c ../target/debug/rebuildctl status

To inspect the queue run:

cd tools; cargo run -- queue ls

An easy way to test the package import is using a command like this:

cargo watch -- cargo run --bin rebuildctl -- pkgs sync-profile --print-json --sync-config contrib/confs/rebuilderd-sync.conf tails

Build a package directly:

cargo run --bin rebuilderd-worker -- \
	build debian 'http://deb.debian.org/debian/pool/main/a/anarchism/anarchism_15.3-3_all.deb' \
	--input-url 'https://buildinfos.debian.net/buildinfo-pool/a/anarchism/anarchism_15.3-3_all.buildinfo' \
	--backend 'debian=./rebuilder-debian.sh'

Dependencies

Debian: pkg-config liblzma-dev libssl-dev libsqlite3-dev libzstd-dev

Funding

Rebuilderd development is currently funded by:

  • kpcyrd's savings account
  • Google and The Linux Foundation
  • People like you and me on github sponsors

License

GPLv3+

More Repositories

1

sn0int

Semi-automatic OSINT framework and package manager
Rust
1,559
star
2

sniffglue

Secure multithreaded packet sniffer
Rust
958
star
3

rshijack

tcp connection hijacker, rust rewrite of shijack
Rust
395
star
4

authoscope

Scriptable network authentication cracker (formerly `badtouch`)
Rust
359
star
5

mini-docker-rust

Very small rust docker image
Dockerfile
171
star
6

spotify-launcher

Client for spotify's apt repository in Rust for Arch Linux
Rust
152
star
7

i-probably-didnt-backdoor-this

A practical experiment on supply-chain security using reproducible builds
Dockerfile
148
star
8

nude-rs

High performance nudity detection in rust
Rust
125
star
9

sh4d0wup

Signing-key abuse and update exploitation framework
Rust
119
star
10

libredefender

Imagine the information security compliance guideline says you need an antivirus but you run Arch Linux
Rust
117
star
11

ipfs.ink

PROJECT HAS BEEN SHUTDOWN - Publish and render markdown essays to and from ipfs
JavaScript
109
star
12

pacman-bintrans

Experimental binary transparency for pacman with sigstore and rekor
Rust
83
star
13

boxxy-rs

Linkable sandbox explorer
Rust
70
star
14

acme-redirect

Tiny http daemon that answers acme challenges and redirects everything else to https
Rust
68
star
15

arch-audit-gtk

Arch Linux Security Update Notifications
Rust
55
star
16

narnia

🚧 EXPERIMENTAL 🚧 Secure hidden service webserver
Rust
49
star
17

yrd

cjdns swiss army knife
Python
48
star
18

repro-env

Dependency lockfiles for reproducible build environments πŸ“¦πŸ”’
Rust
33
star
19

archlinux-userland-fs-cmp

Forensic tool to read all installed packages from a mounted Arch Linux drive and compare the filesystem to a trusted source
Rust
32
star
20

defcon26-pow

Fast defcon 26 quals pow solver
Rust
26
star
21

backseat-signed

Authenticate the cryptographic chain-of-custody of Linux distributions (like Arch Linux and Debian) to their source code inputs
Rust
25
star
22

what-the-src

Source code of https://whatsrc.org/
Rust
24
star
23

progpick

Bruteforce with a stream of permutations of a specific pattern
Rust
23
star
24

syscallz-rs

Simple seccomp library for rust
Rust
22
star
25

sn0int-modules

Lua
21
star
26

cargo-debstatus

cargo-tree for debian packaging
Rust
20
star
27

tr1pd

tamper resistant audit log
Rust
17
star
28

forensic-adb

Tokio based client library for the Android Debug Bridge (adb) based on mozdevice
Rust
16
star
29

snail

Parasitic network manager
Rust
15
star
30

rocket_failure

Semantic error handling for rocket applications
Rust
15
star
31

auth-tarball-from-git

Authenticate a tarball through a signed tag in a git repository (with reproducible builds)
Rust
15
star
32

apt-swarm

πŸ₯Έ p2p gossip network for update transparency, based on pgp πŸ₯Έ
Rust
15
star
33

worker-ratelimit

General purpose rate limiting library for Cloudflare Workers
Rust
14
star
34

laundry5

Shuffles your socks - rotating proxy frontend server
Rust
13
star
35

kmod-rs

Bindings to libkmod to manage linux kernel modules
Rust
13
star
36

ismyarchverifiedyet

🚧 Experimental script to query rebuilderd for results 🚧
Python
13
star
37

chrootable-https

Sandbox+chroot friendly https client
Rust
12
star
38

brchd

Data exfiltration toolkit
Rust
12
star
39

nmcssh

Solving Zooko's triangle for ssh authentication
Python
11
star
40

updlockfiles

Manage lockfiles in PKGBUILDs for upstreams that don't ship them, `updpkgsums` for dependency trees (Arch Linux tooling)
Rust
11
star
41

booty

Minimal forensic/exfiltration/evil-maid/rescue live boot system
Shell
10
star
42

burritun

Wrap a tun device in a tap device
Rust
10
star
43

archlinux-inputs-fsck

Lint repository of PKGBUILDs for cryptographically pinned inputs
Rust
10
star
44

rp2040-37c3-oled

Pure Rust firmware for 37c3 logo animation (waveshare-rp2040-zero with 128x64 oled screen - i2c sda: gpio14, scl: gpio15)
Rust
10
star
45

nessus-rs

Nessus Vulnerability Scanner API client
Rust
8
star
46

a2p

fancy html5 file upload, webrtc seeding swarm, auto torrent and scp interface
JavaScript
7
star
47

homeassistant-rs

home-assistant api client
Rust
6
star
48

masshype

Util for massive cjdns routers
JavaScript
6
star
49

memry

mem'ry, tar pipe curl
JavaScript
6
star
50

stalkerware-indicators-rs

Parser for Echap's stalkerware-indicators repo
Rust
6
star
51

summarize-cli

Attempt to summarize text from `stdin`, using a large language model (locally and offline), to `stdout`
Rust
6
star
52

signal-whois

Resolve a signal username or link to a signal uuid
Rust
6
star
53

sloppy-rfc4880

Pure rust parser for RFC-4880 (OpenPGP Message Format)
Rust
5
star
54

rebuilderd-debian-buildinfo-crawler

Reproducible Builds: Scraper/Parser for https://buildinfos.debian.net into structured data
Rust
5
star
55

syrup-rs

Simple abstraction around pancurses for chat-like interfaces
Rust
5
star
56

autovoice

irc bot to automatically give +v to users after they've been in the channel for some time
Rust
5
star
57

signal-doh-ech

🚧 Experimental source dump for pluggable transport for signal-desktop, not fully implemented, do not use in production 🚧
Rust
5
star
58

archlinux-linux-reproducible

Binary reproducible fork of the Arch Linux kernel package
Shell
4
star
59

46snihdnat

4 to 6 server name indication hybrid destination network address translation
JavaScript
4
star
60

mrsc

mpsc with requests
Rust
4
star
61

tls.li

Hardened TLS configuration examples
CSS
4
star
62

cjdns-rs

Admin API implementation of cjdns
Rust
4
star
63

os-version

Rust
4
star
64

webhook-server

Multiprocess sandboxed webhook daemon
Rust
4
star
65

ipfs-mirror

ipfs mirror utils with leveldb cache for immutable files
Python
4
star
66

archlinux-scan-malloc-usable-size

Scan the symbols of all ELF binaries in all Arch Linux packages for usage of malloc_usable_size (-D_FORTIFY_SOURCE=3 compatibility)
Rust
4
star
67

game-dont-panic

Pure Rust firmware, bare metal Space Invaders/Endoparasitic crossover game for waveshare-rp2040-zero, with a 128x64 OLED i2c screen, a rotary encoder and a button
Rust
4
star
68

elf2nucleus

Integrate micronucleus into the cargo buildsystem, flash an AVR firmware from an elf file
Rust
4
star
69

promisc

cjdns peering bot
Python
3
star
70

jenkins-debian

personal fork of jenkins.debian.net
Shell
3
star
71

onionjson

Tor2Web for json
HTML
3
star
72

wrbt-web

Web implementation of wrbt
HTML
3
star
73

csrf.fun

Cross Site Request Forgery Debugger
JavaScript
3
star
74

hype-qr

QRify cjdns connect strings
JavaScript
3
star
75

d3xs

Physical access control (Rust firmware)
Rust
3
star
76

annex-accumulate

Super folder for git-annex drives
Python
3
star
77

cloudflare-worker-rust

Build a Hello World WebAssembly web-service with Rust and run it locally with Cloudflare's workerd
Rust
3
star
78

sn0int-signal

Rust
2
star
79

huesaverd

Rust
2
star
80

abuild-reusesig

Rust
2
star
81

embedded-triple

Embed the target triple into the binary
Rust
2
star
82

s2ws

Expose Spawn to WebSockets
JavaScript
2
star
83

ysf-sn0int-modules

my sn0int modules or patches
Lua
2
star
84

attiny85-hello-world

Hello World Rust firmware for digispark attiny85 microcontroller
Rust
2
star
85

BadCrypto

A challenge for my future self
Python
1
star
86

shepard

The hackers monitoring
Python
1
star
87

labsh

Restricted shell for docker build server
Rust
1
star
88

dotfiles

Shell
1
star
89

waflz

Link preview irc bot
Rust
1
star
90

scdoc

personal mirror
C
1
star
91

PKGBUILD-acmetool

Shell
1
star
92

rust-diesel-bug-2365

Rust
1
star
93

kpcyrd

1
star
94

pkgbuild-signal-desktop

Send pull requests for the signal-desktop Arch Linux package here
Shell
1
star
95

updvcspins

Manage pinned VCS repositories in PKGBUILDs (Arch Linux tooling)
Rust
1
star
96

wrbt-httpd

Authorize peering requests on remote servers
Python
1
star
97

namecoin-zones

Converts the namecoin blockchain to dns zones
Python
1
star
98

not-butter

there is no butter
JavaScript
1
star
99

aur-repro

Reproducible Builds for packages in the Arch User Repository (AUR)
Shell
1
star
100

iam

Simple whois server implementation
Shell
1
star