merkdir creates Merkle trees from your directories, and can generate inclusion proofs for any files in those directories.
Let's say you have a bunch of files in your ~/Documents folder, some public, some private. Run merkdir on the directory, and you get a root hash which anonymously represents all your documents. Now you can publish that, sign it, timestamp it, whatever you like.
At some later date, you can publish one of these (unchanged) documents along with the inclusion proof generated by merkdir. This will provably connect the file back to root hash that you published/signed/etc in the past.
Using the power of Merkle trees, merkdir allows you to prove things about millions of files at once. It can make signing or timestamping all the files in your home directory efficient and secure.
The real world applications are slim, but interesting. Maybe I'm a whistleblower with thousands of files, but I don't want to release them all right now. This tool allows you to easily prove that a file you release later was part of the original collection.
Another use case might be an organization that has a large archive of important files. This tool allows them to efficiently sign all of these files by just signing the root hash.
Pre-built binaries can be installed from the releases page. Otherwise you can build from source by using just. The Go compiler is of course required.
If you use Homebrew, you can install merkdir through the my personal tap.
brew tap makew0rld/tap
brew install merkdir
You can update it with:
brew upgrade merkdir
# Create a Merkle tree that holds hashes of all your documents
$ merkdir gen -o documents_tree.merkdir ~/Documents
Finding files...
Found 2339 files. Starting hashing...
100% |████████████████████████████████████████| (1.6/1.6 GB, 5.0 GB/s)
Root hash: 3e1db8e48dd101bed67ccd117ad011fa76aca26c38ce1ab1612010d5140618b1
# Now publish that hash, sign it, etc
# If you need it again:
$ merkdir root --hex documents_tree.merkdir
3e1db8e48dd101bed67ccd117ad011fa76aca26c38ce1ab1612010d5140618b1
# Generate a proof for one file
$ merkdir inclusion -t documents_tree.merkdir -f "name/of/file.txt" -o my_proof.merkdir
# Now publish that along with the file itself
# This proves the file is a part of the tree represented by the root hash
# Verify that a file on disk hasn't changed since the tree was generated
$ merkdir verify-file -t my_merkle_tree.merkdir -n "name/of/file.txt"
# Verify an inclusion proof you received
# You get the root hash as output, and must compare it to the expected root hash
$ merkdir verify-inclusion -p some_inclusion_proof.bin -f path/to/file.pdf --hex
3e1db8e48dd101bed67ccd117ad011fa76aca26c38ce1ab1612010d5140618b1
# Or compare it to a root hash directly:
$ merkdir verify-inclusion -p some_inclusion_proof.bin -f path/to/file.pdf --hash "abc123..."
OK: proof and file match given root hash
# Get info on a tree file
$ merkdir info documents_tree.merkdir
Root hash: 3e1db8e48dd101bed67ccd117ad011fa76aca26c38ce1ab1612010d5140618b1
FS root: /home/makeworld/Documents
Num. of files: 2339
Creation time: 2023-12-26 19:33:29 -0500 EST
# Or for a proof, relative to the tree it's from
$ merkdir info -p some_proof.merkdir documents_tree.merkdir
File index: 2137
File name: My Document.ods
Nonce: bc0d0ba51259d0ba0475f754bc4a7cf4
Proof length: 10 hashesAll merkdir output files are CBOR, so they can be easily used by other tools.
merkdir uses the fast and secure BLAKE3 hash algorithm.
Inclusion proofs are designed to be shared publicly, and so don't expose filenames or even direct file hashes. The only information relevant to your filesystem that they reveal is the number of files in the original Merkle tree.
Other tools like IPFS or Iroh also create Merkle trees over directories. The main difference is the merkdir is singularly focused on doing this, and crucially makes generating inclusion proofs easy. Currently I'm not aware of a way to generate inclusion proofs with these tools.
merkdir is licensed under the GPL v3.0. See the LICENSE file for details.