• Stars
    star
    133
  • Rank 270,987 (Top 6 %)
  • Language
    C#
  • License
    MIT License
  • Created over 8 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

.NET library for xxHash, FNV, MurmurHash3 and SipHash algorithms

HashDepot

NuGet Version Build Status

I have been implementing various hash functions that are absent in .NET framework. I decided to converge them into a library. My primary goals are to provide well-tested and performant implementations. The library currently supports SipHash, MurmurHash3, xxHash and FNV-1a.

To install it on NuGet:

Install-Package HashDepot

Supported Hash Algorithms

I try to add anything that's not in C# runtime and quite popular. For instance, there are multiple xxHash implementations for C# but they differentiate in terms of API complexity and performance. Although I didn't try out SIMD optimizations, the existing code is quite fast.

xxHash

This one claims to be one of the fastest hash functions and it's actually amazing. Even without any SIMD optimizations, it outperforms everything, even a plain checksum by a factor of two. The implementation assumes little endian machines. Example usage:

var buffer = Encoding.UTF8.GetBytes("some string");
uint result = XXHash.Hash32(buffer, seed: 123);
ulong result = XXHash.Hash64(buffer); // default seed is zero

SipHash

SipHash is resistant to hash-flood attacks against hashtables and uses a key parameter to ensure HMAC-like authenticity yet faster. Unfortuantely a native .NET implementation does not exist. It is my take on it, and it is really fast for a managed environment. It's standard SipHash-2-4 implementation with 64-bit. To use it:

var buffer = Encoding.UTF8.GetBytes("some string");
var key = new byte[16] { .. your random key here .. };
ulong result = SipHash24.Hash64(buffer, key);

If you have a larger buffer than 2GB it's better to use streaming functions instead.

MurmurHash3

MurmurHash3 provides a good balance between performance and homogenity but is essentially prone to hash-flood attacks (trivial to force collisions). HashDepot implements its x86 flavor (not x64). An example use is:

var buffer = Encoding.UTF8.GetBytes("some string");
uint seed = // .. preferred seed value ...
uint result = MurmurHash3.Hash32(buffer, seed);

FNV-1a

A straightforward implementation of FNV-1a hash algorithm for .NET. Usage is very simple. For instance to calculate 32-bit FNV-1a hash of ASCII string "some string":

var buffer = Encoding.UTF8.GetBytes("some string");
uint result = Fnv1a.Hash32(buffer); // 32-bit hash
ulong result = Fnv1a.Hash64(buffer); // 64-bit hash

Streaming and Async functions

All hashes also provide stream-based (albeit slow) functions with their async variants too. In order to get the hash of a stream just call the function with a stream instead of a memory buffer:

ulong result = XXHash.Hash64(stream);

If you'd like to run it asynchronously, use the async variant:

uint result = await MurmurHash3.Hash32Async(stream);

Benchmarks

Benchmarks are performed on a 1MB buffer.

BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.22621.819) Snapdragon Compute Platform, 1 CPU, 8 logical and 8 physical cores .NET SDK=7.0.100 [Host] : .NET 6.0.11 (6.0.1122.52304), Arm64 RyuJIT AdvSIMD DefaultJob : .NET 6.0.11 (6.0.1122.52304), Arm64 RyuJIT AdvSIMD

Method Mean Error StdDev Median
Checksum_32 234.9 us 4.61 us 7.17 us 231.4 us
XXHash_32 248.3 us 2.13 us 1.66 us 248.3 us
XXHash_64 231.1 us 4.40 us 4.89 us 231.4 us
MurmurHash3_x86 498.9 us 6.19 us 5.79 us 498.5 us
SipHash24_32 497.4 us 4.30 us 3.35 us 498.1 us
Fnv1a_32 1,112.1 us 21.93 us 25.25 us 1,108.1 us
Fnv1a_64 1,064.2 us 10.09 us 8.95 us 1,065.1 us

Contributing

You're more than welcome to contribute fixes or new hash algorithms. Please keep these in mind:

  • Make sure the code builds without warnings.
  • Include unit tests for the fixed bug, or the new feature.
  • If you're proposing a new hash algorithm, please make sure that it's not in C# runtime, there isn't an existing library that is already tremendously popular, and HashDepot's simplistic approach would provide a benefit over the alternatives.

License

MIT License. See LICENSE file for details

More Repositories

1

sozluk-cgi

one of the first versions of ekşi sözlük code as of may 1999
Pascal
235
star
2

SimpleBase

.NET library for encoding/decoding Base16, Base32, Base58 and Base85.
C#
142
star
3

streetcoder

Code examples used in the book Street Coder https://streetcoder.org
JavaScript
83
star
4

TurkishId

Validator/generator for Turkish Republic Citizen ID numbers (TC Kimlik No)
C#
77
star
5

fatalvision

FatalVision DOS-based GUI library for Turbo Pascal
Pascal
48
star
6

wolverine

An offline mail reader for FidoNet-style networks supporting BlueWave and QWK formats.
Pascal
27
star
7

NoJoy

Tool for quickly enabling/disabling game controllers
C#
25
star
8

PathCleaner

Cleanup tool for polluted PATH environment variable
C#
22
star
9

yubitell

Silently extract a YubiKey serial number
HTML
20
star
10

trid

Parser and data type for Turkish Citizenship ID numbers
Rust
19
star
11

unbom

Mass remove UTF-8 BOM markers from text files
C#
18
star
12

path

Command-line tool to manage PATH environment variable
C#
16
star
13

mazecpc

Amstrad CPC Edition of "10 PRINT" maze generator challenge (12 bytes)
Assembly
15
star
14

ssgmatrix

The Matrix screensaver
Pascal
14
star
15

Genesis

Windows 10 Initial Configuration Script
PowerShell
14
star
16

eksi-yedek

ekşi sözlük XML backup reader
JavaScript
12
star
17

genav

DOS anti virus software
Pascal
11
star
18

cinali

cinali internet downloader
C#
10
star
19

screenfetch-ps

PowerShell version of screenfetch
PowerShell
8
star
20

tweet_deleter

JavaScript
8
star
21

Eksi.Tasks

A set of NAnt commands to wrap JSLint, JSMin and CSSMin
JavaScript
7
star
22

poly

MRP Software for PVC Window Manufacturers
Pascal
6
star
23

xdream

An old IRC script for XiRCON
Tcl
6
star
24

soc

ssg's own commander (yet another total commander clone)
Pascal
5
star
25

ExeTools

PowerShell
4
star
26

act2txt

Tool to convert Adobe Photoshop ACT palette files to Paint.NET format
C#
4
star
27

ssg

profile repository
3
star
28

xmlite

Unsued sample remover from FastTracker II .XM files.
Pascal
3
star
29

melchior

simple network status monitoring tool
Pascal
3
star
30

simplex

Simplex maximizer
C#
3
star
31

yello

A Hello World application that is a hybrid Z80 and BASIC code
Assembly
2
star
32

amele

Tool to convert EF6 [Index] attributes to EF Core declarative syntax
C#
2
star
33

run-ssg-run

128-byte intro released in 7DX 2012 demoscene party
Assembly
2
star
34

xcode

my xcode settings
2
star
35

nojoy-rs

Rust
1
star
36

TurboSurfer

Pascal
1
star
37

ourobs

HTML
1
star
38

Move-Shadow

PowerShell cmdlets to move folders transparently between drives
PowerShell
1
star