• Stars
    star
    741
  • Rank 61,194 (Top 2 %)
  • Language
    Rust
  • License
    MIT License
  • Created about 1 year ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The fastest hashing algorithm ๐Ÿ“ˆ

GxHash

Build & Test

GxHash is a blazingly fast and robust non-cryptographic hashing algorithm.

Usage

cargo add gxhash

Used directly as a hash function:

let bytes: &[u8] = "hello world".as_bytes();
let seed = 1234;
println!(" 32-bit hash: {:x}", gxhash::gxhash32(&bytes, seed));
println!(" 64-bit hash: {:x}", gxhash::gxhash64(&bytes, seed));
println!("128-bit hash: {:x}", gxhash::gxhash128(&bytes, seed));

Used in HashMap/HashSet:

// Type alias for HashSet::<String, GxBuildHasher>
let mut hashset = gxhash::GxHashSet::default();
hashset.insert("hello world");

Features

Blazingly Fast ๐Ÿš€

Up to this date, GxHash is the fastest non-cryptographic hashing algorithm of its class, for all input sizes. This performance is possible mostly thanks to heavy usage of SIMD intrinsics, high ILP construction and a small bytecode (easily inlined and cached).
See the benchmarks.

Highly Robust ๐Ÿ—ฟ

GxHash uses several rounds of hardware-accelerated AES block cipher for efficient bit mixing.
Thanks to this, GxHash passes all SMHasher tests, which is the de facto quality benchmark for non-cryptographic hash functions, gathering most of the existing algorithms. GxHash has low collisions, uniform distribution and high avalanche properties.

Check out the paper for more technical details.

Portability

Architecture Compatibility

GxHash is compatible with:

  • X86 processors with AES-NI intrinsics
  • ARM processors with NEON intrinsics

Warning Other platforms are currently not supported (there is no fallback). The behavior on these platforms is undefined.

Hashes Stability

All generated hashes for a given version of GxHash are stable, meaning that for a given input the output hash will be the same across all supported platforms. An exception to this is the AVX2 version of GxHash (nightly).

Benchmarks

To run the benchmarks locally use one of the following:

# Benchmark throughput
cargo bench --bench throughput
# Benchmark performance of GxHash's Hasher when used in a HashSet
cargo bench --bench hashset
# Benchmark throughput and get output as a markdown table
cargo bench --bench throughput --features bench-md
# Benchmark throughput and get output as .svg plots
cargo bench --bench throughput --features bench-plot

GxHash is continuously benchmarked on X86 and ARM Github runners.
Benchmark

Lastest Benchmark Results:
aarch64 x86_64 x86_64-avx2

Security

DOS Resistance

GxHash is a seeded hashing algorithm, meaning that depending on the seed used, it will generate completely different hashes. The default HasherBuilder (GxHasherBuilder::default()) uses seed randomization, making any HashMap/HashSet more DOS resistant, as it will make it much more difficult for attackers to be able to predict which hashes may collide without knowing the seed used. This does not mean however that it is completely DOS resistant. This has to be analyzed further.

Multicollisions Resistance

GxHash uses a 128-bit internal state (and even 256-bit with the avx2 feature). This makes GxHash a widepipe construction when generating hashes of size 64-bit or smaller, which had amongst other properties to be inherently more resistant to multicollision attacks. See this paper for more details.

Cryptographic Properties

GxHash is a non-cryptographic hashing algorithm, thus it is not recommended to use it as a cryptographic algorithm (it is not a replacement for SHA). It has not been assessed if GxHash is preimage resistant and how difficult it is to be reversed.

Contributing

  • Feel free to submit PRs
  • Repository is entirely usable via cargo commands
  • Versioning is the following
    • Major for stability breaking changes (output hashes for a same input are different after changes)
    • Minor for API changes/removal
    • Patch for new APIs, bug fixes and performance improvements

โ„น๏ธ cargo-asm is an easy way to view the actual generated assembly code (cargo asm gxhash::gxhash::gxhash64) (method #[inline] should be removed otherwise it won't be seen by the tool)
โ„น๏ธ AMD ฮผProf gives some useful insights on time spent per instruction.

Publication

Author note: I'm committed to the open dissemination of scientific knowledge. In an era where access to information is more democratized than ever, I believe that science should be freely available to all โ€“ both for consumption and contribution. Traditional scientific journals often involve significant financial costs, which can introduce biases and can shift the focus from purely scientific endeavors to what is currently trendy.

To counter this trend and to uphold the true spirit of research, I have chosen to share my work on "gxhash" directly on GitHub, ensuring that it's openly accessible to anyone interested. Additionally, the use of a free Zenodo DOI ensures that this research is citable and can be referenced in other works, just as traditional publications are.

I strongly believe in a world where science is not behind paywalls, and I am in for a more inclusive, unbiased, and open scientific community.

Publication:
PDF

Cite this publication / algorithm:
DOI

More Repositories

1

grid-shader-unity

A grid shader for Unity ๐ŸŒ
ShaderLab
250
star
2

project-curator

A convenient tool to help cleaning and maintaining Unity Projects ๐Ÿงนโœจ
C#
150
star
3

kmeans-clustering-unity

2D and 3D clustering in Unity using k-means
C#
45
star
4

blender-importer-unity

A tool to fix orientation issues from Blender to Unity
C#
31
star
5

dr-dotnet

๐Ÿฉบ One-click diagnosis of your dotnet applications. Works both locally or remotely as a web service. Based on the lowest level dotnet profiling APIs and using the rust language ๐Ÿฆ€ for a minimal runtime penalty.
Rust
29
star
6

node-graph-wpf

A state machine node graph framework for WPF
C#
20
star
7

normals-effect-unity

A geometry shader to highlight face and vertex normals and tangents. Can also be used as an Image Effect.
ShaderLab
15
star
8

gxhash-csharp

C#
13
star
9

wireframe-effect-unity

A wireframe shader for Unity. Also works as an screen image effect
ShaderLab
13
star
10

tags-for-windows

MacOS style tags for Windows ! ๐ŸŸฉ๐ŸŸจ๐ŸŸฅ๐ŸŸช๐ŸŸฆ
C#
13
star
11

density-effect-unity

ShaderLab
10
star
12

ffidji

๐Ÿถ FFIDJI is a tool to automatically generate bindings between languages, like calling Rust code from C# for instance.
Rust
8
star
13

open-cmd-here

A Windows shell tool to open a new command line window at this location ๐Ÿ–ฅ๏ธ
C#
7
star
14

risu

Blazingly fast and ultra-efficient multi-protocol read-through caching proxy.
Rust
7
star
15

property-adorners-unity

Sometimes, creating a custom Editor for a script in Unity can feel like an overkill. These handy attributes can easily do the job in a lot of situations.
C#
6
star
16

deep-learning-in-csharp

Small C# library to build and run DFF neural networks
C#
5
star
17

brutus

A scalper bot that you can command via discord.
C#
4
star
18

reed

"The green reed which bends in the wind is stronger than the mighty oak which breaks in a storm"
C#
4
star
19

optimized-linked-list

Implementation of a LinkedList using a single continuous memory block (reference / allocation free)
C#
4
star
20

dotnet-dependency-grapher

C#
3
star
21

hash-pass-php

Handy but ultra secure hash based password generation
PHP
2
star
22

grpc-net-client-migration-perf-repro

C#
2
star
23

caching-benchmarks

C#
2
star
24

kollect

Social and competitive waste collection mobile app ๐Ÿ—‘๏ธ (mock-up)
C#
1
star
25

c-sharpe

Don't try this at home C# programming
C#
1
star
26

watchman

1
star
27

quick-version

A very simple Window shell tool to create a file version with a right click !
C#
1
star
28

blanquer3d

TypeScript
1
star
29

com-registration-tool

Tool to register/unregister install/uninstall Windows COM assemblies
C#
1
star
30

hash-pass

Hash-based password generator, implemented with Flutter + Dart
Dart
1
star
31

dot-underscore-io

Read and write ._xxx type MacOS filesystem metadata files
C#
1
star
32

sketch-auto-export

JavaScript
1
star
33

pic-to-lego

Create a LEGO piece of art from a picture !
Python
1
star
34

croissantage-defender

Anti croissantage solution. ๐Ÿฅโš”๏ธ๐Ÿ›ก๏ธ This is a revolution.
Rust
1
star
35

unity-nuget

Unity managed assemblies (UnityEngine.dll, UnityEditor.dll, ...) as NuGet packages to simplify development of C# libraries for Unity 3D
Batchfile
1
star
36

ogxd.github.io

HTML
1
star