• Stars
    star
    120
  • Rank 295,983 (Top 6 %)
  • Language
    Solidity
  • License
    Apache License 2.0
  • Created almost 2 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Benchmark Generator for Smart-Contract Fuzzers

Daedaluzz: A Benchmark Generator for Smart-Contract Fuzzers

Daedaluzz is a tool for automatically generating benchmarks for smart-contract fuzzers. The benchmark-generation approach takes inspiration from the Fuzzle benchmark generator for C-based fuzzers.

A key goal is to make it possible to compare as many different fuzzers as possible. For this reason, the benchmarks intentionally use a limited subset of Solidity to avoid language features that some tools could handle differently (or not at all).

Benchmark-Generation Approach

Each generated benchmark contract contains many assertions (some can fail, but others cannot due to infeasible path conditions). A fuzzer can create sequences of transactions that make those assertions fail. We can measure a fuzzer's performance by the number of distinct assertion violations that are found.

On a high level, each contract keeps track of the current position in a 2-dimensional maze of a fixed dimension (for instance, 7x7). Each transaction can move the current position to explore the maze. Some locations in the maze are unreachable (so-called "walls"), while others may contain a "bug" that can be found by the fuzzer when providing specific transaction inputs. Some of these bugs cannot be reached since the path conditions are infeasible.

The generated benchmarks try to capture two key challenges when fuzzing smart contracts:

  1. code that can only be reached by satisfying complex transaction-input constraints
  2. code that can only be reached by first reaching a specific state of the contract through multiple prior transactions

In the future, we may extend the benchmark generator to incorporate other challenges we often encounter when fuzzing real-world contracts. We are more than happy to discuss ideas for such extensions!

Benchmark Examples

We have used Daedaluzz to generate a set of 5 benchmark contracts (see generated-mazes folder). Yet, one can easily generate new ones by modifying the random seed and other hyperparameters (for instance, numFuncParams, dimX, and maxDepth).

One can also tweak the benchmark generator to emit custom benchmark variants for fuzzers that do not support the default benchmarks. For instance, we have used it to produce custom benchmark instances for the Foundry/Forge fuzzer; see the foundry-code-generation branch and the files ending in .foundry.sol in the generated-mazes folder.

Benchmarking Infrastructure

The repository also includes infrastructure (for instance, scripts and Dockerfiles) to run several popular fuzzers on the generated benchmarks:

The run-all-fuzzers.sh script can be used to benchmark the above fuzzers, and the show-stats.py script can be used to aggregate and visualize the results.

Frequently Asked Questions

Why do the generated benchmarks use inputs of type uint64?

There are three main reasons:

  1. We observed that some fuzzers—specifically, ones that use constraint solvers—struggle with arithmetic operations of larger bit width (e.g., uint256). We wanted the benchmarks to be compatible with as many fuzzers as possible.
  2. By defaulting to 64-bit arithmetic, we could easily tweak Daedaluzz to generate benchmarks for other languages, such as C, and compare with even more fuzzers.
  3. One can easily increase the number of input bytes (potentially increasing the difficulty of benchmarks) by adjusting the numFuncParams parameter.

We are open to revisiting this design choice in the future.

How did you come up with the current hyperparameters?

We tried several settings but did not exhaustively evaluate each hyperparameter. The main goal was to find a configuration that produces challenging benchmarks that fit into the EVM code size limit.

We are open to making changes to these hyperparameters in the future.

We want to avoid that fuzzer developers overfit their tools to this specific version of the benchmarks. For this reason, we are considering adjusting the benchmarks regularly (for instance, once or twice a year). Benchmark generation tools, such as Daedaluzz, simplify this task significantly.

Will a fuzzer that performs well on these benchmarks also perform well for my code?

Not necessarily. The only way to know is to try the fuzzer on your code. If you have the time, you should always try as many fuzzers as possible. However, it is probably not worth your time to try a fuzzer on your code if it performs poorly on these benchmarks.

More Repositories

1

smart-contract-best-practices

A guide to smart contract security best practices
HTML
7,473
star
2

ethereum-developer-tools-list

A guide to available tools and platforms for developing on Ethereum.
5,321
star
3

quorum

A permissioned implementation of Ethereum supporting data privacy
Go
4,581
star
4

mythril

Security analysis tool for EVM bytecode. Supports smart contracts built for Ethereum, Hedera, Quorum, Vechain, Rootstock, Tron and other EVM-compatible blockchains.
Python
3,817
star
5

Tokens

Ethereum Token Contracts
JavaScript
2,059
star
6

eth-lightwallet

Lightweight JS Wallet for Node and the browser
JavaScript
1,466
star
7

gnark

gnark is a fast zk-SNARK library that offers a high-level API to design circuits. The library is open source and developed under the Apache 2.0 license
Go
1,383
star
8

surya

A set of utilities for exploring Solidity contracts
JavaScript
1,076
star
9

ethql

A GraphQL interface to Ethereum 🔥
TypeScript
623
star
10

abi-decoder

Nodejs and Javascript library for decoding data params and events from ethereum transactions
JavaScript
605
star
11

vscode-solidity-auditor

Solidity language support and visual security auditor for Visual Studio Code
JavaScript
568
star
12

teku

Java Implementation of the Ethereum 2.0 Beacon Chain
Java
557
star
13

cakeshop

An integrated development environment and SDK for Ethereum-like ledgers
JavaScript
507
star
14

gnark-crypto

gnark-crypto provides elliptic curve and pairing-based cryptography on BN, BLS12, BLS24 and BW6 curves. It also provides various algorithms (algebra, crypto) of particular interest to zero knowledge proof systems.
Go
487
star
15

Token-Factory

Basic Token Factory dapp.
JavaScript
479
star
16

constellation

Peer-to-peer encrypted message exchange
Haskell
379
star
17

UniversalToken

Implementation of Universal Token for Assets and Payments
JavaScript
346
star
18

doc.linea

Linea documentation
JavaScript
338
star
19

quorum-examples

Examples for Quorum
Shell
317
star
20

scribble

Scribble instrumentation tool
TypeScript
315
star
21

anonymous-zether

A private payment system for Ethereum-based blockchains, with no trusted setup.
Solidity
295
star
22

defi-score

DeFi Score: An open framework for evaluating DeFi protocols
Python
280
star
23

EthOn

EthOn - The Ethereum Ontology
HTML
245
star
24

solidity-metrics

Solidity Code Metrics
JavaScript
235
star
25

Mahuta

IPFS Storage service with search capability
Java
230
star
26

tessera

Tessera - Enterprise Implementation of Quorum's transaction manager
Java
169
star
27

PLCRVoting

Partial Lock Commit Reveal Voting System that utilizes ERC20 Tokens
JavaScript
169
star
28

ethjsonrpc

Python JSON-RPC client for the Ethereum blockchain
Python
156
star
29

linea-attestation-registry

Verax is a shared registry for storing attestations of public interest on EVM chains, designed to enhance data discoverability and consumption for dApps across the network.
TypeScript
129
star
30

zero-knowledge-proofs

Zero Knowledge Proofs and how they can be implemented in Quorum
C++
128
star
31

evm-dafny

An EVM interpreter in Dafny
Dafny
125
star
32

python-solidity-parser

An experimental Solidity parser for Python built on top of a robust ANTLR4 grammar 📚
Python
125
star
33

truffle-security

MythX smart contract security verification plugin for Truffle Framework
JavaScript
124
star
34

solc-typed-ast

A TypeScript package providing a normalized typed Solidity AST along with the utilities necessary to generate the AST (from Solc) and traverse/manipulate it.
TypeScript
123
star
35

web3signer

Web3Signer is an open-source signing service capable of signing on multiple platforms (Ethereum1 and 2, Filecoin) using private keys stored in an external vault, or encrypted on a disk.
Java
122
star
36

btcrelay-fetchd

Just the fetchd script of btcrelay
Python
116
star
37

ethsigner

DEPRECATED. A transaction signing application to be used with a web3 provider.
Java
112
star
38

ethereum-dissectors

🔍Wireshark dissectors for Ethereum devp2p protocols
C
109
star
39

quorum-dev-quickstart

The Quorum Developer Quickstart utility can be used to rapidly generate local Quorum blockchain networks for development and demo purposes using Besu, GoQuorum, and Codefi Orchestrate.
Solidity
108
star
40

truffle-webpack-demo

A demo Webpack + React App using truffle-solidity-loader
JavaScript
95
star
41

orion

Orion is a PegaSys component for doing private transactions
Java
92
star
42

blockchainSecurityDB

JavaScript
90
star
43

quorum-kubernetes

Helm charts for Hyperledger Besu and GoQuorum
Mustache
85
star
44

quorum-docs

Documentation assets for Quorum
84
star
45

mythx-cli

A command line interface for the MythX smart contract security analysis API
Python
83
star
46

gpact

General Purpose Atomic Crosschain Transaction Protocol
Java
81
star
47

bytecode-verifier

Compile Solidity source code and verify its bytecode matches the blockchain
JavaScript
80
star
48

goff

goff (go finite field) is a unix-like tool that generates fast field arithmetic in Go.
Go
76
star
49

starknet-snap

The MetaMask Snap for Starknet
TypeScript
74
star
50

linea-contracts

Linea smart-contracts
Solidity
73
star
51

eth2.0-dafny

Eth2.0 spec in Dafny
Dafny
72
star
52

zsl-q

ZSL on Quorum
C++
71
star
53

security-workshop-for-devs

Secure smart contract development workshop hosted by ConsenSys Diligence and MythX.
70
star
54

Legions

Ethereum/EVM Node Security Toolkit
Python
69
star
55

support-metamask-io

Public-facing repository of content live on support.metamask.io. Open for contributions and suggestions.
MDX
66
star
56

quorum-docker-Nnodes

Run a bunch of Quorum nodes, each in a separate Docker container.
Shell
65
star
57

Project-Alchemy

Ethereum-Zcash Integration effort
63
star
58

handel

Multi-Signature Aggregation in a Large Byzantine Committees
Go
53
star
59

qubernetes

Quorum on Kubernetes.
Go
52
star
60

Uniswap-audit-report-2018-12

51
star
61

quorum-tools

Tools for running Quorum clusters and integration tests
Haskell
51
star
62

doc.teku

ConsenSys Ethereum 2.0 client
CSS
47
star
63

vscode-solidity-metrics

Generate Solidity Source Code Metrics, Complexity and Risk profile reports for your project.
JavaScript
46
star
64

private-networks-deployment-scripts

This repository contains out-of-the-box deployment scripts for private PoA networks
Shell
45
star
65

awesome-quorum

A curated list of awesome softwares, libraries, tools, articles, educational resources, discussion channels and more to build on ConsenSys Quorum.
45
star
66

wittgenstein

Simulator for some PoS or consensus algorithms. Includes dfinity, casper IMD and others
Java
45
star
67

linea-tutorials

An EVM-equivalent zk-rollup for scaling Ethereum dapps
Shell
43
star
68

vscode-ethover

Ethereum Account Address Hover Info and Actions
JavaScript
42
star
69

permissioning-smart-contracts

Smart contracts for the Besu permissioning system
TypeScript
41
star
70

besu-sample-networks

Hyperledger Besu Ethereum client quick-start makes you able to simply test all Besu features.
40
star
71

linea-token-list

Linea Token List
TypeScript
39
star
72

0x-review

Security review of 0x smart contracts
HTML
39
star
73

quorum-key-manager

A universal Key & Account Management solution for blockchain applications.
Go
39
star
74

mythx-playground

Exercises to go along with smart contract security workshops by MythX and ConsenSys Diligence
Solidity
39
star
75

kubernetes-action

GitHub Action to run kubectl
Dockerfile
38
star
76

evm-analyzer-benchmark-suite

A benchmark suite for evaluating the precision of EVM code analysis tools.
HTML
38
star
77

quorum-cloud

Deploy Quorum network in a cloud provider of choice
HCL
36
star
78

quorum.js

Quorum.js is an extension to web3.js providing support for JP Morgan's Quorum API
JavaScript
36
star
79

web3js-eea

EEA JavaScript libraries.
JavaScript
35
star
80

truffle-solidity-loader

A Webpack loader that will parse and provision Solidity files to Javascript using Truffle for compilation
JavaScript
35
star
81

secureum-diligence-bootcamp

Solidity
35
star
82

rimble-app-demo

React Ethereum dApp demonstrating onboarding and transaction UX
JavaScript
35
star
83

linea-monorepo

The principal Linea repository. This mainly includes the smart contracts covering Linea's core functions, the prover in charge of generating ZK proofs, the coordinator responsible for multiple orchestrations, and the postman to execute bridge messages.
Go
35
star
84

linea-tracer

Part of the Linea stack responsible for extracting data from the execution of an EVM client in order to construct large matrices called execution traces.
Java
34
star
85

infura-sdk

Infura NFT SDK
TypeScript
34
star
86

web3studio-soy

Static Websites on the Distributed Web
JavaScript
33
star
87

pythx

A Python library for the MythX smart contract security analysis platform
Python
33
star
88

quorum-aws

Tools for deploying Quorum clusters to AWS
HCL
33
star
89

react-metamask

JavaScript
32
star
90

diligence-fuzzing

Python
32
star
91

boilerplate-react

React app boilerplate by ConsenSys France
JavaScript
29
star
92

hellhound

HellHound is a decentralized blind computation platform.
Go
29
star
93

aragraph

Visualize your Aragon DAO Templates
JavaScript
29
star
94

quorum-wizard

Quorum Wizard is a command line tool that allow users to set up a development Quorum network on their local machine in less than 2 minutes.
JavaScript
28
star
95

doc.goquorum

Documentation site for GoQuorum, the ConsenSys Enterprise Ethereum client
CSS
27
star
96

quorum-explorer

A light-weight front-end explorer for Besu and GoQuorum to visualise private networks and deploy smart contracts
TypeScript
27
star
97

mythxjs

TypeScript
26
star
98

hackathon-2021-dapp-workshop

JavaScript
25
star
99

0x_audit_report_2018-07-23

0x Protocol v2 Audit
HTML
24
star
100

web3js-quorum

JavaScript
24
star