SharpFuzz: AFL-based fuzz testing for .NET

SharpFuzz is a tool that brings the power of afl-fuzz to .NET platform. If you want to learn more about fuzzing, my motivation for writing SharpFuzz, the types of bugs it can find, or the technical details about how the integration with afl-fuzz works, read my blog post SharpFuzz: Bringing the power of afl-fuzz to .NET platform.

CVE
Articles
Trophies
Requirements
Installation
Usage
Advanced topics
Acknowledgements

CVE

Articles

Trophies

If you find some interesting bugs with SharpFuzz, and are comfortable with sharing them, I would love to add them to this list. Please send me an email, make a pull request for the README file, or file an issue.

Requirements

AFL works on Linux and macOS. If you are using Windows, you can use any Linux distribution that works under the Windows Subsystem for Linux. For native Windows support, you can use libFuzzer instead of AFL.

You will need GNU make and a working compiler (gcc or clang) in order to compile afl-fuzz. You will also need to have the .NET Core 2.1 or greater installed on your machine in order to instrument .NET assemblies with SharpFuzz.

To simplify your fuzzing experience, it's also recommended to install PowerShell.

Installation

You can install afl-fuzz and SharpFuzz.CommandLine global .NET tool by running the following script:

#/bin/sh
set -eux

# Download and extract the latest afl-fuzz source package
wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
tar -xvf afl-latest.tgz

rm afl-latest.tgz
cd afl-2.52b/

# Install afl-fuzz
sudo make install
cd ..
rm -rf afl-2.52b/

# Install SharpFuzz.CommandLine global .NET tool
dotnet tool install --global SharpFuzz.CommandLine

Usage

This tutorial assumes that you are somewhat familiar with afl-fuzz. If you don't know anything about it, you should first read the AFL quick start guide and the afl-fuzz README. If you have enough time, I would also recommend reading Understanding the status screen and Technical whitepaper for afl-fuzz.

As an example, we are going to fuzz Jil, which is a fast JSON serializer and deserializer (see SharpFuzz.Samples for many more examples of complete fuzzing projects).

1. Create a new .NET console project, then add Jil and SharpFuzz packages to it by running the following commands:

dotnet add package Jil
dotnet add package SharpFuzz

2. In your Main function, call SharpFuzz.Fuzzer.OutOfProcess.Run with the function that you want to test as a parameter:

using System;
using System.IO;
using SharpFuzz;

namespace Jil.Fuzz
{
  public class Program
  {
    public static void Main(string[] args)
    {
      Fuzzer.OutOfProcess.Run(stream =>
      {
        try
        {
          using (var reader = new StreamReader(stream))
          {
            JSON.DeserializeDynamic(reader);
          }
        }
        catch (DeserializationException) { }
      });
    }
  }
}

We want to fuzz the deserialization capabilities of Jil, which is why we are calling the JSON.DeserializeDynamic method. The input data will be provided to us via the stream parameter (if the code you are testing takes its input as a string, you can use an additional overload of Fuzzer.OutOfProcess.Run that accepts Action<string>).

If the code passed to Fuzzer.OutOfProcess.Run throws an exception, it will be reported to afl-fuzz as a crash. However, we want to treat only unexpected exceptions as bugs. DeserializationException is what we expect when we encounter an invalid JSON input, which is why we catch it in our example.

3. Create a directory with some test cases (one test is usually more than enough). Test files should contain some input that is accepted by your code as valid, and should also be as small as possible. For example, this is the JSON I'm using for testing JSON deserializers:

{"menu":{"id":1,"val":"X","pop":{"a":[{"click":"Open()"},{"click":"Close()"}]}}}

4. Let's say that your project is called Fuzzing.csproj and that your test cases are in the Testcases directory. Start fuzzing by running the fuzz.ps1 script like this:

pwsh scripts/fuzz.ps1 Jil.Fuzz.csproj -i Testcases

For formats such as HTML, JavaScript, JSON, or SQL, the fuzzing process can be greatly improved with the usage of a dictionary file. AFL comes with bunch of dictionaries, which you can find after installation in /usr/local/share/afl/dictionaries/. With this in mind, we can improve our fuzzing of Jil like this:

pwsh scripts/fuzz.ps1 Jil.Fuzz.csproj -i Testcases \
  -x /usr/local/share/afl/dictionaries/json.dict

5. Sit back and relax! You will often have some useful results within minutes, but sometimes it can take more than a day, so be patient.

The input files responsible for unhandled exceptions will appear in the findings/crashes directory. The total number of unique crashes will be displayed in red on the afl-fuzz status screen.

In practice, the real number of unique exceptions will often be much lower than the reported number, which is why it's usually best to write a small program that just goes through the crashing inputs, runs the fuzzing function on each of them, and saves only the inputs that produce unique stack traces.

Advanced topics

Acknowledgements

Joe Ranweiler and the MORSE team - libFuzzer support on Windows
Michal Zalewski - american fuzzy lop
Dmitry Vyukov - go-fuzz: randomized testing for Go
Rody Kersten - Kelinci: AFL-based fuzzing for Java
Jb Evain - Mono.Cecil
0xd4d - dnlib
Guido Vranken - go-fuzz: libFuzzer support

Metalnem/sharpfuzz

Metalnem

Reviews

Repository Details