⚠️ This repository works with LLVM versions 4.0 to 8.0 and is now archived. Please find examples for more recent versions in-tree: https://github.com/llvm/llvm-project/blob/main/llvm/examples/OrcV2Examples/

JitFromScratch

Collection of examples from my talks in the LLVM Social Berlin and C++ User Group Berlin that implement various aspects of a JIT compiler based on the LLVM Orc libraries.

Contribute

The repository follows a perfect history policy to foster traceability and understanding. If you find a bug or want to submit an improvement, please file a pull request against the respective step.

Structure

The examples are organized around a nonsense command line program that compiles the code for a simple function at runtime:

template <size_t sizeOfArray>
int *integerDistances(const int (&x)[sizeOfArray], int *y) {
  int items = arrayElements(x);
  int *results = customIntAllocator(items);

  for (int i = 0; i < items; i++) {
    results[i] = abs(x[i] - y[i]);
  }

  return results;
}

Steps

The example project is built in a series of self-contained steps. You can immediatedly explore and build each change in your Browser with GitPod:

Step	Change	Description
0	`87cb540`	Wire up LLVM
1	`82465fc`	Add lit tests
2	`b1b268b`	Add TravisCI config to run tests in prebuilt docker
3	`81a9d8b`	Initialize LLVM for native target codegen
4	`731f482`	Add minimal JIT compiler based on LLJIT
5	`73a565a`	Create empty module and pass it to the JIT
6	`7690ac1`	In debug mode dump extra info when passing -debug -debug-only=jitfromscratch
7	`3b47ce8`	Generate function that takes two ints and returns zero
8	`8a4bba8`	Add basic sanity checks for IR code
9	`402e9d1`	Request and run trivial function
10	`e08ff57`	Emit IR code for substraction and use it in the integerDistances function
11	`8cf690b`	Emit variable names to make IR more readable
12	`03b8701`	Allow symbol resolution from the host process
13	`a0cdaad`	Emit call to stdlib function abs in JITed code
14	`2653289`	Emit unrolled loop in JITed code
15	`2291ee4`	Emit call to allocator in JITed code
16	`bf72028`	Remove wrapper function and call JITed code directly
17	`96f4c8e`	Break free from LLJIT
18	`a4b88a0`	Implement GDB JIT interface
19	`b7212c3`	Add optimization passes controlled via -Ox command line flag
20	`75dff62`	Simple name-based object cache

Build and run locally

$ git clone https://github.com/weliveindetail/JitFromScratch jitfromscratch
$ mkdir build && cd build
$ cmake -GNinja -DLLVM_DIR=/path/to/llvm-build/lib/cmake/llvm ../jitfromscratch
$ ninja run

The project was tested on Ubuntu 19.04 and macOS 10.14. Please find real-world examples for commands and expected output on Travis CI.

Build and run in docker

$ git clone https://github.com/weliveindetail/JitFromScratch jitfromscratch
$ cd jitfromscratch
$ docker run -it -v $(pwd):/host/jitfromscratch -e REMOTE=/host/jitfromscratch -e CHECKOUT=$(git rev-parse HEAD) weliveindetail/jitfromscratch:llvm09-test-release

Further images can be pulled from dockerhub or created manually based on the given Dockerfiles.

Setup vscode

Install the C/C++, CMake Tools and CodeLLDB extensions and use the given configuration files.

Previous Versions

You can easily diff for a specific 8.0 to 9.0 change set, e.g.:

$ git diff origin/llvm08/master origin/llvm09/master
$ git diff origin/llvm08/steps/A00 origin/llvm09/steps/A00 -- CMakeLists.txt
$ git diff origin/llvm08/steps/A18 origin/llvm09/steps/A18 -- JitFromScratch.cpp

In LLVM 8, the Orc library was almost entirely rewritten and I used the opportunity to also refine my examples. You can still find the old examples here:

weliveindetail/JitFromScratch

weliveindetail

Reviews

Repository Details