• Stars
    star
    10,532
  • Rank 3,262 (Top 0.07 %)
  • Language
    C
  • License
    MIT License
  • Created over 5 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

mimalloc is a compact general purpose allocator with excellent performance.

mimalloc

ย 

mimalloc (pronounced "me-malloc") is a general purpose allocator with excellent performance characteristics. Initially developed by Daan Leijen for the runtime systems of the Koka and Lean languages.

Latest release tag: v2.1.4 (2024-04-22).
Latest v1 tag: v1.8.4 (2024-04-22).

mimalloc is a drop-in replacement for malloc and can be used in other programs without code changes, for example, on dynamically linked ELF-based systems (Linux, BSD, etc.) you can use it as:

> LD_PRELOAD=/usr/lib/libmimalloc.so  myprogram

It also includes a robust way to override the default allocator in Windows. Notable aspects of the design include:

  • small and consistent: the library is about 8k LOC using simple and consistent data structures. This makes it very suitable to integrate and adapt in other projects. For runtime systems it provides hooks for a monotonic heartbeat and deferred freeing (for bounded worst-case times with reference counting). Partly due to its simplicity, mimalloc has been ported to many systems (Windows, macOS, Linux, WASM, various BSD's, Haiku, MUSL, etc) and has excellent support for dynamic overriding. At the same time, it is an industrial strength allocator that runs (very) large scale distributed services on thousands of machines with excellent worst case latencies.
  • free list sharding: instead of one big free list (per size class) we have many smaller lists per "mimalloc page" which reduces fragmentation and increases locality -- things that are allocated close in time get allocated close in memory. (A mimalloc page contains blocks of one size class and is usually 64KiB on a 64-bit system).
  • free list multi-sharding: the big idea! Not only do we shard the free list per mimalloc page, but for each page we have multiple free lists. In particular, there is one list for thread-local free operations, and another one for concurrent free operations. Free-ing from another thread can now be a single CAS without needing sophisticated coordination between threads. Since there will be thousands of separate free lists, contention is naturally distributed over the heap, and the chance of contending on a single location will be low -- this is quite similar to randomized algorithms like skip lists where adding a random oracle removes the need for a more complex algorithm.
  • eager page purging: when a "page" becomes empty (with increased chance due to free list sharding) the memory is marked to the OS as unused (reset or decommitted) reducing (real) memory pressure and fragmentation, especially in long running programs.
  • secure: mimalloc can be built in secure mode, adding guard pages, randomized allocation, encrypted free lists, etc. to protect against various heap vulnerabilities. The performance penalty is usually around 10% on average over our benchmarks.
  • first-class heaps: efficiently create and use multiple heaps to allocate across different regions. A heap can be destroyed at once instead of deallocating each object separately.
  • bounded: it does not suffer from blowup [1], has bounded worst-case allocation times (wcat) (upto OS primitives), bounded space overhead (~0.2% meta-data, with low internal fragmentation), and has no internal points of contention using only atomic operations.
  • fast: In our benchmarks (see below), mimalloc outperforms other leading allocators (jemalloc, tcmalloc, Hoard, etc), and often uses less memory. A nice property is that it does consistently well over a wide range of benchmarks. There is also good huge OS page support for larger server programs.

The documentation gives a full overview of the API. You can read more on the design of mimalloc in the technical report which also has detailed benchmark results.

Enjoy!

Branches

  • master: latest stable release (based on dev-slice).
  • dev: development branch for mimalloc v1. Use this branch for submitting PR's.
  • dev-slice: development branch for mimalloc v2. This branch is downstream of dev (and is essentially equal to dev except for src/segment.c)

Releases

Note: the v2.x version has a different algorithm for managing internal mimalloc pages (as slices) that tends to use reduce memory usage and fragmentation compared to mimalloc v1.x (especially for large workloads). Should otherwise have similar performance (see below); please report if you observe any significant performance regression.

  • 2024-04-22, v1.8.4, v2.1.4: Fixes various bugs and build issues. Add MI_LIBC_MUSL cmake flag for musl builds. Free-ing code is refactored into a separate module (free.c). Mimalloc page info is simplified with the block size directly available (and new block_size_shift to improve aligned block free-ing). New approach to collection of abandoned segments: When a thread terminates the segments it owns are abandoned (containing still live objects) and these can be reclaimed by other threads. We no longer use a list of abandoned segments but this is now done using bitmaps in arena's which is more concurrent (and more aggressive). Abandoned memory can now also be reclaimed if a thread frees an object in an abandoned page (which can be disabled using mi_option_abandoned_reclaim_on_free). The option mi_option_max_segment_reclaim gives a maximum percentage of abandoned segments that can be reclaimed per try (=10%).

  • 2023-04-24, v1.8.2, v2.1.2: Fixes build issues on freeBSD, musl, and C17 (UE 5.1.1). Reduce code size/complexity by removing regions and segment-cache's and only use arenas with improved memory purging -- this may improve memory usage as well for larger services. Renamed options for consistency. Improved Valgrind and ASAN checking.

  • 2023-04-03, v1.8.1, v2.1.1: Fixes build issues on some platforms.

  • 2023-03-29, v1.8.0, v2.1.0: Improved support dynamic overriding on Windows 11. Improved tracing precision with asan and Valgrind, and added Windows event tracing ETW (contributed by Xinglong He). Created an OS abstraction layer to make it easier to port and separate platform dependent code (in src/prim). Fixed C++ STL compilation on older Microsoft C++ compilers, and various small bug fixes.

  • 2022-12-23, v1.7.9, v2.0.9: Supports building with asan and improved Valgrind support. Support arbitrary large alignments (in particular for std::pmr pools). Added C++ STL allocators attached to a specific heap (thanks @vmarkovtsev). Heap walks now visit all object (including huge objects). Support Windows nano server containers (by Johannes Schindelin,@dscho). Various small bug fixes.

  • 2022-11-03, v1.7.7, v2.0.7: Initial support for Valgrind for leak testing and heap block overflow detection. Initial support for attaching heaps to a speficic memory area (only in v2). Fix realloc behavior for zero size blocks, remove restriction to integral multiple of the alignment in alloc_align, improved aligned allocation performance, reduced contention with many threads on few processors (thank you @dposluns!), vs2022 support, support pkg-config, .

  • 2022-04-14, v1.7.6, v2.0.6: fix fallback path for aligned OS allocation on Windows, improve Windows aligned allocation even when compiling with older SDK's, fix dynamic overriding on macOS Monterey, fix MSVC C++ dynamic overriding, fix warnings under Clang 14, improve performance if many OS threads are created and destroyed, fix statistics for large object allocations, using MIMALLOC_VERBOSE=1 has no maximum on the number of error messages, various small fixes.

  • 2022-02-14, v1.7.5, v2.0.5 (alpha): fix malloc override on Windows 11, fix compilation with musl, potentially reduced committed memory, add bin/minject for Windows, improved wasm support, faster aligned allocation, various small fixes.

  • Older release notes

Special thanks to:

  • David Carlier (@devnexen) for his many contributions, and making mimalloc work better on many less common operating systems, like Haiku, Dragonfly, etc.
  • Mary Feofanova (@mary3000), Evgeniy Moiseenko, and Manuel Pรถter (@mpoeter) for making mimalloc TSAN checkable, and finding memory model bugs using the genMC model checker.
  • Weipeng Liu (@pongba), Zhuowei Li, Junhua Wang, and Jakub Szymanski, for their early support of mimalloc and deployment at large scale services, leading to many improvements in the mimalloc algorithms for large workloads.
  • Jason Gibson (@jasongibson) for exhaustive testing on large scale workloads and server environments, and finding complex bugs in (early versions of) mimalloc.
  • Manuel Pรถter (@mpoeter) and Sam Gross(@colesbury) for finding an ABA concurrency issue in abandoned segment reclamation. Sam also created the no GIL Python fork which uses mimalloc internally.

Usage

mimalloc is used in various large scale low-latency services and programs, for example:

Building

Windows

Open ide/vs2022/mimalloc.sln in Visual Studio 2022 and build. The mimalloc project builds a static library (in out/msvc-x64), while the mimalloc-override project builds a DLL for overriding malloc in the entire program.

macOS, Linux, BSD, etc.

We use cmake1 as the build system:

> mkdir -p out/release
> cd out/release
> cmake ../..
> make

This builds the library as a shared (dynamic) library (.so or .dylib), a static library (.a), and as a single object file (.o).

> sudo make install (install the library and header files in /usr/local/lib and /usr/local/include)

You can build the debug version which does many internal checks and maintains detailed statistics as:

> mkdir -p out/debug
> cd out/debug
> cmake -DCMAKE_BUILD_TYPE=Debug ../..
> make

This will name the shared library as libmimalloc-debug.so.

Finally, you can build a secure version that uses guard pages, encrypted free lists, etc., as:

> mkdir -p out/secure
> cd out/secure
> cmake -DMI_SECURE=ON ../..
> make

This will name the shared library as libmimalloc-secure.so. Use ccmake2 instead of cmake to see and customize all the available build options.

Notes:

  1. Install CMake: sudo apt-get install cmake
  2. Install CCMake: sudo apt-get install cmake-curses-gui

Single source

You can also directly build the single src/static.c file as part of your project without needing cmake at all. Make sure to also add the mimalloc include directory to the include path.

Using the library

The preferred usage is including <mimalloc.h>, linking with the shared- or static library, and using the mi_malloc API exclusively for allocation. For example,

> gcc -o myprogram -lmimalloc myfile.c

mimalloc uses only safe OS calls (mmap and VirtualAlloc) and can co-exist with other allocators linked to the same program. If you use cmake, you can simply use:

find_package(mimalloc 1.4 REQUIRED)

in your CMakeLists.txt to find a locally installed mimalloc. Then use either:

target_link_libraries(myapp PUBLIC mimalloc)

to link with the shared (dynamic) library, or:

target_link_libraries(myapp PUBLIC mimalloc-static)

to link with the static library. See test\CMakeLists.txt for an example.

For best performance in C++ programs, it is also recommended to override the global new and delete operators. For convenience, mimalloc provides mimalloc-new-delete.h which does this for you -- just include it in a single(!) source file in your project. In C++, mimalloc also provides the mi_stl_allocator struct which implements the std::allocator interface.

You can pass environment variables to print verbose messages (MIMALLOC_VERBOSE=1) and statistics (MIMALLOC_SHOW_STATS=1) (in the debug version):

> env MIMALLOC_SHOW_STATS=1 ./cfrac 175451865205073170563711388363

175451865205073170563711388363 = 374456281610909315237213 * 468551

heap stats:     peak      total      freed       unit
normal   2:    16.4 kb    17.5 mb    17.5 mb      16 b   ok
normal   3:    16.3 kb    15.2 mb    15.2 mb      24 b   ok
normal   4:      64 b      4.6 kb     4.6 kb      32 b   ok
normal   5:      80 b    118.4 kb   118.4 kb      40 b   ok
normal   6:      48 b       48 b       48 b       48 b   ok
normal  17:     960 b      960 b      960 b      320 b   ok

heap stats:     peak      total      freed       unit
    normal:    33.9 kb    32.8 mb    32.8 mb       1 b   ok
      huge:       0 b        0 b        0 b        1 b   ok
     total:    33.9 kb    32.8 mb    32.8 mb       1 b   ok
malloc requested:         32.8 mb

 committed:    58.2 kb    58.2 kb    58.2 kb       1 b   ok
  reserved:     2.0 mb     2.0 mb     2.0 mb       1 b   ok
     reset:       0 b        0 b        0 b        1 b   ok
  segments:       1          1          1
-abandoned:       0
     pages:       6          6          6
-abandoned:       0
     mmaps:       3
 mmap fast:       0
 mmap slow:       1
   threads:       0
   elapsed:     2.022s
   process: user: 1.781s, system: 0.016s, faults: 756, reclaims: 0, rss: 2.7 mb

The above model of using the mi_ prefixed API is not always possible though in existing programs that already use the standard malloc interface, and another option is to override the standard malloc interface completely and redirect all calls to the mimalloc library instead .

Environment Options

You can set further options either programmatically (using mi_option_set), or via environment variables:

  • MIMALLOC_SHOW_STATS=1: show statistics when the program terminates.
  • MIMALLOC_VERBOSE=1: show verbose messages.
  • MIMALLOC_SHOW_ERRORS=1: show error and warning messages.

Advanced options:

  • MIMALLOC_ARENA_EAGER_COMMIT=2: turns on eager commit for the large arenas (usually 1GiB) from which mimalloc allocates segments and pages. Set this to 2 (default) to only enable this on overcommit systems (e.g. Linux). Set this to 1 to enable explicitly on other systems as well (like Windows or macOS) which may improve performance (as the whole arena is committed at once). Note that eager commit only increases the commit but not the actual the peak resident set (rss) so it is generally ok to enable this.
  • MIMALLOC_PURGE_DELAY=N: the delay in N milli-seconds (by default 10) after which mimalloc will purge OS pages that are not in use. This signals to the OS that the underlying physical memory can be reused which can reduce memory fragmentation especially in long running (server) programs. Setting N to 0 purges immediately when a page becomes unused which can improve memory usage but also decreases performance. Setting N to a higher value like 100 can improve performance (sometimes by a lot) at the cost of potentially using more memory at times. Setting it to -1 disables purging completely.
  • MIMALLOC_PURGE_DECOMMITS=1: By default "purging" memory means unused memory is decommitted (MEM_DECOMMIT on Windows, MADV_DONTNEED (which decresease rss immediately) on mmap systems). Set this to 0 to instead "reset" unused memory on a purge (MEM_RESET on Windows, generally MADV_FREE (which does not decrease rss immediately) on mmap systems). Mimalloc generally does not "free" OS memory but only "purges" OS memory, in other words, it tries to keep virtual address ranges and decommits within those ranges (to make the underlying physical memory available to other processes).

Further options for large workloads and services:

  • MIMALLOC_USE_NUMA_NODES=N: pretend there are at most N NUMA nodes. If not set, the actual NUMA nodes are detected at runtime. Setting N to 1 may avoid problems in some virtual environments. Also, setting it to a lower number than the actual NUMA nodes is fine and will only cause threads to potentially allocate more memory across actual NUMA nodes (but this can happen in any case as NUMA local allocation is always a best effort but not guaranteed).
  • MIMALLOC_ALLOW_LARGE_OS_PAGES=1: use large OS pages (2 or 4MiB) when available; for some workloads this can significantly improve performance. When this option is disabled, it also disables transparent huge pages (THP) for the process (on Linux and Android). Use MIMALLOC_VERBOSE to check if the large OS pages are enabled -- usually one needs to explicitly give permissions for large OS pages (as on Windows and Linux). However, sometimes the OS is very slow to reserve contiguous physical memory for large OS pages so use with care on systems that can have fragmented memory (for that reason, we generally recommend to use MIMALLOC_RESERVE_HUGE_OS_PAGES instead whenever possible).
  • MIMALLOC_RESERVE_HUGE_OS_PAGES=N: where N is the number of 1GiB huge OS pages. This reserves the huge pages at startup and sometimes this can give a large (latency) performance improvement on big workloads. Usually it is better to not use MIMALLOC_ALLOW_LARGE_OS_PAGES=1 in combination with this setting. Just like large OS pages, use with care as reserving contiguous physical memory can take a long time when memory is fragmented (but reserving the huge pages is done at startup only once). Note that we usually need to explicitly give permission for huge OS pages (as on Windows and Linux)). With huge OS pages, it may be beneficial to set the setting MIMALLOC_EAGER_COMMIT_DELAY=N (N is 1 by default) to delay the initial N segments (of 4MiB) of a thread to not allocate in the huge OS pages; this prevents threads that are short lived and allocate just a little to take up space in the huge OS page area (which cannot be purged as huge OS pages are pinned to physical memory). The huge pages are usually allocated evenly among NUMA nodes. We can use MIMALLOC_RESERVE_HUGE_OS_PAGES_AT=N where N is the numa node (starting at 0) to allocate all the huge pages at a specific numa node instead.

Use caution when using fork in combination with either large or huge OS pages: on a fork, the OS uses copy-on-write for all pages in the original process including the huge OS pages. When any memory is now written in that area, the OS will copy the entire 1GiB huge page (or 2MiB large page) which can cause the memory usage to grow in large increments.

Secure Mode

mimalloc can be build in secure mode by using the -DMI_SECURE=ON flags in cmake. This build enables various mitigations to make mimalloc more robust against exploits. In particular:

  • All internal mimalloc pages are surrounded by guard pages and the heap metadata is behind a guard page as well (so a buffer overflow exploit cannot reach into the metadata).
  • All free list pointers are encoded with per-page keys which is used both to prevent overwrites with a known pointer, as well as to detect heap corruption.
  • Double free's are detected (and ignored).
  • The free lists are initialized in a random order and allocation randomly chooses between extension and reuse within a page to mitigate against attacks that rely on a predicable allocation order. Similarly, the larger heap blocks allocated by mimalloc from the OS are also address randomized.

As always, evaluate with care as part of an overall security strategy as all of the above are mitigations but not guarantees.

Debug Mode

When mimalloc is built using debug mode, various checks are done at runtime to catch development errors.

  • Statistics are maintained in detail for each object size. They can be shown using MIMALLOC_SHOW_STATS=1 at runtime.
  • All objects have padding at the end to detect (byte precise) heap block overflows.
  • Double free's, and freeing invalid heap pointers are detected.
  • Corrupted free-lists and some forms of use-after-free are detected.

Overriding Standard Malloc

Overriding the standard malloc (and new) can be done either dynamically or statically.

Dynamic override

This is the recommended way to override the standard malloc interface.

Dynamic Override on Linux, BSD

On these ELF-based systems we preload the mimalloc shared library so all calls to the standard malloc interface are resolved to the mimalloc library.

> env LD_PRELOAD=/usr/lib/libmimalloc.so myprogram

You can set extra environment variables to check that mimalloc is running, like:

> env MIMALLOC_VERBOSE=1 LD_PRELOAD=/usr/lib/libmimalloc.so myprogram

or run with the debug version to get detailed statistics:

> env MIMALLOC_SHOW_STATS=1 LD_PRELOAD=/usr/lib/libmimalloc-debug.so myprogram

Dynamic Override on MacOS

On macOS we can also preload the mimalloc shared library so all calls to the standard malloc interface are resolved to the mimalloc library.

> env DYLD_INSERT_LIBRARIES=/usr/lib/libmimalloc.dylib myprogram

Note that certain security restrictions may apply when doing this from the shell.

Dynamic Override on Windows

Dynamically overriding on mimalloc on Windows is robust and has the particular advantage to be able to redirect all malloc/free calls that go through the (dynamic) C runtime allocator, including those from other DLL's or libraries. As it intercepts all allocation calls on a low level, it can be used reliably on large programs that include other 3rd party components. There are four requirements to make the overriding work robustly:

  1. Use the C-runtime library as a DLL (using the /MD or /MDd switch).
  2. Link your program explicitly with mimalloc-override.dll library. To ensure the mimalloc-override.dll is loaded at run-time it is easiest to insert some call to the mimalloc API in the main function, like mi_version() (or use the /INCLUDE:mi_version switch on the linker). See the mimalloc-override-test project for an example on how to use this.
  3. The mimalloc-redirect.dll (or mimalloc-redirect32.dll) must be put in the same folder as the main mimalloc-override.dll at runtime (as it is a dependency of that DLL). The redirection DLL ensures that all calls to the C runtime malloc API get redirected to mimalloc functions (which reside in mimalloc-override.dll).
  4. Ensure the mimalloc-override.dll comes as early as possible in the import list of the final executable (so it can intercept all potential allocations).

For best performance on Windows with C++, it is also recommended to also override the new/delete operations (by including mimalloc-new-delete.h a single(!) source file in your project).

The environment variable MIMALLOC_DISABLE_REDIRECT=1 can be used to disable dynamic overriding at run-time. Use MIMALLOC_VERBOSE=1 to check if mimalloc was successfully redirected.

We cannot always re-link an executable with mimalloc-override.dll, and similarly, we cannot always ensure the the DLL comes first in the import table of the final executable. In many cases though we can patch existing executables without any recompilation if they are linked with the dynamic C runtime (ucrtbase.dll) -- just put the mimalloc-override.dll into the import table (and put mimalloc-redirect.dll in the same folder) Such patching can be done for example with CFF Explorer or the minject program.

Static override

On Unix-like systems, you can also statically link with mimalloc to override the standard malloc interface. The recommended way is to link the final program with the mimalloc single object file (mimalloc.o). We use an object file instead of a library file as linkers give preference to that over archives to resolve symbols. To ensure that the standard malloc interface resolves to the mimalloc library, link it as the first object file. For example:

> gcc -o myprogram mimalloc.o  myfile1.c ...

Another way to override statically that works on all platforms, is to link statically to mimalloc (as shown in the introduction) and include a header file in each source file that re-defines malloc etc. to mi_malloc. This is provided by mimalloc-override.h. This only works reliably though if all sources are under your control or otherwise mixing of pointers from different heaps may occur!

Tools

Generally, we recommend using the standard allocator with memory tracking tools, but mimalloc can also be build to support the address sanitizer or the excellent Valgrind tool. Moreover, it can be build to support Windows event tracing (ETW). This has a small performance overhead but does allow detecting memory leaks and byte-precise buffer overflows directly on final executables. See also the test/test-wrong.c file to test with various tools.

Valgrind

To build with valgrind support, use the MI_TRACK_VALGRIND=ON cmake option:

> cmake ../.. -DMI_TRACK_VALGRIND=ON

This can also be combined with secure mode or debug mode. You can then run your programs directly under valgrind:

> valgrind <myprogram>

If you rely on overriding malloc/free by mimalloc (instead of using the mi_malloc/mi_free API directly), you also need to tell valgrind to not intercept those calls itself, and use:

> MIMALLOC_SHOW_STATS=1 valgrind  --soname-synonyms=somalloc=*mimalloc* -- <myprogram>

By setting the MIMALLOC_SHOW_STATS environment variable you can check that mimalloc is indeed used and not the standard allocator. Even though the Valgrind option is called --soname-synonyms, this also works when overriding with a static library or object file. Unfortunately, it is not possible to dynamically override mimalloc using LD_PRELOAD together with valgrind. See also the test/test-wrong.c file to test with valgrind.

Valgrind support is in its initial development -- please report any issues.

ASAN

To build with the address sanitizer, use the -DMI_TRACK_ASAN=ON cmake option:

> cmake ../.. -DMI_TRACK_ASAN=ON

This can also be combined with secure mode or debug mode. You can then run your programs as:'

> ASAN_OPTIONS=verbosity=1 <myprogram>

When you link a program with an address sanitizer build of mimalloc, you should generally compile that program too with the address sanitizer enabled. For example, assuming you build mimalloc in out/debug:

clang -g -o test-wrong -Iinclude test/test-wrong.c out/debug/libmimalloc-asan-debug.a -lpthread -fsanitize=address -fsanitize-recover=address

Since the address sanitizer redirects the standard allocation functions, on some platforms (macOSX for example) it is required to compile mimalloc with -DMI_OVERRIDE=OFF. Adress sanitizer support is in its initial development -- please report any issues.

ETW

Event tracing for Windows (ETW) provides a high performance way to capture all allocations though mimalloc and analyze them later. To build with ETW support, use the -DMI_TRACK_ETW=ON cmake option.

You can then capture an allocation trace using the Windows performance recorder (WPR), using the src/prim/windows/etw-mimalloc.wprp profile. In an admin prompt, you can use:

> wpr -start src\prim\windows\etw-mimalloc.wprp -filemode
> <my_mimalloc_program>
> wpr -stop <my_mimalloc_program>.etl

and then open <my_mimalloc_program>.etl in the Windows Performance Analyzer (WPA), or use a tool like TraceControl that is specialized for analyzing mimalloc traces.

Performance

Last update: 2021-01-30

We tested mimalloc against many other top allocators over a wide range of benchmarks, ranging from various real world programs to synthetic benchmarks that see how the allocator behaves under more extreme circumstances. In our benchmark suite, mimalloc outperforms other leading allocators (jemalloc, tcmalloc, Hoard, etc), and has a similar memory footprint. A nice property is that it does consistently well over the wide range of benchmarks.

General memory allocators are interesting as there exists no algorithm that is optimal -- for a given allocator one can usually construct a workload where it does not do so well. The goal is thus to find an allocation strategy that performs well over a wide range of benchmarks without suffering from (too much) underperformance in less common situations.

As always, interpret these results with care since some benchmarks test synthetic or uncommon situations that may never apply to your workloads. For example, most allocators do not do well on xmalloc-testN but that includes even the best industrial allocators like jemalloc and tcmalloc that are used in some of the world's largest systems (like Chrome or FreeBSD).

Also, the benchmarks here do not measure the behaviour on very large and long-running server workloads, or worst-case latencies of allocation. Much work has gone into mimalloc to work well on such workloads (for example, to reduce virtual memory fragmentation on long-running services) but such optimizations are not always reflected in the current benchmark suite.

We show here only an overview -- for more specific details and further benchmarks we refer to the technical report. The benchmark suite is automated and available separately as mimalloc-bench.

Benchmark Results on a 16-core AMD 5950x (Zen3)

Testing on the 16-core AMD 5950x processor at 3.4Ghz (4.9Ghz boost), with with 32GiB memory at 3600Mhz, running Ubuntu 20.04 with glibc 2.31 and GCC 9.3.0.

We measure three versions of mimalloc: the main version mi (tag:v1.7.0), the new v2.0 beta version as xmi (tag:v2.0.0), and the main version in secure mode as smi (tag:v1.7.0).

The other allocators are Google's tcmalloc (tc, tag:gperftools-2.8.1) used in Chrome, Facebook's jemalloc (je, tag:5.2.1) by Jason Evans used in Firefox and FreeBSD, the Intel thread building blocks allocator (tbb, tag:v2020.3), rpmalloc (rp,tag:1.4.1) by Mattias Jansson, the original scalable Hoard (git:d880f72) allocator by Emery Berger [1], the memory compacting Mesh (git:67ff31a) allocator by Bobby Powers et al [8], and finally the default system allocator (glibc, 2.31) (based on PtMalloc2).

Any benchmarks ending in N run on all 32 logical cores in parallel. Results are averaged over 10 runs and reported relative to mimalloc (where 1.2 means it took 1.2ร— longer to run). The legend also contains the overall relative score between the allocators where 100 points is the maximum if an allocator is fastest on all benchmarks.

The single threaded cfrac benchmark by Dave Barrett is an implementation of continued fraction factorization which uses many small short-lived allocations. All allocators do well on such common usage, where mimalloc is just a tad faster than tcmalloc and jemalloc.

The leanN program is interesting as a large realistic and concurrent workload of the Lean theorem prover compiling its own standard library, and there is a 13% speedup over tcmalloc. This is quite significant: if Lean spends 20% of its time in the allocator that means that mimalloc is 1.6ร— faster than tcmalloc here. (This is surprising as that is not measured in a pure allocation benchmark like alloc-test. We conjecture that we see this outsized improvement here because mimalloc has better locality in the allocation which improves performance for the other computations in a program as well).

The single threaded redis benchmark again show that most allocators do well on such workloads.

The larsonN server benchmark by Larson and Krishnan [2] allocates and frees between threads. They observed this behavior (which they call bleeding) in actual server applications, and the benchmark simulates this. Here, mimalloc is quite a bit faster than tcmalloc and jemalloc probably due to the object migration between different threads.

The mstressN workload performs many allocations and re-allocations, and migrates objects between threads (as in larsonN). However, it also creates and destroys the N worker threads a few times keeping some objects alive beyond the life time of the allocating thread. We observed this behavior in many larger server applications.

The rptestN benchmark by Mattias Jansson is a allocator test originally designed for rpmalloc, and tries to simulate realistic allocation patterns over multiple threads. Here the differences between allocators become more apparent.

The second benchmark set tests specific aspects of the allocators and shows even more extreme differences between them.

The alloc-test, by OLogN Technologies AG, is a very allocation intensive benchmark doing millions of allocations in various size classes. The test is scaled such that when an allocator performs almost identically on alloc-test1 as alloc-testN it means that it scales linearly.

The sh6bench and sh8bench benchmarks are developed by MicroQuill as part of SmartHeap. In sh6bench mimalloc does much better than the others (more than 2.5ร— faster than jemalloc). We cannot explain this well but believe it is caused in part by the "reverse" free-ing pattern in sh6bench. The sh8bench is a variation with object migration between threads; whereas tcmalloc did well on sh6bench, the addition of object migration causes it to be 10ร— slower than before.

The xmalloc-testN benchmark by Lever and Boreham [5] and Christian Eder, simulates an asymmetric workload where some threads only allocate, and others only free -- they observed this pattern in larger server applications. Here we see that the mimalloc technique of having non-contended sharded thread free lists pays off as it outperforms others by a very large margin. Only rpmalloc, tbb, and glibc also scale well on this benchmark.

The cache-scratch benchmark by Emery Berger [1], and introduced with the Hoard allocator to test for passive-false sharing of cache lines. With a single thread they all perform the same, but when running with multiple threads the potential allocator induced false sharing of the cache lines can cause large run-time differences. Crundal [6] describes in detail why the false cache line sharing occurs in the tcmalloc design, and also discusses how this can be avoided with some small implementation changes. Only the tbb, rpmalloc and mesh allocators also avoid the cache line sharing completely, while Hoard and glibc seem to mitigate the effects. Kukanov and Voss [7] describe in detail how the design of tbb avoids the false cache line sharing.

On a 36-core Intel Xeon

For completeness, here are the results on a big Amazon c5.18xlarge instance consisting of a 2ร—18-core Intel Xeon (Cascade Lake) at 3.4GHz (boost 3.5GHz) with 144GiB ECC memory, running Ubuntu 20.04 with glibc 2.31, GCC 9.3.0, and Clang 10.0.0. This time, the mimalloc allocators (mi, xmi, and smi) were compiled with the Clang compiler instead of GCC. The results are similar to the AMD results but it is interesting to see the differences in the larsonN, mstressN, and xmalloc-testN benchmarks.

Peak Working Set

The following figure shows the peak working set (rss) of the allocators on the benchmarks (on the c5.18xlarge instance).

Note that the xmalloc-testN memory usage should be disregarded as it allocates more the faster the program runs. Similarly, memory usage of larsonN, mstressN, rptestN and sh8bench can vary depending on scheduling and speed. Nevertheless, we hope to improve the memory usage on mstressN and rptestN (just as cfrac, larsonN and sh8bench have a small working set which skews the results).

References

  • [1] Emery D. Berger, Kathryn S. McKinley, Robert D. Blumofe, and Paul R. Wilson. Hoard: A Scalable Memory Allocator for Multithreaded Applications the Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-IX). Cambridge, MA, November 2000. pdf

  • [2] P. Larson and M. Krishnan. Memory allocation for long-running server applications. In ISMM, Vancouver, B.C., Canada, 1998. pdf

  • [3] D. Grunwald, B. Zorn, and R. Henderson. Improving the cache locality of memory allocation. In R. Cartwright, editor, Proceedings of the Conference on Programming Language Design and Implementation, pages 177โ€“186, New York, NY, USA, June 1993. pdf

  • [4] J. Barnes and P. Hut. A hierarchical O(n*log(n)) force-calculation algorithm. Nature, 324:446-449, 1986.

  • [5] C. Lever, and D. Boreham. Malloc() Performance in a Multithreaded Linux Environment. In USENIX Annual Technical Conference, Freenix Session. San Diego, CA. Jun. 2000. Available at https://github.com/kuszmaul/SuperMalloc/tree/master/tests

  • [6] Timothy Crundal. Reducing Active-False Sharing in TCMalloc. 2016. CS16S1 project at the Australian National University. pdf

  • [7] Alexey Kukanov, and Michael J Voss. The Foundations for Scalable Multi-Core Software in Intel Threading Building Blocks. Intel Technology Journal 11 (4). 2007

  • [8] Bobby Powers, David Tench, Emery D. Berger, and Andrew McGregor. Mesh: Compacting Memory Management for C/C++ In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'19), June 2019, pages 333-โ€“346.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

Older Release Notes

  • 2021-11-14, v1.7.3, v2.0.3 (beta): improved WASM support, improved macOS support and performance (including M1), improved performance for v2 for large objects, Python integration improvements, more standard installation directories, various small fixes.

  • 2021-06-17, v1.7.2, v2.0.2 (beta): support M1, better installation layout on Linux, fix thread_id on Android, prefer 2-6TiB area for aligned allocation to work better on pre-windows 8, various small fixes.

  • 2021-04-06, v1.7.1, v2.0.1 (beta): fix bug in arena allocation for huge pages, improved aslr on large allocations, initial M1 support (still experimental).

  • 2021-01-31, v2.0.0: beta release 2.0: new slice algorithm for managing internal mimalloc pages.

  • 2021-01-31, v1.7.0: stable release 1.7: support explicit user provided memory regions, more precise statistics, improve macOS overriding, initial support for Apple M1, improved DragonFly support, faster memcpy on Windows, various small fixes.

  • 2020-09-24, v1.6.7: stable release 1.6: using standard C atomics, passing tsan testing, improved handling of failing to commit on Windows, add mi_process_info api call.

  • 2020-08-06, v1.6.4: stable release 1.6: improved error recovery in low-memory situations, support for IllumOS and Haiku, NUMA support for Vista/XP, improved NUMA detection for AMD Ryzen, ubsan support.

  • 2020-05-05, v1.6.3: stable release 1.6: improved behavior in out-of-memory situations, improved malloc zones on macOS, build PIC static libraries by default, add option to abort on out-of-memory, line buffered statistics.

  • 2020-04-20, v1.6.2: stable release 1.6: fix compilation on Android, MingW, Raspberry, and Conda, stability fix for Windows 7, fix multiple mimalloc instances in one executable, fix strnlen overload, fix aligned debug padding.

  • 2020-02-17, v1.6.1: stable release 1.6: minor updates (build with clang-cl, fix alignment issue for small objects).

  • 2020-02-09, v1.6.0: stable release 1.6: fixed potential memory leak, improved overriding and thread local support on FreeBSD, NetBSD, DragonFly, and macOSX. New byte-precise heap block overflow detection in debug mode (besides the double-free detection and free-list corruption detection). Add nodiscard attribute to most allocation functions. Enable MIMALLOC_PAGE_RESET by default. New reclamation strategy for abandoned heap pages for better memory footprint.

  • 2020-02-09, v1.5.0: stable release 1.5: improved free performance, small bug fixes.

  • 2020-01-22, v1.4.0: stable release 1.4: improved performance for delayed OS page reset, more eager concurrent free, addition of STL allocator, fixed potential memory leak.

  • 2020-01-15, v1.3.0: stable release 1.3: bug fixes, improved randomness and stronger free list encoding in secure mode.

  • 2019-12-22, v1.2.2: stable release 1.2: minor updates.

  • 2019-11-22, v1.2.0: stable release 1.2: bug fixes, improved secure mode (free list corruption checks, double free mitigation). Improved dynamic overriding on Windows.

  • 2019-10-07, v1.1.0: stable release 1.1.

  • 2019-09-01, v1.0.8: pre-release 8: more robust windows dynamic overriding, initial huge page support.

  • 2019-08-10, v1.0.6: pre-release 6: various performance improvements.

More Repositories

1

vscode

Visual Studio Code
TypeScript
163,565
star
2

PowerToys

Windows system utilities to maximize productivity
C#
110,602
star
3

TypeScript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TypeScript
100,730
star
4

terminal

The new Windows Terminal and the original Windows console host, all in the same place!
C++
94,835
star
5

Web-Dev-For-Beginners

24 Lessons, 12 Weeks, Get Started as a Web Developer
JavaScript
83,418
star
6

ML-For-Beginners

12 weeks, 26 lessons, 52 quizzes, classic Machine Learning for all
HTML
69,631
star
7

generative-ai-for-beginners

21 Lessons, Get Started Building with Generative AI ๐Ÿ”— https://microsoft.github.io/generative-ai-for-beginners/
Jupyter Notebook
64,519
star
8

playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript
64,013
star
9

monaco-editor

A browser based code editor
JavaScript
35,437
star
10

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Python
35,130
star
11

AI-For-Beginners

12 Weeks, 24 Lessons, AI for All!
Jupyter Notebook
34,704
star
12

autogen

A programming framework for agentic AI ๐Ÿค–
Jupyter Notebook
32,470
star
13

MS-DOS

The original sources of MS-DOS 1.25, 2.0, and 4.0 for reference purposes
Assembly
30,714
star
14

Data-Science-For-Beginners

10 Weeks, 20 Lessons, Data Science for All!
Jupyter Notebook
28,136
star
15

calculator

Windows Calculator: A simple yet powerful calculator that ships with Windows
C++
27,371
star
16

cascadia-code

This is a fun, new monospaced font that includes programming ligatures and is designed to enhance the modern look and feel of the Windows Terminal.
Python
25,726
star
17

JARVIS

JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf
Python
23,519
star
18

api-guidelines

Microsoft REST API Guidelines
22,661
star
19

winget-cli

WinGet is the Windows Package Manager. This project includes a CLI (Command Line Interface), PowerShell modules, and a COM (Component Object Model) API (Application Programming Interface).
C++
20,495
star
20

unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
Python
19,889
star
21

vcpkg

C++ Library Manager for Windows, Linux, and MacOS
CMake
19,600
star
22

fluentui

Fluent UI web represents a collection of utilities, React components, and web components for building web applications.
TypeScript
18,419
star
23

semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
C#
17,792
star
24

graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system
Python
17,750
star
25

CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
C++
17,412
star
26

WSL

Issues found on WSL
PowerShell
17,372
star
27

LightGBM

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
C++
16,470
star
28

AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
C++
16,327
star
29

react-native-windows

A framework for building native Windows apps with React.
C++
16,310
star
30

recommenders

Best Practices on Recommendation Systems
Python
16,075
star
31

IoT-For-Beginners

12 Weeks, 24 Lessons, IoT for All!
C++
15,360
star
32

qlib

Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.
Python
15,308
star
33

dotnet

This repo is the official home of .NET on GitHub. It's a great starting point to find many .NET OSS projects from Microsoft and the community, including many that are part of the .NET Foundation.
HTML
14,370
star
34

Bringing-Old-Photos-Back-to-Life

Bringing Old Photo Back to Life (CVPR 2020 oral)
Python
14,132
star
35

ai-edu

AI education materials for Chinese students, teachers and IT professionals.
HTML
13,485
star
36

pyright

Static Type Checker for Python
Python
13,195
star
37

nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
Python
13,084
star
38

guidance

A guidance language for controlling large language models.
Jupyter Notebook
11,777
star
39

TypeScript-Node-Starter

A reference example for TypeScript and Node with a detailed README describing how to use the two together.
SCSS
11,314
star
40

Swin-Transformer

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".
Python
11,187
star
41

TypeScript-React-Starter

A starter template for TypeScript and React with a detailed README describing how to use the two together.
TypeScript
11,081
star
42

frontend-bootcamp

Frontend Workshop from HTML/CSS/JS to TypeScript/React/Redux
TypeScript
10,807
star
43

windows-rs

Rust for Windows
Rust
10,411
star
44

wslg

Enabling the Windows Subsystem for Linux to include support for Wayland and X server related scenarios
C++
10,165
star
45

language-server-protocol

Defines a common protocol for language servers.
HTML
10,093
star
46

sql-server-samples

Azure Data SQL Samples - Official Microsoft GitHub Repository containing code samples for SQL Server, Azure SQL, Azure Synapse, and Azure SQL Edge
9,950
star
47

onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
C++
9,837
star
48

fast

The adaptive interface system for modern web experiences.
TypeScript
9,271
star
49

computervision-recipes

Best Practices, code samples, and documentation for Computer Vision.
Jupyter Notebook
9,264
star
50

napajs

Napa.js: a multi-threaded JavaScript runtime
C++
9,256
star
51

Windows-universal-samples

API samples for the Universal Windows Platform.
JavaScript
9,253
star
52

LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
Python
9,145
star
53

fluentui-emoji

A collection of familiar, friendly, and modern emoji from Microsoft
Python
9,068
star
54

vscode-tips-and-tricks

Collection of helpful tips and tricks for VS Code.
9,038
star
55

playwright-python

Python version of the Playwright testing and automation library.
Python
8,990
star
56

STL

MSVC's implementation of the C++ Standard Library.
C++
8,978
star
57

react-native-code-push

React Native module for CodePush
C
8,643
star
58

vscode-extension-samples

Sample code illustrating the VS Code extension API.
TypeScript
8,628
star
59

inshellisense

IDE style command line auto complete
TypeScript
8,402
star
60

reverse-proxy

A toolkit for developing high-performance HTTP reverse proxy applications.
C#
8,398
star
61

reactxp

Library for cross-platform app development.
TypeScript
8,289
star
62

WSL2-Linux-Kernel

The source for the Linux kernel used in Windows Subsystem for Linux 2 (WSL2)
C
8,037
star
63

ailab

Experience, Learn and Code the latest breakthrough innovations with Microsoft AI
C#
7,699
star
64

c9-python-getting-started

Sample code for Channel 9 Python for Beginners course
Jupyter Notebook
7,642
star
65

UFO

A UI-Focused Agent for Windows OS Interaction.
Python
7,633
star
66

cpprestsdk

The C++ REST SDK is a Microsoft project for cloud-based client-server communication in native code using a modern asynchronous C++ API design. This project aims to help C++ developers connect to and interact with services.
C++
7,573
star
67

botframework-sdk

Bot Framework provides the most comprehensive experience for building conversation applications.
JavaScript
7,484
star
68

azuredatastudio

Azure Data Studio is a data management and development tool with connectivity to popular cloud and on-premises databases. Azure Data Studio supports Windows, macOS, and Linux, with immediate capability to connect to Azure SQL and SQL Server. Browse the extension library for more database support options including MySQL, PostreSQL, and MongoDB.
TypeScript
7,182
star
69

winget-pkgs

The Microsoft community Windows Package Manager manifest repository
6,981
star
70

Windows-driver-samples

This repo contains driver samples prepared for use with Microsoft Visual Studio and the Windows Driver Kit (WDK). It contains both Universal Windows Driver and desktop-only driver samples.
C
6,924
star
71

winfile

Original Windows File Manager (winfile) with enhancements
C
6,437
star
72

nlp-recipes

Natural Language Processing Best Practices & Examples
Python
6,379
star
73

WinObjC

Objective-C for Windows
C
6,241
star
74

SandDance

Visually explore, understand, and present your data.
TypeScript
6,091
star
75

VFSForGit

Virtual File System for Git: Enable Git at Enterprise Scale
C#
5,979
star
76

GSL

Guidelines Support Library
C++
5,957
star
77

MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
C#
5,943
star
78

fluentui-system-icons

Fluent System Icons are a collection of familiar, friendly and modern icons from Microsoft.
HTML
5,934
star
79

vscode-go

An extension for VS Code which provides support for the Go language. We have moved to https://github.com/golang/vscode-go
TypeScript
5,932
star
80

microsoft-ui-xaml

Windows UI Library: the latest Windows 10 native controls and Fluent styles for your applications
5,861
star
81

vscode-recipes

JavaScript
5,859
star
82

rushstack

Monorepo for tools developed by the Rush Stack community
TypeScript
5,840
star
83

MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
Python
5,782
star
84

vscode-docs

Public documentation for Visual Studio Code
Markdown
5,650
star
85

ethr

Ethr is a Comprehensive Network Measurement Tool for TCP, UDP & ICMP.
Go
5,642
star
86

FASTER

Fast persistent recoverable log and key-value store + cache, in C# and C++.
C#
5,630
star
87

vscode-cpptools

Official repository for the Microsoft C/C++ extension for VS Code.
TypeScript
5,501
star
88

DirectX-Graphics-Samples

This repo contains the DirectX Graphics samples that demonstrate how to build graphics intensive applications on Windows.
C++
5,440
star
89

promptbase

All things prompt engineering
Python
5,367
star
90

BosqueLanguage

The Bosque programming language is an experiment in regularized design for a machine assisted rapid and reliable software development lifecycle.
TypeScript
5,282
star
91

TaskWeaver

A code-first agent framework for seamlessly planning and executing data analytics tasks.
Python
5,258
star
92

Detours

Detours is a software package for monitoring and instrumenting API calls on Windows. It is distributed in source code form.
C++
5,139
star
93

tsyringe

Lightweight dependency injection container for JavaScript/TypeScript
TypeScript
5,104
star
94

DeepSpeedExamples

Example models using DeepSpeed
Python
5,092
star
95

SynapseML

Simple and Distributed Machine Learning
Scala
5,041
star
96

Windows-classic-samples

This repo contains samples that demonstrate the API used in Windows classic desktop applications.
5,040
star
97

sudo

It's sudo, for Windows
Rust
4,998
star
98

TypeScript-Handbook

Deprecated, please use the TypeScript-Website repo instead
JavaScript
4,883
star
99

vscode-dev-containers

NOTE: Most of the contents of this repository have been migrated to the new devcontainers GitHub org (https://github.com/devcontainers). See https://github.com/devcontainers/template-starter and https://github.com/devcontainers/feature-starter for information on creating your own!
Shell
4,713
star
100

tsdoc

A doc comment standard for TypeScript
TypeScript
4,705
star