mmm

Manual memory management for golang.

Have a look at FreeTree for a real-world example of how to use mmm.

What you should definitely know..

..before using mmm

Go doesn't provide any manual memory management primitives. For very good reasons. This has been talked about numerous times on the go-nuts mailing list, have a look over there for detailed discussions.

To make it short: unless you are absolutely certain that you have no better alternative and that you understand all of the tradeoffs involved, please do not use this library.

mmm is no black magic: it simply allocates memory segments outside of the GC-managed heap and provides a simple API (Read(), Write(), Pointer()) that abstracts away all of the evil stuff actually going on behind the scenes.

The performances of Go's garbage collector depend heavily on the number of pointers your software is using. No matter how much performance you gain by using mmm, you could have had the same gains had you redesigned your software to avoid the use of pointers entirely.

This is the raison d'être of mmm: in some cases, purposefully (re)designing your software to avoid the use of pointers actually leads to code that is overly complex, harder to reason about, and thus, harder to maintain. In such cases, mmm might allow you to completely eliminate the GC overhead issues in your software, while keeping your original design (with minimal changes to your implementation, of course).

Note that mmm heavily relies on Go's implementation of interfaces.

Finally, for the adventurous, you'll find most of the ugly stuff here and there.

UPDATE: this was discussed at length in this HN thread.

..once you've decided to use mmm

• Never point to data on the GC-managed heap using a pointer stored on an unmanaged heap.

If the only references to your garbage-collectable data are stored in an unmanaged memory chunk, and thus non-existent to the eyes of the GC, your data will be automatically deallocated. As it should be.

• mmm provides support for the following types: interfaces, arrays, structs, numerics/boolean (bool/int/uint/float/complex and their variants), unsafe.Pointer, and any possible combination of the above.

Slices and string are thus not supported, use arrays and byte arrays instead.

• mmm doesn't provide synchronization of reads and writes on a MemChunk.

It's entirely up to you to decide how you want to manage thread-safety.

Install

go get -u github.com/teh-cmc/mmm

Example

Here's a simple example of usage (code here):

package main

/////
// Simple example of usage
//
//	 go run examples/simple.go
//
/////

import (
	"fmt"
	"log"
	"unsafe"

	"github.com/teh-cmc/mmm"
)

type Coordinate struct {
	x, y int
}

func main() {
	// create a new memory chunk that contains 3 Coordinate structures
	mc, err := mmm.NewMemChunk(Coordinate{}, 3)
	if err != nil {
		log.Fatal(err)
	}

	// print 3
	fmt.Println(mc.NbObjects())

	// write {3,9} at index 0, then print {3,9}
	fmt.Println(mc.Write(0, Coordinate{3, 9}))
	// write {17,2} at index 1, then print {17,2}
	fmt.Println(mc.Write(1, Coordinate{17, 2}))
	// write {42,42} at index 2, then print {42,42}
	fmt.Println(mc.Write(2, Coordinate{42, 42}))

	// print {17,2}
	fmt.Println(mc.Read(1))
	// print {42,42}
	fmt.Println(*((*Coordinate)(unsafe.Pointer(mc.Pointer(2)))))

	// free memory chunk
	if err := mc.Delete(); err != nil {
		log.Fatal(err)
	}
}

Demonstration

Complete code for the following demonstration is available here.

All of the results shown below were computed using a DELL XPS 15-9530 ([email protected]).

Case A: managed heap, 10 million pointers to int

Let's see what happens when we store 10 millions pointers to integer on the managed heap:

// build 10 million pointers to integer on the managed heap
ints := make([]*int, 10*1e6)
// init our pointers
for i := range ints {
	j := i
	ints[i] = &j
}

for i := 0; i < 5; i++ {
	// randomly print one of our integers to make sure it's all working
	// as expected, and to prevent them from being optimized away
	fmt.Printf("\tvalue @ index %d: %d\n", i*1e4, *(ints[i*1e4]))

	// run GC
	now := time.Now().UnixNano()
	runtime.GC()
	fmt.Printf("\tGC time (managed heap, 10 million pointers): %d us\n", (time.Now().UnixNano()-now)/1e3)
}

This prints:

value @ index 0: 0
GC time (managed heap, 10 million pointers): 329840 us
value @ index 10000: 10000
GC time (managed heap, 10 million pointers): 325375 us
value @ index 20000: 20000
GC time (managed heap, 10 million pointers): 323367 us
value @ index 30000: 30000
GC time (managed heap, 10 million pointers): 327905 us
value @ index 40000: 40000
GC time (managed heap, 10 million pointers): 326469 us

That's an average ~326ms per GC call. Let's move to case B where we will start using mmm's memory chunks.

Case B: unmanaged heap, pointers generated on-the-fly

mmm doesn't store any pointer, it doesn't need to.

Since the data is stored on an unmanaged heap, it cannot be collected even if there's no reference to it. This allows mmm to generate pointers only when something's actually reading or writing to the data.

In pratice, it looks like that:

// build 10 million integers on an unmanaged heap
intz, err := mmm.NewMemChunk(int(0), 10*1e6)
if err != nil {
	log.Fatal(err)
}
// init our integers
for i := 0; i < int(intz.NbObjects()); i++ {
	intz.Write(i, i)
}

for i := 0; i < 5; i++ {
	// randomly print one of our integers to make sure it's all working
	// as expected (pointer to data is generated on-the-fly)
	fmt.Printf("\tvalue @ index %d: %d\n", i*1e4, intz.Read(i*1e4))

	// run GC
	now := time.Now().UnixNano()
	runtime.GC()
	fmt.Printf("\tGC time (unmanaged heap, pointers generated on-the-fly): %d us\n", (time.Now().UnixNano()-now)/1e3)

This prints:

value @ index 0: 0
GC time (unmanaged heap, pointers generated on-the-fly): 999 us
value @ index 10000: 10000
GC time (unmanaged heap, pointers generated on-the-fly): 665 us
value @ index 20000: 20000
GC time (unmanaged heap, pointers generated on-the-fly): 827 us
value @ index 30000: 30000
GC time (unmanaged heap, pointers generated on-the-fly): 882 us
value @ index 40000: 40000
GC time (unmanaged heap, pointers generated on-the-fly): 1016 us

That's an average ~0.9ms per GC call.

We went from a ~326ms average to a ~0.9ms average; but the comparison isn't really fair now, is it? In case A we were storing every pointer, here we're simply not storing any.

That leads us to case C, in which we build pointers to each and every integer that's in our unmanaged heap.

Case C: unmanaged heap, storing all generated pointers

What happens when we build and store 10 million pointers: one for each and every integer that's in our unmanaged memory chunk?

// build 10 million unsafe pointers on the managed heap
ptrs := make([]unsafe.Pointer, 10*1e6)
// init those pointers so that they point to the unmanaged heap
for i := range ptrs {
	ptrs[i] = unsafe.Pointer(intz.Pointer(i))
}

for i := 0; i < 5; i++ {
	// randomly print one of our integers to make sure it's all working
	// as expected
	fmt.Printf("\tvalue @ index %d: %d\n", i*1e4, *(*int)(ptrs[i*1e4]))

	// run GC
	now := time.Now().UnixNano()
	runtime.GC()
	fmt.Printf("\tGC time (unmanaged heap, all generated pointers stored): %d us\n", (time.Now().UnixNano()-now)/1e3)
}

This prints:

value @ index 0: 0
GC time (unmanaged heap, all generated pointers stored): 47196 us
value @ index 10000: 10000
GC time (unmanaged heap, all generated pointers stored): 47307 us
value @ index 20000: 20000
GC time (unmanaged heap, all generated pointers stored): 47485 us
value @ index 30000: 30000
GC time (unmanaged heap, all generated pointers stored): 47145 us
value @ index 40000: 40000
GC time (unmanaged heap, all generated pointers stored): 47221 us

The results here are pretty interesting: on the one hand this is ~47 times slower than case B (in which we used mmm's memory chunks but didn't actually store any pointer), but on the other hand this is still 6 times faster than case A (in which we used native pointers) because unsafe pointers require far less work from the GC.

Six times faster is already quite the good deal, but why stop there? As we've already pointed out, mmm doesn't need to store references to its data... so.. don't.

This is what case D is all about, in which we will convert those pointers into simple numeric references and store them as such.

Case D: unmanaged heap, storing numeric references

Instead of storing (unsafe) pointers, let's treat these pointers as what they really are: simple numeric references.

// build 10 million numeric references on the managed heap
refs := make([]uintptr, 10*1e6)
// init those references so that they each contain one of the addresses in
// our unmanaged heap
for i := range refs {
	refs[i] = uintptr(ptrs[i])
}

// get rid of those unsafe pointers we stored
ptrs = nil

for i := 0; i < 5; i++ {
	// randomly print one of our integers to make sure it's all working
	// as expected
	fmt.Printf("\tvalue @ index %d: %d\n", i*1e4, *(*int)(unsafe.Pointer(refs[i*1e4])))

	// run GC
	now := time.Now().UnixNano()
	runtime.GC()
	fmt.Printf("\tGC time (unmanaged heap, all numeric references stored): %d us\n", (time.Now().UnixNano()-now)/1e3)
}

This prints:

value @ index 0: 0
GC time (unmanaged heap, all numeric references stored): 715 us
value @ index 10000: 10000
GC time (unmanaged heap, all numeric references stored): 783 us
value @ index 20000: 20000
GC time (unmanaged heap, all numeric references stored): 882 us
value @ index 30000: 30000
GC time (unmanaged heap, all numeric references stored): 711 us
value @ index 40000: 40000
GC time (unmanaged heap, all numeric references stored): 723 us

We're basically back to the results of case B. As far as the GC is concerned, those pointers don't exist, which translates into sub-millisecond GC calls.

Still, the memory they point to does exist, and is just one cast away from being read from and written to.

We now have everything we need to build pointer-based software without any GC overhead, and without any design modification: this is basically how FreeTree is implemented.

License

The MIT License (MIT) - see LICENSE for more details

Copyright (c) 2015 Clement 'cmc' Rey [email protected]