• Stars
    star
    473
  • Rank 92,209 (Top 2 %)
  • Language
    Go
  • License
    MIT License
  • Created about 4 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Zero-allocation reflection library for Go

go-reflect

Go GoDoc codecov Go Report Card

Zero-allocation reflection library for Go

Features

  • 100% Compatibility APIs with reflect library
  • No allocation occurs when using the reflect.Type features
  • You can choose to escape ( reflect.ValueOf ) or noescape ( reflect.ValueNoEscapeOf ) when creating reflect.Value

Status

All the tests in the reflect library have been passed except the tests that use some private functions.

Installation

go get github.com/goccy/go-reflect

How to use

Replace import statement from reflect to github.com/goccy/go-reflect

-import "reflect"
+import "github.com/goccy/go-reflect"

Benchmarks

Source https://github.com/goccy/go-reflect/blob/master/benchmark_test.go

Benchmark about reflect.Type

$ go test -bench TypeOf
goos: darwin
goarch: amd64
pkg: github.com/goccy/go-reflect
Benchmark_TypeOf_Reflect-12             100000000               13.8 ns/op             8 B/op          1 allocs/op
Benchmark_TypeOf_GoReflect-12           2000000000               1.70 ns/op            0 B/op          0 allocs/op
PASS
ok      github.com/goccy/go-reflect     5.369s

Benchmark about reflect.Value

$ go test -bench ValueOf
goos: darwin
goarch: amd64
pkg: github.com/goccy/go-reflect
Benchmark_ValueOf_Reflect-12            100000000               13.0 ns/op             8 B/op          1 allocs/op
Benchmark_ValueOf_GoReflect-12          300000000                4.64 ns/op            0 B/op          0 allocs/op
PASS
ok      github.com/goccy/go-reflect     3.578s

Real World Example

Implements Fast Marshaler

I would like to introduce the technique I use for github.com/goccy/go-json.
Using this technique, allocation can be suppressed to once for any marshaler.

Original Source is https://github.com/goccy/go-reflect/blob/master/benchmark_marshaler_test.go

package reflect_test

import (
    "errors"
    "strconv"
    "sync"
    "testing"
    "unsafe"

    "github.com/goccy/go-reflect"
)

var (
    typeToEncoderMap sync.Map
    bufpool          = sync.Pool{
        New: func() interface{} {
            return &buffer{
                b: make([]byte, 0, 1024),
            }
        },
    }
)

type buffer struct {
    b []byte
}

type encoder func(*buffer, unsafe.Pointer) error

func Marshal(v interface{}) ([]byte, error) {

    // Technique 1.
    // Get type information and pointer from interface{} value without allocation.
    typ, ptr := reflect.TypeAndPtrOf(v)
    typeID := reflect.TypeID(typ)

    // Technique 2.
    // Reuse the buffer once allocated using sync.Pool
    buf := bufpool.Get().(*buffer)
    buf.b = buf.b[:0]
    defer bufpool.Put(buf)

    // Technique 3.
    // builds a optimized path by typeID and caches it
    if enc, ok := typeToEncoderMap.Load(typeID); ok {
        if err := enc.(encoder)(buf, ptr); err != nil {
            return nil, err
        }

        // allocate a new buffer required length only
        b := make([]byte, len(buf.b))
        copy(b, buf.b)
        return b, nil
    }

    // First time,
    // builds a optimized path by type and caches it with typeID.
    enc, err := compile(typ)
    if err != nil {
        return nil, err
    }
    typeToEncoderMap.Store(typeID, enc)
    if err := enc(buf, ptr); err != nil {
        return nil, err
    }

    // allocate a new buffer required length only
    b := make([]byte, len(buf.b))
    copy(b, buf.b)
    return b, nil
}

func compile(typ reflect.Type) (encoder, error) {
    switch typ.Kind() {
    case reflect.Struct:
        return compileStruct(typ)
    case reflect.Int:
        return compileInt(typ)
    }
    return nil, errors.New("unsupported type")
}

func compileStruct(typ reflect.Type) (encoder, error) {

    encoders := []encoder{}

    for i := 0; i < typ.NumField(); i++ {
        field := typ.Field(i)
        enc, err := compile(field.Type)
        if err != nil {
            return nil, err
        }
        offset := field.Offset
        encoders = append(encoders, func(buf *buffer, p unsafe.Pointer) error {
            return enc(buf, unsafe.Pointer(uintptr(p)+offset))
        })
    }
    return func(buf *buffer, p unsafe.Pointer) error {
        buf.b = append(buf.b, '{')
        for _, enc := range encoders {
            if err := enc(buf, p); err != nil {
                return err
            }
        }
        buf.b = append(buf.b, '}')
        return nil
    }, nil
}

func compileInt(typ reflect.Type) (encoder, error) {
    return func(buf *buffer, p unsafe.Pointer) error {
        value := *(*int)(p)
        buf.b = strconv.AppendInt(buf.b, int64(value), 10)
        return nil
    }, nil
}

func Benchmark_Marshal(b *testing.B) {
    b.ReportAllocs()
    for n := 0; n < b.N; n++ {
        bytes, err := Marshal(struct{ I int }{10})
        if err != nil {
            b.Fatal(err)
        }
        if string(bytes) != "{10}" {
            b.Fatal("unexpected error")
        }
    }
}

The benchmark result is as follows.

$ go test -bench Benchmark_Marshal
goos: darwin
goarch: amd64
pkg: github.com/goccy/go-reflect
Benchmark_Marshal-16            16586372                71.0 ns/op             4 B/op          1 allocs/op
PASS

More Repositories

1

go-json

Fast JSON encoder/decoder compatible with encoding/json for Go
Go
2,348
star
2

go-yaml

YAML support for the Go language
Go
880
star
3

bigquery-emulator

BigQuery emulator server implemented in Go
Go
779
star
4

go-graphviz

Go bindings for Graphviz
Go
495
star
5

perl-motion

Perl for iOS and OS X
Objective-C
181
star
6

go-zetasql

Go bindings for ZetaSQL
Go
80
star
7

gperl

fastest perl like language
C++
72
star
8

go-jit

JIT compile library for Go
Go
71
star
9

rebirth

Supports live reloading for Go
Go
67
star
10

go-zetasqlite

A database driver library that interprets ZetaSQL queries and runs them using SQLite3
Go
52
star
11

p5-Compiler-Lexer

Lexical Analyzer for Perl5
Perl
46
star
12

p5-Compiler-CodeGenerator-LLVM

Create LLVM IR for Perl5
C++
40
star
13

kubejob

A library for managing Kubernetes Job in Go
Go
36
star
14

p5-Compiler-Parser

Create Abstract Syntax Tree for Perl5
Perl
33
star
15

p5-Compiler-Tools-CopyPasteDetector

detect Copy and Paste of Perl5 Codes
Perl
29
star
16

go-execbin

Analyze the binary outputted by `go build` to get type information etc.
Go
15
star
17

kubetest

A CLI for distributed execution of tasks on Kubernetes
Go
15
star
18

kpoward

kubernetes port forwarding utility library for Go
Go
10
star
19

p5-Test-AutoGenerator

automatically generate perl test code.
Perl
9
star
20

p5-App-Ikaros

distributed testing framework for jenkins
Perl
8
star
21

go-service-tracer

Visualize the dependencies between Microservices of gRPC methods implemented in Go
Go
7
star
22

iroonga

Groonga for iOS
C
6
star
23

treport

A fast scalable repository scanning tool
Go
5
star
24

go-json-fuzz

fuzzing test for goccy/go-json
Go
5
star
25

RecordKit

Record or stream video from the screen, and audio from the app and microphone
Objective-C
5
star
26

p5-Compiler-Tools-Transpiler

Transpile Perl5 code to JavaScript code
Perl
4
star
27

echo-tools

utility tools for labstack/echo
Go
4
star
28

glisp

lisp based very fast functional language
C
4
star
29

p5-App-Harmonia

generate model layer codes of your application for Parse.com
Perl
4
star
30

go-gcpurl

Parse the URL to get the GCP projectID in Go
Go
3
star
31

gmacs

emacs like editor
C++
3
star
32

cgo-math

Generate libm bridge for resolving undefined symbol in cgo
Go
2
star
33

cgo-multipkg-example

This contains of issues and solutions for binding multi-package libraries with cgo.
C
2
star
34

goccy

1
star
35

FilterGenerator

Automatically generate picture's filter code for iOS and Android.
Objective-C
1
star
36

p5-Compiler-Tools-UselessModuleDetector

detect useless modules
Perl
1
star
37

binarian

BinaryHack library for Gopher
Go
1
star
38

zetasql-proto

ZetaSQL Protocol Buffers
1
star
39

picoredis

header only redis client
C
1
star
40

earth-cupsule

Convert OpenStreetMap data around the world ( over 1TB ) to portable data
Go
1
star
41

PhotoFilterProcessor

generates photo filter data using CIFilter for iOS
Objective-C
1
star
42

nopbx

Provides method of removing project.pbxproj from your project. Also, release from conflict of project file.
Ruby
1
star
43

go-wasmbind-tools

A variety of tools for Go's wasm binding
Go
1
star