• Stars
    star
    560
  • Rank 78,979 (Top 2 %)
  • Language
    Go
  • License
    MIT License
  • Created over 9 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Better binary packing for Go

Build Status GoDoc

struc

Struc exists to pack and unpack C-style structures from bytes, which is useful for binary files and network protocols. It could be considered an alternative to encoding/binary, which requires massive boilerplate for some similar operations.

Take a look at an example comparing struc and encoding/binary

Struc considers usability first. That said, it does cache reflection data and aims to be competitive with encoding/binary struct packing in every way, including performance.

Example struct

type Example struct {
    Var   int `struc:"int32,sizeof=Str"`
    Str   string
    Weird []byte `struc:"[8]int64"`
    Var   []int `struc:"[]int32,little"`
}

Struct tag format

  • Var []int `struc:"[]int32,little,sizeof=StringField"` will pack Var as a slice of little-endian int32, and link it as the size of StringField.
  • sizeof=: Indicates this field is a number used to track the length of a another field. sizeof fields are automatically updated on Pack() based on the current length of the tracked field, and are used to size the target field during Unpack().
  • Bare values will be parsed as type and endianness.

Endian formats

  • big (default)
  • little

Recognized types

  • pad - this type ignores field contents and is backed by a [length]byte containing nulls
  • bool
  • byte
  • int8, uint8
  • int16, uint16
  • int32, uint32
  • int64, uint64
  • float32
  • float64

Types can be indicated as arrays/slices using [] syntax. Example: []int64, [8]int32.

Bare slice types (those with no [size]) must have a linked Sizeof field.

Private fields are ignored when packing and unpacking.

Example code

package main

import (
    "bytes"
    "github.com/lunixbochs/struc"
)

type Example struct {
    A int `struc:"big"`

    // B will be encoded/decoded as a 16-bit int (a "short")
    // but is stored as a native int in the struct
    B int `struc:"int16"`

    // the sizeof key links a buffer's size to any int field
    Size int `struc:"int8,little,sizeof=Str"`
    Str  string

    // you can get freaky if you want
    Str2 string `struc:"[5]int64"`
}

func main() {
    var buf bytes.Buffer
    t := &Example{1, 2, 0, "test", "test2"}
    err := struc.Pack(&buf, t)
    o := &Example{}
    err = struc.Unpack(&buf, o)
}

Benchmark

BenchmarkEncode uses struc. Stdlib benchmarks use equivalent encoding/binary code. Manual encodes without any reflection, and should be considered an upper bound on performance (which generated code based on struc definitions should be able to achieve).

BenchmarkEncode        1000000   1265 ns/op
BenchmarkStdlibEncode  1000000   1855 ns/op
BenchmarkManualEncode  5000000    284 ns/op
BenchmarkDecode        1000000   1259 ns/op
BenchmarkStdlibDecode  1000000   1656 ns/op
BenchmarkManualDecode  20000000  89.0 ns/op

More Repositories

1

ActualVim

Sublime Text 3 input mode using Neovim. Issues are closed, feel free to submit Pull Requests if you have bug fixes however.
Python
1,562
star
2

usercorn

dynamic binary analysis via platform emulation
Go
876
star
3

patchkit

binary patching from Python
C
626
star
4

SublimeXiki

Xiki in Sublime Text
Python
512
star
5

sublimelint

Error highlighting in Sublime Text.
Python
250
star
6

glshim

OpenGL 1.x driver shim for OpenGL ES devices.
C
153
star
7

vtclean

strips terminal escapes from text, can preserve color
Go
105
star
8

revsync

realtime cross-tool collaborative reverse engineering
Python
100
star
9

lib43

portable libc optimized for code size and readability
C
80
star
10

tinygles

Software-rendered OpenGL ES
C
76
star
11

reslate

A solid backbone for your `.slate.js.`
JavaScript
66
star
12

mpwn

single file ctf/exploit client library - python3, type annotated
Python
66
star
13

og

Language and tool enhancements for Go
Go
51
star
14

feeds

transcribe audio feeds into public web ui
Python
41
star
15

meta

code sometimes leaks into the space between projects
Python
41
star
16

pingbin

service to check internet accessibility
Go
41
star
17

pitybas

a faithful TI-BASIC implementation
Python
33
star
18

glues

fork of http://code.google.com/p/glues/
C++
26
star
19

go-keychain

Simple OS keychain bindings for password storage in Go (Golang)
Go
26
star
20

bnrepl

Run your Binary Ninja Python console in a separate Terminal window.
Python
18
star
21

sublimevim

a (deprecated) WIP vim input plugin for Sublime Text 2
Python
17
star
22

lorcon

Fork of https://code.google.com/p/lorcon/
C
14
star
23

community

Talon Community Repo (New API)
Python
14
star
24

ghostrace

Golang syscall firehose (programmatic strace/dtruss)
C
13
star
25

n64-saleae-logic

N64 controller protocol analyzer
C++
13
star
26

precorn

[WIP] pivot a running process into an emulator for instrumentation
C
13
star
27

capstr

(fast) Capstone Go bindings
Go
12
star
28

crossldso

link a linux ELF .so library into memory with python and call functions in it, even if you're not on linux
Python
10
star
29

microlathe

LockIT Pro JTAG proxy + GDB stub
Python
9
star
30

project-euler

Project Euler polyglot
Prolog
8
star
31

uberserver

matchmaking/chat lobby server for the spring rts project
Python
8
star
32

orca

Shell
7
star
33

inscount

stable instruction counter based on qemu-user (--target-list=i386-linux-user,x86_64-linux-user)
C
7
star
34

EnableWebGL

A tweak to enable WebGL on iOS
Logos
6
star
35

sublime-syntaxget

A faster way to change Syntax highlighting modes in Sublime Text 2
Python
6
star
36

localdns

serves DNS for observed DHCP leases
Go
6
star
37

voicecode-commando

voicecode command cheat sheet
HTML
5
star
38

subasm

subleq assembler / interpreter / rop chain
Python
5
star
39

fs-uae-gles

A GL ES port of FS-UAE
C++
4
star
40

linters

default linters for sublimelint
Python
4
star
41

smolcc

C
4
star
42

unicorn-tools

4
star
43

AppleScripting

Sublime Text 3 package for editing and running AppleScript
Python
4
star
44

pynamed

No-nonsense DNS server stub in Python using twisted.names
Python
4
star
45

argjoy

Golang method invocation with arg codecs and optional args
Go
4
star
46

talon_wm

Window management in Talon
Python
3
star
47

binutils-wasm

binutils compiled to wasm with every single target
Shell
3
star
48

pyadc

WIP: basic ADC client daemon in Python
Python
3
star
49

tftpd-cgi

A simple TFTP server capable of serving CGI scripts.
Python
3
star
50

go-clip

Clipboard bindings for Golang (without shelling out)
Go
3
star
51

preload-hooks

LD_PRELOAD framework
C
3
star
52

luaish

Go
2
star
53

libgpu

basic software rasterizer
C
2
star
54

vaporbat

Python Steam client
Python
2
star
55

maze

maze generator (eventually solver? was going to be an experiment in parallel maze solving)
Go
2
star
56

tactis

TIS-100 emulator / library
C
2
star
57

pypyninja

pypy interpreter shim for Binary Ninja
C
2
star
58

idascript

Python
2
star
59

uberspring

Python client/server for the Spring RTS network protocol
Python
2
star
60

pytiger

pytiger mirror, with various fixes for pyadc
C
2
star
61

webster

https://bochs.info/webster/ - interface to webster 1913 unabridged
HTML
2
star
62

qemu-2.5-unicorn

C
1
star
63

railsync

Go
1
star
64

location-sentry

Tweak to show GPS-using apps on iOS
Objective-C
1
star
65

minorgems-gles

minorGems plus OpenGL ES, OpenPandora compatibility
C++
1
star
66

ids

Python
1
star
67

bearfield

A small, efficient, easy to use MongoDB object layer.
Python
1
star
68

ti-omap5-sgx-ddk-linux

C
1
star
69

LockButton

menubar icon to lock your mac
Objective-C
1
star