• Stars
    star
    601
  • Rank 74,537 (Top 2 %)
  • Language
    Go
  • License
    MIT License
  • Created about 8 years ago
  • Updated over 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fast Go decoder for dynamic JSON

DJSON

GoDoc Build Status LICENSE

DJSON is a JSON decoder for Go that is 2~ to 3~ times faster than the standard encoding/json and the existing solutions, when dealing with arbitrary JSON payload. See benchmarks below.
It is a good approach for people who are using json.Unmarshal together with interface{}, don't know what the schema is, and still want good performance with minimal changes.

Motivation

While searching for a JSON parser solution for my projects, that is faster than the standard library, with zero reflection tests, allocates less memory and is still safe(I didn't want the "unsafe" package in my production code, in order to reduce memory consumption).
I found that almost all implemtations are just wrappers around the standard library and aren't fast enough for my needs.
I encountered two projects: ujson that is the UltraJSON implementation and jsonparser, that is a pretty awesome project.
ujson seems to be faster than encoding/json but still doesn't meet my requirements.
jsonparser seems to be really fast, and I even use it for some of my new projects.
However, its API is different, and I would need to change too much of my code in order to work with it.
Also, for my processing work that involves ETL, changing and setting new fields on the JSON object, I need to transform the jsonparser result to map[string]interface{} and it seems that it loses its power.

Advantages and Stability

As you can see in the benchmark below, DJSON is faster and allocates less memory than the other alternatives.
The current version is 1.0.0-alpha.1, and I'm waiting to hear from you if there are any issues or bug reports, to make it stable.
(comment: there is a test file named decode_test that contains a test case that compares the results to encoding/json - feel free to add more values if you find they are important)
I'm also plaining to add the DecodeStream(io.ReaderCloser) method(or NewDecoder(io.ReaderCloser)), to support stream decoding without breaking performance.

Benchmark

There are 3 benchmark types: small, medium and large payloads.
All the 3 are taken from the jsonparser project, and they try to simulate a real-life usage. Each result from the different benchmark types is shown in a metric table below. The lower the metrics are, the better the result is. Time/op is in nanoseconds, B/op is how many bytes were allocated per op and allocs/op is the total number of memory allocations.
Benchmark results that are better than encoding/json are marked in bold text.
The Benchmark tests run on AWS EC2 instance(c4.xlarge). see: screenshots

Compared libraries:

Small payload

Each library in the test gets a small payload to process that weighs 134 bytes.
You can see the payload here, and the test screenshot here.

Library Time/op B/op allocs/op
encoding/json 8646 1993 60
ugorji/go/codec 9272 4513 41
antonholmquist/jason 7336 3201 49
bitly/go-simplejson 5253 2241 36
Jeffail/gabs 4788 1409 33
mreiferson/go-ujson 3897 1393 35
a8m/djson 2534 1137 25
a8m/djson.AllocString 2195 1169 13

Medium payload

Each library in the test gets a medium payload to process that weighs 1.7KB.
You can see the payload here, and the test screenshot here.

Library Time/op B/op allocs/op
encoding/json 42029 10652 218
ugorji/go/codec 65007 15267 313
antonholmquist/jason 45676 17476 224
bitly/go-simplejson 45164 17156 219
Jeffail/gabs 41045 10515 211
mreiferson/go-ujson 33213 11506 267
a8m/djson 22871 10100 195
a8m/djson.AllocString 19296 10619 87

Large payload

Each library in the test gets a large payload to process that weighs 28KB.
You can see the payload here, and the test screenshot here.

Library Time/op B/op allocs/op
encoding/json 717882 212827 3247
ugorji/go/codec 1052347 239130 4426
antonholmquist/jason 751910 277931 3257
bitly/go-simplejson 753663 277628 3252
Jeffail/gabs 714304 212740 3241
mreiferson/go-ujson 599868 235789 4057
a8m/djson 437031 210997 2932
a8m/djson.AllocString 372382 214053 1413

LICENSE

MIT

More Repositories

1

golang-cheat-sheet

An overview of Go syntax and features.
8,366
star
2

angular-filter

Bunch of useful filters for AngularJS (with no external dependencies!)
JavaScript
2,927
star
3

envsubst

Environment variables substitution for Go
Go
732
star
4

pb

Console progress bar for Rust
Rust
580
star
5

reflect-examples

Bunch of examples for dealing with the reflect package
555
star
6

rql

Resource Query Language for REST
Go
337
star
7

syncmap

A typed implementation of the Go sync.Map using code generation
Go
256
star
8

mark

A markdown processor written in Go. built for fun.
Go
203
star
9

play

Play something while waiting for your command to finish
Go
185
star
10

kinesis-producer

An aggregated records producer for Amazon Kinesis
Go
147
star
11

enter

A CLI for generating ER diagrams for Ent schema
Go
131
star
12

ng-pipes

Bunch of useful pipes for Angular2 (with no external dependencies!)
TypeScript
118
star
13

tree

An implementation of the Unix tree command written in Go, that can be used programmatically
Go
92
star
14

agile

Like Underscore, but with zero callbacks and really more fun, v0.0.2
JavaScript
69
star
15

ent-graphql-example

The code for https://entgo.io/docs/tutorial-setup
Go
60
star
16

doqmentdb

A Promise-Based DocumentDB ODM Client for NodeJS
JavaScript
52
star
17

ng-translation

Fast, Easy and Dynamic translation for AngularJS
JavaScript
43
star
18

pb-scala

Console progress bar for Scala
Scala
37
star
19

documentdb

Go driver for Microsoft Azure DocumentDB
Go
33
star
20

expect

Minimalistic BDD-style assertions for Go (inspired by expect.js)
Go
32
star
21

deep-keys

Create an array composed of the own enumerable property names (including nested) of an object.
JavaScript
23
star
22

lease

Generic lease implementation using DynamoDB
Go
20
star
23

angular-code-mirror

2 way binding codemirror for AngularJS based on google-prettify
JavaScript
13
star
24

entclean

Clean ent/schemas
Go
11
star
25

errors

An experimental error handling package for Go
Go
10
star
26

clog

Pretty colorful cli logger for NodeJS(with table, success and more...)
JavaScript
9
star
27

s3tree

s3tree is a tree command for Amazon S3
Go
7
star
28

entspatial

An example repository for working with MySQL spatial data types in ent
Go
7
star
29

gotips-talk-2018

"Did you know that..." talk. Go-Israel meetup, Jan 2018
Go
6
star
30

maman14

maman14 - assembler
C
5
star
31

go-documentdb-example

A users CRUD app using Martini and DocumentDB
Go
4
star
32

obj-parse

Get and Set object properties in a Fast and Elegant way. (with caching and no dependencies!)
JavaScript
4
star
33

obj-del

Remove multiple keys by path - safety.
JavaScript
3
star
34

entsize

Print ent/schema size
Go
3
star
35

flag.js

cli flag parsing
JavaScript
2
star
36

dynamose

A Promise-Based DynamoDB Client
JavaScript
2
star
37

ent-sync-example

The code for https://entgo.io/blog/2021/11/1/sync-objects-in-external-database
Go
2
star
38

stringify.js

like JSON.stringify, but more sense
JavaScript
1
star
39

mark-cli

Mark command-line tool
Go
1
star
40

koa-documentdb-example

A users CRUD app using Koa and DoQmentDB(DocumentDB wrapper)
JavaScript
1
star
41

obj-is

is-function's creator
JavaScript
1
star
42

entraffle

A raffle for Ent Discord members
Go
1
star