• Stars
    star
    869
  • Rank 52,497 (Top 2 %)
  • Language
    C++
  • License
    Other
  • Created over 13 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

JSON NIFs for Erlang

Jiffy - JSON NIFs for Erlang

A JSON parser as a NIF. This is a complete rewrite of the work I did in EEP0018 that was based on Yajl. This new version is a hand crafted state machine that does its best to be as quick and efficient as possible while not placing any constraints on the parsed JSON.

Build Status

Usage

Jiffy is a simple API. The only thing that might catch you off guard is that the return type of jiffy:encode/1 is an iolist even though it returns a binary most of the time.

A quick note on unicode. Jiffy only understands UTF-8 in binaries. End of story.

Errors are raised as error exceptions.

Eshell V5.8.2  (abort with ^G)
1> jiffy:decode(<<"{\"foo\": \"bar\"}">>).
{[{<<"foo">>,<<"bar">>}]}
2> Doc = {[{foo, [<<"bing">>, 2.3, true]}]}.
{[{foo,[<<"bing">>,2.3,true]}]}
3> jiffy:encode(Doc).
<<"{\"foo\":[\"bing\",2.3,true]}">>

jiffy:decode/1,2

  • jiffy:decode(IoData)
  • jiffy:decode(IoData, Options)

The options for decode are:

  • return_maps - Tell Jiffy to return objects using the maps data type on VMs that support it. This raises an error on VMs that don't support maps.
  • {null_term, Term} - Returns the specified Term instead of null when decoding JSON. This is for people that wish to use undefined instead of null.
  • use_nil - Returns the atom nil instead of null when decoding JSON. This is a short hand for {null_term, nil}.
  • return_trailer - If any non-whitespace is found after the first JSON term is decoded the return value of decode/2 becomes {has_trailer, FirstTerm, RestData::iodata()}. This is useful to decode multiple terms in a single binary.
  • dedupe_keys - If a key is repeated in a JSON object this flag will ensure that the parsed object only contains a single entry containing the last value seen. This mirrors the parsing beahvior of virtually every other JSON parser.
  • copy_strings - Normally, when strings are decoded, they are created as sub-binaries of the input data. With some workloads, this leads to an undesirable bloating of memory: Strings in the decode result keep a reference to the full JSON document alive. Setting this option will instead allocate new binaries for each string, so the original JSON document can be garbage collected even though the decode result is still in use.
  • {bytes_per_red, N} where N >= 0 - This controls the number of bytes that Jiffy will process as an equivalent to a reduction. Each 20 reductions we consume 1% of our allocated time slice for the current process. When the Erlang VM indicates we need to return from the NIF.
  • {bytes_per_iter, N} where N >= 0 - Backwards compatible option that is converted into the bytes_per_red value.

jiffy:encode/1,2

  • jiffy:encode(EJSON)
  • jiffy:encode(EJSON, Options)

where EJSON is a valid representation of JSON in Erlang according to the table below.

The options for encode are:

  • uescape - Escapes UTF-8 sequences to produce a 7-bit clean output
  • pretty - Produce JSON using two-space indentation
  • force_utf8 - Force strings to encode as UTF-8 by fixing broken surrogate pairs and/or using the replacement character to remove broken UTF-8 sequences in data.
  • use_nil - Encodes the atom nil as null.
  • escape_forward_slashes - Escapes the / character which can be useful when encoding URLs in some cases.
  • {bytes_per_red, N} - Refer to the decode options
  • {bytes_per_iter, N} - Refer to the decode options

Data Format

Erlang                          JSON            Erlang
==========================================================================

null                       -> null           -> null
true                       -> true           -> true
false                      -> false          -> false
"hi"                       -> [104, 105]     -> [104, 105]
<<"hi">>                   -> "hi"           -> <<"hi">>
hi                         -> "hi"           -> <<"hi">>
1                          -> 1              -> 1
1.25                       -> 1.25           -> 1.25
[]                         -> []             -> []
[true, 1.0]                -> [true, 1.0]    -> [true, 1.0]
{[]}                       -> {}             -> {[]}
{[{foo, bar}]}             -> {"foo": "bar"} -> {[{<<"foo">>, <<"bar">>}]}
{[{<<"foo">>, <<"bar">>}]} -> {"foo": "bar"} -> {[{<<"foo">>, <<"bar">>}]}
#{<<"foo">> => <<"bar">>}  -> {"foo": "bar"} -> #{<<"foo">> => <<"bar">>}

N.B. The last entry in this table is only valid for VM's that support the maps data type (i.e., 17.0 and newer) and client code must pass the return_maps option to jiffy:decode/2.

Improvements over EEP0018

Jiffy should be in all ways an improvement over EEP0018. It no longer imposes limits on the nesting depth. It is capable of encoding and decoding large numbers and it does quite a bit more validation of UTF-8 in strings.

More Repositories

1

python-spidermonkey

Python/JavaScript bridge module, making use of Mozilla's spidermonkey JavaScript implementation.
C
307
star
2

nif-examples

Examples for Erlang NIFs
C
105
star
3

emonk

Emonk!
C
42
star
4

couchdb

Development branches
Erlang
37
star
5

erlang-native-compiler

A standalone executable for compiling native code for Erlang.
Erlang
20
star
6

knit

Another Erlang Release/Upgrade Tool
Erlang
16
star
7

wsgiref2

WSGI v2.0 reference implementation
Python
13
star
8

erljson_bench

Benchmarking for Erlang JSON libraries
C
11
star
9

XD

A Programming Language
10
star
10

namespace

Create composite Python namespace packages
Python
9
star
11

johnny

Various natively implemented data structures.
C
8
star
12

loadvenv

Load virtual environments as a subshell
Shell
5
star
13

sleepy

A misbehaving NIF
Erlang
5
star
14

zero-to-emonk

Talk preparation
C
5
star
15

futonproxy

Simple proxy for hacking Futon
Python
4
star
16

couch_events

Event distribution app
Erlang
4
star
17

dsprof

Erlang dirty scheduler profiling
Erlang
3
star
18

rust-nif-examples

Some example NIFs written in Rust using Rustler
Erlang
3
star
19

jsonical

Canonical JSON serialization in Python
Python
2
star
20

TileDB-Memory-Tracking

Experiments Tracking Memory in TileDB
R
1
star
21

ruzzle-solver

Solving Ruzzleβ„’ Because Reasons
Python
1
star
22

tdbtk

Playing with Rust
Rust
1
star
23

nutty

Python
1
star
24

vcpkg-freeze

Create a static copy of installed vcpkg ports
Python
1
star
25

sam

A Language Server Protocol implementation for Erlang
Erlang
1
star
26

DetectingHandPosesWIthVision

Copy of Apple's Example Code
Swift
1
star
27

nifspeed

Simple tests to try and gauge Erlang NIF performance.
C
1
star