• Stars
    star
    197
  • Rank 197,722 (Top 4 %)
  • Language
    Lua
  • License
    MIT License
  • Created over 9 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Customizable Lua Serializer

binser - Customizable Lua Serializer

Build Status

There already exists a number of serializers for Lua, each with their own uses, limitations, and quirks. binser is yet another robust, pure Lua serializer that specializes in serializing Lua data with lots of userdata and custom classes and types. binser is a binary serializer and does not serialize data into human readable representation or use the Lua parser to read expressions. This makes it safe and moderately fast, especially on LuaJIT. binser also handles cycles, self-references, and metatables.

How to Use

Example

local binser = require "binser"

local mydata = binser.serialize(45, {4, 8, 12, 16}, "Hello, World!")

print(binser.deserializeN(mydata, 3))
-- 45	table: 0x7fa60054bdb0	Hello, World!

Serializing and Deserializing

local str = binser.serialize(...)

Serialize (almost) any Lua data into a Lua string. Numbers, strings, tables, booleans, and nil are all fully supported by default. Custom userdata and custom types, both identified by metatables, can also be supported by specifying a custom serialization function. Unserializable data should throw an error. Aliased to binser.s.

local results, len = binser.deserialize(str[, index])

Deserialize any string previously serialized by binser. Can optionally start at an index in the string (to drop leading characters). Index is 1 by default. Unrecognized data should throw an error. Results is a list of length len. Aliased to binser.d.

local ... = binser.deserializeN(str, n[, index])

Deserializes at most n values from str. The default value for n is one, so binser.deserializeN(str) will deserialize exactly one value from string, and ignore the rest of the string. Can optionally start at a given index, which is 1 by default. Aliased to binser.dn.

Custom types

local metatable = binser.register(metatable, name, serialize, deserialize)

Registers a custom type, identified by its metatable, to be serialized. Registering types has two main purposes. First, it allows custom serialization and deserialization for userdata and tables that contain userdata, which can't otherwise be serialized in a uniform way. Second, it allows efficient serialization of small tables with large metatables, as registered metatables are not serialized.

The metatable parameter is the metatable the identifies the type. The name parameter is the type name used in serialization. The only requirement for names is that they are unique. The serialize and deserialize parameters are a pair of functions that construct and destruct and instance of the type. serialize can return any number of serializable Lua objects, and deserialize should accept the arguments returned by serialize. serialize and deserialize can also be specified in metatable._serialize and metatable._deserialize respectively.

If serialize and deserialize are omitted, then default table serializers are used, which work very well for most tables. If your type describes userdata, however, serialize and deserialize must be provided.

local class = binser.registerClass(class[, name])

Registers a class as a custom type. binser currently supports 30log and middleclass. name is an optional parameter that defaults to class.name.

local metatable = binser.unregister(name)

Users should seldom need this, but to explicitly unregister a type, call this.

Templates

If binser's already compact serialization isn't enough, and you don't want to write complex and error prone custom serializers, binser has a functionality called templating. Templates specify the layout of a custom type, so that table keys don't need to be serialized many times. To specify a template, add the _template key to the metatable of your type.

An example:

local template = {
	"name", "age", "salary", "email",
	nested = {"more", "nested", "keys"}
}

local Employee_MT = {
	name = "Employee",
}

local joe = setmetatable({
	name = "Joe",
	age = 11,
	salary = "$1,000,000",
	email = "[email protected]",
	nested = {
		more = "blah",
		nested = "FUBAR",
		keys = "lost"
	}
}, Employee_MT)

-- Print length of serialized employee without templating
-- 117
binser.registerClass(Employee_MT)
print(#binser.s(joe))
binser.unregister(Employee_MT)

-- Print length of serialized employee with templating
-- 72
Employee_MT._template = template
binser.registerClass(Employee_MT)
print(#binser.s(joe))

In the above example, the resulting serialized value with templating is nearly half of the size of the default table serialization.

Resources

If there are certain objects that don't need to be serialized at all, like images, audio, or any system resource, binser can mark them as such to only serialize a reference to them. Resources must be registered in a similar way to custom types and given a unique name.

local resource = binser.registerResource(resource, name)

Registers a resource.

local resource = binser.unregisterResource(name)

Resources can be unregistered in a similar manner as custom types.

File IO

Mostly for convenience, binser has functions for writing and reading to files. These work through Lua's built in IO.

binser.writeFile(filepath, ...)

Serializes Lua objects and writes them to a file. Overwrites the previous file.

binser.appendFile(filepath, ...)

Same as writing to a file, but doesn't overwrite the old file.

local results, len = binser.readFile(filepath)

Reads and deserializes a file.

The trio of file convenience function have shortened aliases as well.

Function Alias
binser.writeFile binser.w
binser.appendFile binser.a
binser.readFile binser.r

Why

Most Lua serializers serialize into valid Lua code, which while very useful, makes it impossible to do things like custom serialization and deserialization. binser was originally written as a way to save game levels with images and other native resources, but is extremely general.

LuaRocks

binser is available as a rock on LuaRocks. Install via:

luarocks install binser

Testing

binser uses busted for testing. Install and run busted from the command line to test.

Notes

  • Serialized strings can contain unprintable and null characters.
  • Serialized data can be appended to other serialized data. (Cool :))
  • The functions binser.serialize, binser.deserialize, and binser.deserializeN can be shortened to binser.s, binser.d, and binser.dn as handy shortcuts.

Bugs

Pull requests are welcome, please help me squash bugs!

More Repositories

1

Fennel

Lua Lisp Language
Fennel
2,443
star
2

tiny-ecs

ECS for Lua
Lua
661
star
3

moonmint

A Web Framework for Lua
Lua
75
star
4

Splash.lua

2D Spatial Hashing in Lua
Lua
71
star
5

mendoza

A Static Site Generator
Janet
54
star
6

fennel.vim

Syntax highlighting for Fennel
Vim Script
51
star
7

Editgrid

gamera and HUMP compatible scaling grid for LΓ–VE
Lua
44
star
8

CommandoKibbles

Cannon Cat Game for Ludum Dare 32
Lua
39
star
9

luamd

Markdown to HTML in pure Lua.
Lua
33
star
10

corope

Lua threading utility using coroutines.
Lua
29
star
11

littleserver

A simple test HTTP server for Janet, circlet, and jpm.
13
star
12

ats-lua

An ATS to Lua compiler
ATS
9
star
13

janet-miniz

Janet bindings to the miniz library.
C
7
star
14

ldmath.h

Home rolled header only c game math lib.
C
7
star
15

ats2.vim

A vim package for working with the ATS2 language.
Vim Script
5
star
16

x43bot

Just an IRC bot.
4
star
17

weregoat

Game for Ludum Dare 33
Lua
4
star
18

libatscc2lua

Runtime library for code compiled from ATS to Lua
Lua
4
star
19

mcu-serve

HTTP Server for NodeMCU + ESP32
HTML
4
star
20

WTBUiOS

Boston University WTBU app
Swift
3
star
21

bkdoc

Markup format and compiler that can output to different document formats.
C
3
star
22

dotfiles

My Dotfiles
Shell
2
star
23

EGF

A Java entity-component game framework that uses slick.
Java
2
star
24

go

Go in ClojureScript - Play at
JavaScript
1
star
25

ldoomc

C GL code
C
1
star
26

msp432-morse

Example project for the msp432p401r microcontroller
C
1
star
27

truss-analysis

MATLAB
1
star
28

ezglib

An Easy Game Library in ClojureScript
Clojure
1
star
29

TankLand

A tank simulation written in Clojure
Clojure
1
star
30

heroku-buildpack-moonmint

A Heroku buildpack for LuaJIT and the moonmint server
CMake
1
star