CPP serializers benchmark

tested libraries

• bitsery 5.2.1
• boost 1.77.0
• cereal 1.3.0
• flatbuffers 2.0.0
• yas 7.1.0
• protobuf 3.17.3
• zpp_bits 2.0
• msgpack 3.3.0

GCC 11 (Ubuntu 20.04 x64)

Clang 12.0.1 (Ubuntu 20.04 x64)

Additional tests information

1. deserialization using brief\_syntax, similar to cereal
2. forward/backward compatibility enabled for Monster
3. all components of Vec3 is compressed in [-1.0, 1.0] range with precision 0.01
4. use non-resizable buffer uint8_t[150000] for serialization
5. use stream input/output adapter, underlying type is std::stringstream
6. on deserialization do not check for errors
7. check buffer size on reading, but writing buffer is preallocated std::array<uint8_t, 1000000>
8. doesn't check for buffer size when reading, buffer: std::array<uint8_t, 1000000>
9. use std::stringstream's internal std::string
10. use arena allocator
11. use yas::mem_<io>stream as buffer
12. with yas::no_header and yas::compacted
13. using std::stringstream

NOTE: tests for protobuf and flatbuffers is not 100% fair, because huge amount of CPU cycles goes to converting from generated types, to our defined types.

Why another cpp serializers benchmark

I'm aware that cpp-serializers project already exists, but it's testing set is way too simple and you cannot compile each project to separate executable.

This project contains more realisting data that needs to be serialized.

    enum Color : uint8_t {
        Red,
        Green,
        Blue
    };

    struct Vec3 {
        float x;
        float y;
        float z;
    };

    struct Weapon {
        std::string name;
        int16_t damage;
    };

    struct Monster {
        Vec3 pos;
        int16_t mana;
        int16_t hp;
        std::string name;
        std::vector<uint8_t> inventory;
        Color color;
        std::vector<Weapon> weapons;
        Weapon equipped;
        std::vector<Vec3> path;
    };

All data is random generated, although seed is hard-coded to get predictable results when running same test multiple times.

All projects implement same interface for serialization and deserialization.

struct Buf {
    const uint8_t* ptr;
    size_t bytesCount;
};

class ISerializerTest {
public:
    virtual Buf serialize(const std::vector<MyTypes::Monster>& data) = 0;
    virtual void deserialize(Buf buf, std::vector<MyTypes::Monster>& res) = 0;
    virtual ~ISerializerTest() = default;
};

Testing routine consist of few steps:

• data generation step, in which monsters are generated (default 50 monsters)
• warmup step, in which serialization and deserialization is run 5 times, to warmup cpu cache and check if deserialized data equals to original data.
• measurement step, runs serialization and deserialization multiple times (default 300000 samples), deserialization happens on same object, to avoid costly allocate operations for new object construction each time.

Building & testing

1. Build project

   mkdir build && cd build
   cmake ..
   make

2. Run tests with ctest -VV OR
3. Generate testing results (requires nodejs)

   cd ../tools/
   npm install
   npm start