• Stars
    star
    48
  • Rank 575,491 (Top 12 %)
  • Language
    Crystal
  • License
    MIT License
  • Created about 5 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

BinData - Parsing Binary Data in Crystal Lang

BinData - Parsing Binary Data in Crystal Lang

BinData provides a declarative way to read and write structured binary data.

This means the programmer specifies what the format of the binary data is, and BinData works out how to read and write data in this format. It is an easier (and more readable) alternative.

Build Status

Usage

Firstly, it's recommended that you specify the datas endian.

class Header < BinData
  endian big
end

Then you can specify the structures fields. There are a few different field types:

  1. Core types
    • UInt8 to UInt128 values respectively
    • You can use endian-aware types to mix endianess: , endian: IO::ByteFormat::LittleEndian
  2. Custom types
  3. Bit Fields
    • These are a group of fields who values are defined by the number of bits used to represent their value
    • The total number of bits in a bit field must be divisible by 8
  4. Groups
    • These are embedded BinData class with access to the parent fields
    • Useful when a group of fields are related or optional
  5. Enums
  6. Bools
  7. Arrays and Sets (fixed size and dynamic)

Examples

see the spec helper for all possible manipulations

  enum Inputs
    VGA
    HDMI
    HDMI2
  end

  class Packet < BinData
    endian big

    # Default sets the value at initialisation.
    field start : UInt8 = 0xFF_u8

    # Value procs assign these values before writing to an IO, overwriting any
    # existing value
    field size : UInt16, value: ->{ text.bytesize + 1 }

    # String fields without a length use `\0` null byte termination
    # Length is being calculated by the size field above
    field text : String, length: ->{ size - 1 }

    # Bit fields should only be used when one or more fields are not byte aligned
    # The sum of the bits in a bit field must be divisible by 8
    bit_field do
      # a bits value can be between 1 and 128 bits long
      bits 5, reserved

      # Bool values are a single bit
      bool set_input = false

      # This enum is represented by 2 bits
      bits 2, input : Inputs = Inputs::HDMI2
    end

    # isolated namespace
    group :extended, onlyif: ->{ start == 0xFF } do
      field start : UInt8 = 0xFF_u8

      # Supports custom objects as long as they implement `from_io`
      field header : ExtHeader = ExtHeader.new
    end

    # optionally read the remaining bytes out of io
    remaining_bytes :rest
  end

The object above can then be accessed like any other object

  pack = io.read_bytes(Packet)
  pack.size # => 12
  pack.text # => "hello world"
  pack.input # => Inputs::HDMI
  pack.set_input # => true
  pack.extended.start # => 255

Additionally, BinData fields support a verify proc, which allows data to be verified while reading and writing io.

class VerifyData < BinData
  endian big

  field size : UInt8
  field bytes : Bytes, length: ->{ size }
  field checksum : UInt8, verify: ->{ checksum == bytes.reduce(0) { |acc, i| acc + i } }
end

If the verify proc returns false, a BinData::VerificationException is raised with a message matching the following format.

Failed to verify reading basic at VerifyData.checksum

Inheritance is also supported

ASN.1 Helpers

Included in this library are helpers for decoding and writing ASN.1 data, such as those used in SNMP and LDAP

require "bindata/asn1"

# Build an object
ber = ASN1::BER.new
ber.tag_number = ASN1::BER::UniversalTags::Integer
ber.payload = Bytes[1]

# Write it to an IO:
io.write_bytes(ber)

# Read data out of an IO:
ber = io.read_bytes(ASN1::BER)
ber.tag_class # => ASN1::BER::TagClass::Universal

Real World Examples

More Repositories

1

spider-gazelle

A Rails esque web framework with a focus on speed and extensibility for crystal lang
Crystal
175
star
2

tasker

Scheduled tasks for crystal lang
Crystal
54
star
3

ssh2.cr

libssh2 binding for Crystal language
Crystal
42
star
4

promise

Type aware promises for crystal lang
Crystal
40
star
5

action-controller

A rails-esque controller framework for crystal lang
Crystal
39
star
6

active-model

A rails-esque model framework for crystal lang
Crystal
28
star
7

rethinkdb-orm

RethinkDB ORM for Crystal lang
Crystal
24
star
8

crystal-mqtt

Crystal lang implementation of the MQTT protocol, a lightweight protocol for publish/subscribe messaging
Crystal
19
star
9

ffmpeg

ffmpeg crystal bindings
Crystal
18
star
10

qr-code

a QR Code implementation written in crystal lang
Crystal
17
star
11

crystal-ldap

a Crystal lang LDAP client
Crystal
16
star
12

crystal-snmp

SNMP implementation for crystal lang
Crystal
16
star
13

priority-queue

Priority Queue and Heap implementation for Crystal Lang
Crystal
13
star
14

bisect

Library for maintaining sorted Arrays
Crystal
12
star
15

json-schema

Describe crystal-lang JSON serializable types with JSON Schema
Crystal
12
star
16

pinger

Microlibrary to perform ping requests with Crystal Lang
Crystal
11
star
17

telnet.cr

Telnet protocol helper for crystal lang
Crystal
11
star
18

pars

Parser combinator library for crystal-lang
Crystal
11
star
19

inactive-support

Utilities for crystal-lang
Crystal
10
star
20

pg-orm

Postgres ORM for Crystal Lang
Crystal
9
star
21

mdns

Crystal Lang mDNS and DNS-SD Support
Crystal
8
star
22

tensorflow_lite

tensorflow lite bindings for crystal lang
Crystal
8
star
23

ed25519

Ed25519 high-performance public-key signature system for crystal lang
Crystal
7
star
24

crystal-gpt

ChatGPT plugin template that allows you to focus on writing actions, automatically generating the required metadata
Crystal
7
star
25

secure-remote-password

Crystal implementation of the Secure Remote Password protocol (SRP-6a)
Crystal
6
star
26

simple_retry

a tool for retrying code blocks
Crystal
6
star
27

connect-proxy

crystal lang connect / HTTP proxy implementation
Crystal
6
star
28

crystal-openai

OpenAI ChatGPT, GPT-3, GPT-4, DALLΒ·E, Whisper API Client for Crystal
Crystal
6
star
29

crunits

Physical quantity and units of measure conversion and math for crystal lang
Crystal
6
star
30

secrets-env

Extension to the crystal lang ENV module to support reading secrets
Crystal
6
star
31

v4l2.cr

crystal lang video for linux device helpers / bindings
Crystal
5
star
32

log_helper

Extension for Crystal Log to aid logging key-value data
Crystal
5
star
33

readers-writer

A simple readers writer lock for crystal lang
Crystal
5
star
34

cmac

Crystal implementation of the Cipher-based Message Authentication Code (CMAC)
Crystal
5
star
35

guide

Spider Gazelle Documentation
Python
4
star
36

ntlm

NTLM authentication for crystal lang
Crystal
4
star
37

worker_pool

a basic fiber pool implementation for crystal lang
Crystal
4
star
38

digest-auth

HTTP digest auth for crystal lang
Crystal
4
star
39

matter

A complete Crystal implementation of the Matter protocol specification (https://buildwithmatter.com). Includes full support for controller, device, commissioning, secure communications, device types, and cluster definitions.
Crystal
4
star
40

eventbus

Listen for Postgres database change events and publish them to event listeners
Crystal
3
star
41

knx

KNX protocol support for crystal lang
Crystal
3
star
42

tokenizer

Simplified binary stream tokenization for crystal lang
Crystal
2
star
43

crystal-dtls

DTLS support for crystal lang
Crystal
2
star
44

upload-signer

Provide API for generating pre-signed URLs for file uploads to cloud storage
Crystal
2
star
45

stomp

crystal lang implementation of the STOMP protocol
Crystal
1
star
46

tlv

Matter TLV encoder/decoder
Crystal
1
star
47

gpio.cr

crystal lang bindings for linux gpiod
Crystal
1
star
48

stumpy_resize

resizes stumpy canvas images in pure crystal
Crystal
1
star
49

HKDF

HMAC-based Extract-and-Expand Key Derivation Function (HKDF) for crystal lang
Crystal
1
star
50

SPAKE2_plus

a crystal lang implementation of SPAKE2+, a Password Authenticated Key Exchange (PAKE) protocol
Crystal
1
star
51

panopticon

Distributed tracing for services built in crystal-lang
Crystal
1
star
52

tflite_image

image classification and feature detection with tflite and crystal lang
Crystal
1
star
53

tflite_pipeline

video processing AI pipeline leveraging tflite_image
Crystal
1
star
54

link-header

Crystal Lang HTTP Link Header Parser
Crystal
1
star