• Stars
    star
    121
  • Rank 293,924 (Top 6 %)
  • Language
    Ruby
  • License
    Other
  • Created over 12 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Library for parsing MySQL binary logs in Ruby

Library for parsing MySQL binary logs in Ruby

This library parses a MySQL binary log in pure Ruby and produces hashes as output, much like the following Query event:

{:type=>:query_event,
 :position=>107,
 :filename=>"mysql-bin.000001",
 :header=>
  {:event_type=>2,
   :server_id=>1,
   :flags=>[],
   :event_length=>117,
   :timestamp=>1340414127,
   :next_position=>224},
 :event=>
  {:thread_id=>1,
   :query=>"create table a (id int, a char(100), primary key (id))",
   :status=>
    {:sql_mode=>0,
     :charset=>
      {:character_set_client=>
        {:character_set=>:utf8, :collation=>:utf8_general_ci},
       :collation_connection=>
        {:character_set=>:utf8, :collation=>:utf8_general_ci},
       :collation_server=>
        {:character_set=>:latin1, :collation=>:latin1_swedish_ci}},
     :flags2=>[],
     :catalog=>"std"},
   :elapsed_time=>0,
   :error_code=>0,
   :db=>"test"}}

Status

All event types can be read, but may not be parsed, as not all event types are currently fully supported. Over time this will improve. The current status of event support is documented below.

Event Types

ID Event Type Status
1 start_event_v3 Unsupported (deprecated).
2 query_event Fully supported with all fields parsed.
3 stop_event Fully supported with all fields parsed.
4 rotate_event Fully supported with all fields parsed.
5 intvar_event Fully supported with all fields parsed.
6 load_event Unsupported (deprecated).
7 slave_event Unsupported (deprecated).
8 create_file_event Unsupported (deprecated).
9 append_block_event Unsupported.
10 exec_load_event Unsupported (deprecated).
11 delete_file_event Unsupported.
12 new_load_event Unsupported (deprecated).
13 rand_event Fully supported with all fields parsed.
14 user_var_event Fully supported with all fields parsed.
15 format_description_event Fully supported with all fields parsed.
16 xid_event Fully supported with all fields parsed.
17 begin_load_query_event Unsupported.
18 execute_load_query_event Unsupported.
19 table_map_event Fully supported with all fields parsed.
20 pre_ga_write_rows_event Unsupported (deprecated).
21 pre_ga_update_rows_event Unsupported (deprecated).
22 pre_ga_delete_rows_event Unsupported (deprecated).
23 write_rows_event_v1 Fully supported with all fields parsed.
24 update_rows_event_v1 Fully supported with all fields parsed.
25 delete_rows_event_v1 Fully supported with all fields parsed.
26 incident_event Unsupported.
27 heartbeat_log_event Unsupported.
28 ignorable_log_event Unsupported.
29 rows_query_log_event Fully supported with all fields parsed.
30 write_rows_event_v2 Fully supported with all fields parsed.
31 update_rows_event_v2 Fully supported with all fields parsed.
32 delete_rows_event_v2 Fully supported with all fields parsed.
33 gtid_log_event Fully supported with all fields parsed.
34 anonymous_gtid_log_event Fully supported with all fields parsed.
35 previous_gtids_log_event Fully supported with all fields parsed.
36 transaction_context_event Unsupported.
37 view_change_event Unsupported.
38 xa_prepare_log_event Unsupported.
50 table_metadata_event Specific to Twitter MySQL 5.5.24.t7+. Fully supported with all fields parsed.

Data Types Supported in Row Events

ID SQL Data Type MySQL Internal Type Status
Numeric Types
1 TINYINT MYSQL_TYPE_TINY Fully supported.
2 SMALLINT MYSQL_TYPE_SHORT Fully supported.
9 MEDIUMINT MYSQL_TYPE_INT24 Fully supported.
3 INT MYSQL_TYPE_LONG Fully supported.
8 BIGINT MYSQL_TYPE_LONGLONG Fully supported.
4 FLOAT MYSQL_TYPE_FLOAT Fully supported.
5 DOUBLE MYSQL_TYPE_DOUBLE Fully supported.
246 DECIMAL MYSQL_TYPE_NEWDECIMAL Fully supported using BigDecimal.
Temporal Types
7 TIMESTAMP MYSQL_TYPE_TIMESTAMP Fully supported.
17 TIMESTAMP(n) MYSQL_TYPE_TIMESTAMP2 Fully supported.
12 DATETIME MYSQL_TYPE_DATETIME Fully supported.
18 DATETIME(n) MYSQL_TYPE_DATETIME2 Fully supported.
10 DATE MYSQL_TYPE_DATE Fully supported.
14 DATE MYSQL_TYPE_NEWDATE Unsupported.
11 TIME MYSQL_TYPE_TIME Fully supported.
19 TIME(n) MYSQL_TYPE_TIME2 Unsupported.
13 YEAR MYSQL_TYPE_YEAR Fully supported.
String Types
15
253
254
CHAR
VARCHAR
MYSQL_TYPE_STRING Fully supported.
249
252
250
251
TINYBLOB
BLOB
MEDIUMBLOB
LONGBLOB
MYSQL_TYPE_BLOB Fully supported.
Other Types
247 ENUM MYSQL_TYPE_STRING Supported, but values returned are internal representations.
248 SET MYSQL_TYPE_STRING Supported, but values returned are internal representations.
16 BIT MYSQL_TYPE_BIT Supported, treated as integer of appropriate size.
255 GEOMETRY MYSQL_TYPE_GEOMETRY Supported, treated as BLOB.
245 JSON MYSQL_TYPE_JSON Supported, treated as BLOB.

More Repositories

1

innodb_ruby

A parser for InnoDB file formats, in Ruby
Ruby
1,658
star
2

innodb_diagrams

Diagrams for InnoDB data structures and behaviors
1,420
star
3

avr

Jeremy Cole's AVR Projects
C
33
star
4

yesmark

YesMark™ — the world's most advanced Open Source benchmarking platform.
15
star
5

Temperature_LM75_Derived

Arduino library for I²C Temperature Sensors derived from the LM75
C++
13
star
6

iohist

Histograms from Linux diskstats
9
star
7

blog-files

Files for reference in blog.jcole.us
Perl
8
star
8

mysql_cache_manager

MySQL Cache Manager: Save and restore InnoDB Buffer Pool contents
Ruby
7
star
9

floating_head

Open Source Pan/Tilt Skype Camera for remote workers
Ruby
6
star
10

Maxim_DS2782

Arduino library for the Maxim DS2782 Stand-Alone Fuel Gauge IC
C++
5
star
11

mysql_management_tools

MySQL Management Tools
Ruby
4
star
12

aviation

Aviation
4
star
13

easypxe

A dynamic DHCP+PXE server
Perl
4
star
14

eagle-projects

Electronics design projects in Eagle by Jeremy Cole
3
star
15

AllSensors_DLHR

Arduino library for the AllSensors DLHR Series Low Voltage Digital Pressure Sensors
C++
3
star
16

avruby

An AVR emulator, crazily written in Ruby.
Ruby
3
star
17

unifi_protect

An unofficial implementation of the Ubiquiti UniFi Protect API in Ruby
Ruby
2
star
18

hello

Hello World, ten times in a loop
Assembly
2
star
19

pegged

Cracker Barrel Peg Game Solver
Ruby
2
star
20

TI_TCA9548A

Arduino library for TI TCA9548A Low-Voltage 8-Channel I²C Switch with Reset
C++
2
star
21

metrics_channel

Ruby
2
star
22

weatherlink

Unofficial Davis Instruments WeatherLink/AirLink API in Ruby
Ruby
1
star
23

AllSensors_DLV

Arduino library for the AllSensors DLV Series Low Voltage Digital Pressure Sensors
C++
1
star
24

headshot

Headshots of myself for use in publications
1
star
25

system-summary

Summarize hardware present in a Linux system
Perl
1
star
26

dot-vim

VIM Configuration
Vim Script
1
star
27

rubulator

A stupid Ruby-based calculator which uses only prefix operators.
Ruby
1
star
28

flipper

Tool for managing access to MySQL master-master pairs
Perl
1
star
29

logic_ruby

Simulating transistors in Ruby, why not?
Ruby
1
star
30

ftc_event

A library for accessing a FIRST Tech Challenge scorekeeper event database
Ruby
1
star
31

metrics-channel-java

Java
1
star
32

buffer_cursor

Ruby
1
star
33

coronavirus

R
1
star
34

patches

Patches to various OSS projects
1
star
35

weather_gov

Ruby API for api.weather.gov
Ruby
1
star
36

game_engine_math_ruby

Musings on Foundations of Game Engine Development, Volume 1: Mathematics
Ruby
1
star