• Stars
    star
    149
  • Rank 248,619 (Top 5 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Binary Processing Language

BPL - Binary Processing Language

Build Status Go Report Card GitHub release Coverage Status GoDoc

快速入门

了解 BPL 最快的方式是学习 qbpl 和 qbplproxy 两个实用程序:

qbpl

qbpl 可用来分析任意的文件格式。使用方法如下:

qbpl [-p <protocol>.bpl -o <output>.log] <file>

多数情况下,你不需要指定 -p <protocol>.bpl 参数,我们根据文件后缀来确定应该使用何种 protocol 来解析这个文件。例如:

qbpl 1.gif

不过为了让 qbpl 能够找到所有的 protocols,我们需要先安装:

make install # 这将将所有的bpl文件拷贝到 ~/.qbpl/formats/

qbplproxy

qbplproxy 可用来分析服务器和客户端之间的网络包。它通过代理要分析的服务,让客户端请求自己来分析请求包和返回包。使用方式如下:

qbplproxy -h <listenIp:port> -b <backendIp:port> [-p <protocol>.bpl -f <filter> -o <output>.log]

其中,<listenIp:port> 是 qbplproxy 自身监听的IP和端口,<backendIp:port> 是原始的服务。-f <filter> 是过滤条件,这个条件通过 BPL_FILTER 全局变量传递到 bpl 中。

多数情况下,你不需要指定 -p <protocol>.bpl 参数,qbplproxy 程序可以根据你监听的端口来猜测网络协议。例如:

mongod --port 37017
qbplproxy -h localhost:27017 -b localhost:37017

我们会依据端口 27017 知道你要分析的是 mongodb 的网络协议。

BPL 文法

请参见 BPL 文法

网络协议研究

RTMP 协议

格式描述:

测试:

  1. 启动一个 rtmp server,让其监听 1936 端口(而不是默认的 1935 端口)。比如我们可以用 node-rtsp-rtmp-server
git clone [email protected]:iizukanao/node-rtsp-rtmp-server.git
cd node-rtsp-rtmp-server
修改 config.coffee,将:
  * rtmpServerPort: 1935 改为 rtmpServerPort: 1936;
  * serverPort: 80 改为 serverPort: 8080(这样就不用 sudo 来运行了)
coffee server.coffee
  1. 启动 qbplproxy:
qbplproxy -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee rtmp.log
  1. 推流:
ffmpeg -re -i test.m4v -c:v copy -c:a copy -f flv rtmp://localhost/live/123
  1. 播流:

在 Mac 下可以考虑用 VLC Player,打开网址 rtmp://localhost/live/123 进行播放即可。

  1. 选择性查看

有时候我们并不希望看到所有的信息,rtmp.bpl 支持以 flashVer 作为过滤条件。如:

qbplproxy -f 'flashVer=LNX 9,0,124,2' -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee <output>.log

或者我们直接用 reqMode(用来区分是推流publish还是播流play) 来过滤。如:

qbplproxy -f 'reqMode=play' -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee <output>.log

这样就可以只捕获 VLC Player 的播流过程了。

当然,其实还有一个不用过滤条件的办法:就是让推流直接推到 rtmp server,但是播流请求发到 qbplproxy。

FLV 协议

格式描述:

测试:

  1. 启动一个 rtmp/flv server,让其监听 1935/8135 端口。

  2. 启动 qbplproxy:

qbplproxy -h localhost:8888 -b localhost:8135 -p formats/flv.bpl | tee flv.log
  1. 推流:
ffmpeg -re -i test.m4v -c:v copy -c:a copy -f flv rtmp://localhost/live/123
  1. 播流:

在 Mac 下可以考虑用 VLC Player,打开网址 http://localhost:8888/live/123.flv 进行播放即可。

WebRTC 协议

格式描述:

MongoDB 协议

格式描述:

测试:

  1. 启动 MongoDB,让其监听 37017 端口(而不是默认的 27017 端口):
./mongod --port 37017 --dbpath ~/data/db
  1. 启动 qbplproxy:
qbplproxy -h localhost:27017 -b localhost:37017 -p formats/mongo.bpl | tee mongo.log
  1. 使用 MongoDB,比如通过 mongo shell 操作:
./mongo

文件格式研究

MongoDB binlog 格式

TODO

MySQL binlog 格式

TODO

HLS TS 格式

格式描述:

测试:TODO

FLV 格式

格式描述:

测试:TODO

MP4 格式

格式描述:

测试:

qbpl -p formats/mp4.bpl <example>.mp4

GIF 格式

格式描述:

测试:

qbpl -p formats/gif.bpl formats/1.gif

More Repositories

1

gop

The Go+ programming language is designed for engineering, STEM education, and data science
Go
8,682
star
2

c2go

Convert C to Go
Go
304
star
3

gox

Code generator for the Go language
Go
139
star
4

spx

spx - A Scratch Compatible Go/Go+ 2D Game Engine for STEM education
Go
97
star
5

igop

The Go/Go+ Interpreter
Go
97
star
6

tutorial

Tutorials for GoPlus (The Go+ Language)
Go
38
star
7

hdq

HTML DOM Query Language for Go+
Go
38
star
8

reflectx

Golang reflect package hack tools
Assembly
28
star
9

yaigop

Yet Another Go+ interpreter (still in beta version)
Go
26
star
10

www

Source code of https://goplus.org/
Go
26
star
11

pandas

Flexible and powerful data analysis / manipulation library for Go+, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
25
star
12

builder

Go+ Builder
Vue
25
star
13

libc

Porting libc from C to Go
Go
24
star
14

pyg

Python to Go tools
Go
13
star
15

community

Go+ Community written in Go+
JavaScript
11
star
16

llgo

A Go compiler based on LLVM in order to better integrate Go with the C ecosystem
Go
9
star
17

ispx

The Interpreter of Go+ spx engine
Go
8
star
18

yap

Yet Another Go/Go+ HTTP Web Framework
Go
8
star
19

sqlite

sqlite for Go+
Go
7
star
20

spx-editor

Spx editor
JavaScript
6
star
21

igo

Yet another interpreter for the Go language
Go
6
star
22

mod

Module support for Go/Go+
Go
6
star
23

canvas

HTML Canvas 2D Context API for mobile, desktop and web
Go
5
star
24

FlappyCalf

FlappyCalf - a game powered by Go+ spx game engine
Shell
3
star
25

goplus.github.io

Thoughts on Go+
2
star
26

ipkg

Go
2
star
27

setup-goplus

Set up your GitHub Actions workflow with a specific version of Go+
TypeScript
2
star
28

installer

Go+ Installer for Windows, MacOS, etc.
Go
2
star
29

gopdlv

gopdlv is a debugger for the Go/Go+ programming language
Go
2
star
30

llama2

LLGo support for Inference Llama 2
LLVM
2
star
31

MazePlay

MazePlay - a game powered by Go+ Builder
Shell
1
star
32

AircraftWar

AircraftWar - a game powered by Go+ spx game engine
Shell
1
star
33

xtypes

Golang types convert utils
Go
1
star
34

classfile-project-template

A template for starting a new classfile project
1
star
35

expect

Expect for Go+
Go
1
star
36

show

show.goplus.org
Go
1
star
37

xtools

Extended tools for Go+
Go
1
star
38

run

run.goplus.org
Go
1
star
39

llgoexamples

LLGo Examples
Rust
1
star
40

account

Go+ Account System
1
star
41

goxls

The Go/Go+ Language Server
Go
1
star