• Stars
    star
    161
  • Rank 233,470 (Top 5 %)
  • Language
    C
  • License
    MIT License
  • Created almost 12 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple utility to classify packets into flows. It's so simple that only one task is aimed to finish. For Deep Packet Inspection or flow classification, it's so common to analyze the feature of one specific flow. I have make the attempt to use made-ready tools like tcpflows, tcpslice, tcpsplit, but all these tools try to either decrease the trace volume (under requirement) or resemble the packets into flow payloads (over requirement). I have not found a simple tool to classify the packets into flows without further processing. This is why this program is born.

pkt2flow

Build Status

by chenxm, Shanghai Jiao Tong Univ. [email protected]

2012-2019

©MIT LICENSED

A simple utility to classify packets into flows. It's so simple that only one task is aimed to finish.

For Deep Packet Inspection or flow classification, it's so common to analyze the feature of one specific flow. I have make the attempt to use made-ready tools like tcpflows, tcpslice, tcpsplit, but all these tools try to either decrease the trace volume (under requirement) or resemble the packets into flow payloads (over requirement). I have not found a simple tool to classify the packets into flows without further processing. This is why this program is born.

The inner function of this program behaves using the 4-tuple (src_ip, dst_ip, src_port, dst_port) to seperate the packets into TCP or UDP flows. Each flow will be saved into a pcap file named with 4-tuple and the timestamp of the first packet of the flow. The packets are saved in the order as read from the source. Any further processing like TCP resembling is not performed. The flow timeout is considered as 30 minutes which can be changed in pkt2flow.h.

How to compile

This program is structured and compiled with a tool called SCons (http://www.scons.org/). You can follow simple steps to make a compile (e.g. Ubuntu):

  1. Make sure you have library libpcap in your system.
sudo apt install -y libpcap-dev
  1. Install "Scons" that can be downloaded from its official website given above.
sudo apt install -y scons
  1. Get source code and run scons under the project folder:
git clone https://github.com/caesar0301/pkt2flow.git
cd pkt2flow
scons # You got binary pkt2flow

How to install (optional)

You can optionally let scons automatically handle the installation for you by providing an installation prefix, e.g.:

$ PREFIX=/usr/local
$ scons --prefix=$PREFIX install

This will build pkt2flow and install the binary to /usr/local/bin/pkt2flow. Depending on where you want to install it, you might need to use sudo or become the appropriate user.

Usage

Usage: ./pkt2flow [-huvx] [-o outdir] pcapfile

	Options:
		-h	print this help and exit
		-u	also dump (U)DP flows
		-v	also dump the in(v)alid TCP flows without the SYN option
		-x	also dump non-UDP/non-TCP IP flows
		-o	(o)utput directory

More Repositories

1

awesome-pcaptools

A collection of tools developed by other researchers in the Computer Science area to process network traces. All the right reserved for the original authors.
3,101
star
2

treelib

An efficient implementation of tree data structure in python 2/3.
Python
806
star
3

http-sniffer

A multi-threading tool to sniff TCP flow statistics and embedded HTTP headers from PCAP file. Each TCP flow carrying HTTP is exported to text file in json format.
C
185
star
4

graphsim

Graph similarity algorithms based on NetworkX.
Python
163
star
5

city-cellular-traffic-map

City Cellular Traffic Map (C2TM)
41
star
6

s2g

(S)hapefile to(2) (G)raph/network converter in Python
Python
23
star
7

pcapdpi

Using nDPI/openDPI to detect flow protocols from a PCAP file or live NIC. This program was modified from example in nDPI and I added a periodically cleaning of flow tree to save memory.
HTML
22
star
8

libwayback

A library to parse Wayback Machine of archive.org to get a historical views of web pages. It is a useful tool to research on the evolution of web pages, page structure analysis, and among other interesting topics.
Python
20
star
9

warpdrive

Useful tools to drive personal daily work.
Shell
12
star
10

movr

Human mobility data (in form of <x,y,t>) analysis and visualization in R.
R
11
star
11

ppai

上海车牌拍卖的读秒数据,以及分析数据的若干R函数
R
10
star
12

relogger

A relayer or replicator to send SYSLOG from one or multiple sources to one or multiple destinations.
Python
8
star
13

paper-mstd-code

Data and source code of paper "Analyzing and Modeling Spatio-Temporal Dependence of Cellular Traffic at City Scale", ICC 2015
R
6
star
14

flowmap

Apache Spark utilities to analyze mobile network deployment and user behavior data.
Scala
6
star
15

omnilab-misc

Miscellanous tools for daily work at OMNI-Lab, SJTU. (NO RENAME)
Python
6
star
16

FileTypeDB

A database of common file types on Internet. You can use the database files to determine file distirbution in your apps.
Python
6
star
17

paper-flowmap-code

Tools to mine motifs and meta-structures of human mobility.
Python
4
star
18

phd-dissertation

博士论文《利用移动网络数据的人类时空行为分析与建模研究》,欢迎交流切磋。
TeX
4
star
19

http-dumper

Similar to tcpdump, but it is just to dump packets header (up to HTTP header end) by sniffing NIC in a live mode.
C
3
star
20

FlexVersion

Python library to manipulate Semantic Versioning
Python
3
star
21

cool-dotfiles

Personal collection of dotfiles
Shell
3
star
22

mobox

Python toolkit (library) to process spatiotemporal mobility data, inc. mobility graphs, shapefiles etc. This project was born from the iteration of experimental code of my academic paper on Mobility MetaStructures.
Python
3
star
23

catalog

Under development
Python
2
star
24

Pansdon

Web crawlers. PanSiDong, 盘丝洞
Python
2
star
25

arkose

Knowledge oriented search engine on premise
Python
1
star
26

zshrc

Shell
1
star
27

mrmlf

MultilineInputFormat (MLF) for hadoop MapReduce (v2).
Java
1
star
28

pywpt-batch

This program intends to do the batch testing with WPT and records the results.
Python
1
star
29

paper-tpca-code

Source code of TPCA paper: Trajectory pattern change analysis in campus WiFi networks (MobiGIS2013)
Java
1
star
30

WPI

Web Page Identification from Modern Internet Traffic.
Python
1
star
31

omniR

Package containing useful functions in my researches of human behavior science, mostly for spatio-temporal data.
R
1
star
32

piggybox

Useful Pig UDFs including bag operations, HTTP analysis, UUID etc.
Java
1
star