caesar0301/piggybox

This repository has been archived on 07/Feb/2021
Stars
1
Language
Java
License
GNU General Publi...
Created almost 11 years ago
Updated over 9 years ago

caesar0301/piggybox

caesar0301

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Useful Pig UDFs including bag operations, HTTP analysis, UUID etc.

awesome-pcaptools

A collection of tools developed by other researchers in the Computer Science area to process network traces. All the right reserved for the original authors.

treelib

An efficient implementation of tree data structure in python 2/3.

http-sniffer

A multi-threading tool to sniff TCP flow statistics and embedded HTTP headers from PCAP file. Each TCP flow carrying HTTP is exported to text file in json format.

graphsim

Graph similarity algorithms based on NetworkX.

pkt2flow

A simple utility to classify packets into flows. It's so simple that only one task is aimed to finish. For Deep Packet Inspection or flow classification, it's so common to analyze the feature of one specific flow. I have make the attempt to use made-ready tools like tcpflows, tcpslice, tcpsplit, but all these tools try to either decrease the trace volume (under requirement) or resemble the packets into flow payloads (over requirement). I have not found a simple tool to classify the packets into flows without further processing. This is why this program is born.

city-cellular-traffic-map

City Cellular Traffic Map (C2TM)

s2g

(S)hapefile to(2) (G)raph/network converter in Python

pcapdpi

Using nDPI/openDPI to detect flow protocols from a PCAP file or live NIC. This program was modified from example in nDPI and I added a periodically cleaning of flow tree to save memory.

libwayback

A library to parse Wayback Machine of archive.org to get a historical views of web pages. It is a useful tool to research on the evolution of web pages, page structure analysis, and among other interesting topics.

warpdrive

Useful tools to drive personal daily work.

movr

Human mobility data (in form of <x,y,t>) analysis and visualization in R.

ppai

上海车牌拍卖的读秒数据，以及分析数据的若干R函数

relogger

A relayer or replicator to send SYSLOG from one or multiple sources to one or multiple destinations.

paper-mstd-code

Data and source code of paper "Analyzing and Modeling Spatio-Temporal Dependence of Cellular Traffic at City Scale", ICC 2015

flowmap

Apache Spark utilities to analyze mobile network deployment and user behavior data.

omnilab-misc

Miscellanous tools for daily work at OMNI-Lab, SJTU. (NO RENAME)

FileTypeDB

A database of common file types on Internet. You can use the database files to determine file distirbution in your apps.

paper-flowmap-code

Tools to mine motifs and meta-structures of human mobility.

phd-dissertation

博士论文《利用移动网络数据的人类时空行为分析与建模研究》，欢迎交流切磋。

http-dumper

Similar to tcpdump, but it is just to dump packets header (up to HTTP header end) by sniffing NIC in a live mode.

FlexVersion

Python library to manipulate Semantic Versioning

cool-dotfiles

Personal collection of dotfiles

mobox

Python toolkit (library) to process spatiotemporal mobility data, inc. mobility graphs, shapefiles etc. This project was born from the iteration of experimental code of my academic paper on Mobility MetaStructures.

catalog

Under development

Pansdon

Web crawlers. PanSiDong, 盘丝洞

arkose

Knowledge oriented search engine on premise

zshrc

mrmlf

MultilineInputFormat (MLF) for hadoop MapReduce (v2).

pywpt-batch

This program intends to do the batch testing with WPT and records the results.

paper-tpca-code

Source code of TPCA paper: Trajectory pattern change analysis in campus WiFi networks (MobiGIS2013)

WPI

Web Page Identification from Modern Internet Traffic.

omniR

Package containing useful functions in my researches of human behavior science, mostly for spatio-temporal data.