roma-glushko/tango

Tango

Tool to get insights from the server access logs

Tango is a dependency-free command-line tool for analyzing access logs 💃

Currently, work on this project is in progress. However, a few pre-releases are ready available to use 🎉

Table of Contents

• Installation
• Usage
• Filters
• Report Commands
• Misc Commands
• Config File

Installation

macOS

Tango can be installed on macOS via Homebrew:

brew tap roma-glushko/tango
brew install roma-glushko/tango/tango

To upgrade, try to run:

brew upgrade tango

Linux

Tango is available on Linux via Snapcraft. This means that Tango can be installed on:

• Ubuntu
• Debian
• CentOS
• openSUSE
• Linux Mint
• Fedora
• Kubuntu
• elementary OS
• Arch Linux
• KDE Neon
• Manjaro

To upgrade, try to run:

snap refresh tango

Windows

Tango can be installed on Windows via Scoop:

scoop bucket add tango https://github.com/roma-glushko/scoop-tango.git
scoop install tango

To upgrade, try to run:

scoop update tango

Usage

List of available commands:

tango help

Tango Version:

tango -v

Global Options

Filters

// IP filters
tango --ip-filter "127.0.0.1" custom -l access-log.log -r custom.csv
tango --keep-ip-filter "8.8.8.8" custom -l access-log.log -r custom.csv

// URI filters
tango --uri-filter "/test-page" custom -l access-log.log -r custom.csv
tango --keep-uri-filter "/admin/" custom -l access-log.log -r custom.csv

// Time Frame filter
tango --keep-time-filter "2019-09-15 04:16:00 -0400" --keep-time-filter "2019-09-15 04:35:00 -0400" custom -l access-log.log -r custom.csv

// User Agent filters
tango --ua-filter "iPhone OS 12_3_1 like Mac OS X" custom -l access-log.log -r custom.csv
tango --keep-ua-filter "iPhone OS 12_3_1 like Mac OS X" custom -l access-log.log -r custom.csv

// Asset filter
tango --asset-filter "/pub/static/" --asset-filter "/pub/media/" custom -l access-log.log -r custom.csv

// System IP filter
tango --system-ips "127.0.0.1"  --system-ips "1.2.3.4" custom -l access-log.log -r custom.csv

Other

// Base URL info
tango --base-url "https://example.com/" custom -l access-log.log -r custom.csv

Report Commands

Custom Reports

tango --keep-uri-filter "/newsletter/subscriber/new/" custom -l access-log.log -r custom.csv

Use cases:

• generate a report with all requests from a certain IP
• generate a report with all requests to a certain URL

Geo Reports

tango geo -l access-log.log -r custom.csv

Geo Report uses MaxMind Geo lib to get Geo information. See Geo Lib command for more info.

Use cases:

• collects geo information about all IPs that requested the website
• get request distribution by IP with geo information
• see all IPs sorted by countries/continents/cities

Example of the report:

Browser Reports

tango browser -l access-log.log -r custom.csv

Use cases:

• check how many requests were sent by crawlers
• check what kind of browsers requested the website
• check bandwith that was transmitted to all kind of browsers
• check what crawlers requested the website

Request Reports

tango request -l access-log.log -r custom.csv

Use cases:

• check how many requests were sent to a certain URL
• check all URLs that were responded with 404/50X code
• find requests from security scanners (sort by response codes and look at 404/50X codes which were requested only 1 time)

Pace Reports [Experimental]

tango pace -l access-log.log -r custom.csv

Use cases:

• check which IPs and how many requests they made during a certain time frame
• check count of requests per minutes/hours

Journey Reports [Experimental]

tango journey -l access-log.log -r custom.csv

Misc Commands

Geo Lib

// Install geo library to be able to generate geo reports
tango geo-lib

Tango uses the MaxMind GeoLite2-City database and stores it under:

• macOS - /Users/[username]/.tango/GeoLite2-City.mmdb

To be able to manage the Geo lib, you need to generate acceses under MaxMind Account page

Config File

Put the similar content to a .tango.yaml file under your working directory where you analyze logs:

"asset-filter":
  - "/pub/static/"
  - "/pub/media/"
  - "/media/"
  - "/static/"
"ip-filter":
  - "127.0.0.1"
"system-ips":
  # Fastly IPs
  - "23.235.32.0/20"
  - "43.249.72.0/22"
  - "103.244.50.0/24"
  - "103.245.222.0/23"
  - "103.245.224.0/24"
  - "104.156.80.0/20"
  - "151.101.0.0/16"
  - "157.52.64.0/18"
  - "167.82.0.0/17"
  - "167.82.128.0/20"
  - "167.82.160.0/20"
  - "167.82.224.0/20"
  - "172.111.64.0/18"
  - "185.31.16.0/22"
  - "199.27.72.0/21"
  - "199.232.0.0/16"