• Stars
    star
    240
  • Rank 168,229 (Top 4 %)
  • Language
    Python
  • License
    GNU General Publi...
  • Created about 6 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Passive service locator, a python sniffer that identifies servers, clients, names and much more

passer

Passive service locator, a python sniffer that identifies servers, clients, names and much more

Introduction

Quick notes for getting going with passer, the passive service sniffer. You're responsible for getting permission to sniff.

Passer can work off a live packet capture or from a pcap file (command line parameter, see examples below). It reports live services and clients, ethernet cards on the lan, dns entries, operating systems, and routers - all passively!

If you're using windows or your paths to the support files don't match mine for some other reason, let me know where they are and I'll be glad to update the script.

Installation

Requirements

  • Python >=2.4 and <3.0
  • Python libraries (see requirements.txt)
    • ipaddress
    • pytz
    • scapy>=2.4.0

Optional (but recommended)

  • nmap
    • for Ethernet manufacturers and service strings
  • ettercap, wireshark, and/or arp-scan
    • for more Ethernet manufacturers
  • p0f version 2
    • for the OS fingerprint file

Ubuntu/Debian and deb-based distros

# Install system dependencies
sudo apt-get install arp-scan ettercap-text-only nmap wireshark 
# Install python dependencies
sudo pip install -r requirements.txt
# Prevent Scapy from performing DNS lookups
echo 'noenum = [ Resolve(), TCP_SERVICES, UDP_SERVICES ]' >> ~/.scapy_startup.py

Redhat/CentOS/Fedora and rpm-based distros

# Install system dependencies
sudo yum install arp-scan ettercap nmap wireshark
# Install python dependencies
sudo pip install -r requirements.txt
# Prevent Scapy from performing DNS lookups
echo 'noenum = [ Resolve(), TCP_SERVICES, UDP_SERVICES ]' >> ~/.scapy_startup.py

Windows (incomplete)

To install scapy, see the installation guide.

Docker

Passer also comes packaged as a Docker image. If you don't already have Docker here is a quick and dirty way to install it on Linux:

curl -fsSL https://get.docker.com | sh -

Otherwise, follow the install instructions for your operating system.

For most uses, we recommend the passer script included in this repo. This script will handle all docker-specific

wget https://raw.githubusercontent.com/activecm/passer/master/passer
chmod +x passer

You can then use this script just as you would in any of the examples below. For example:

./passer -i eth0
# The equivalent without using the included script would be:
docker run --rm --name=passer -i --init --net=host --cap-add=net_raw activecm/passer -i eth0

In order to stop passer run:

docker stop passer

Examples

Sniff live as root

/path/to/passer.py

This sniffs from all network interfaces and sends all output lines to your console.

Sniff live as a non-root user

sudo /path/to/passer.py

or

su - -c '/path/to/passer.py'

Sniff live as root, but only from one interface

/path/to/passer.py -i IfaceName

Running route should give some live interfaces you might use.

-i is incompatible with -r.

Read packets from a pcap file; no root privileges needed

/path/to/passer.py -r /path/to/packets.pcap

-r is incompatible with -i.

Accept raw pcap data on stdin

cat packetdata.pcap | ./passer.py -r /proc/self/fd/0
zcat packetdata.pcap.gz | ./passer.py -r /proc/self/fd/0
bzcat packetdata.pcap.bz2 | ./passer.py -r /proc/self/fd/0
tcpdump -i eth0 -qtnp -w - | ./passer.py -r /proc/self/fd/0
# etc...

This lets you capture packets with any tool that can save packets to a pcap file, and later process them with passer on a different system.

Save output lines to a text file for later processing

/path/to/passer.py -l /path/to/networkinfo.txt

Suppress warnings and other debugging info

/path/to/passer.py 2>/dev/null

Show help screen

/path/to/passer.py -h

Save "odd"/unhandled packets to a pcap file

/path/to/passer.py -u /path/to/oddpackets.pcap

This is generally intended for the development process; packets saved to this file are ones that need to have signatures written. If you'd like to help improve the program, get in touch with the author, Bill Stearns ([email protected]). Contributions of odd packets, descriptions of services, and patches to the program are gratefully accepted.

Apply a BPF filter to limit which packets are processed

This should be as simple as placing the BPF filter in single quotes at the end of the command line. As of version 1.16, the underlying library does not appear to successfully use the supplied filter, but there's a workaround. Use tcpdump to do the filtering, and hand the pared-down set of packets to passer on stdin, like above:

tcpdump -r packets.pcap -w - 'icmp or arp' | ./passer.py -r /proc/self/fd/0

See the "Sample filters" section, below, for some suggestions of filters to use in either capturing packets in advance or live sniffing.

Troubleshooting

socket.error: (1, 'Operation not permitted')

You're probably trying to sniff live as a non-root user. Either log in as root, use sudo/su, or work with pcap files.

If passer crashes or won't work on your system, send me an email (Bill Stearns, [email protected]). It would be very helpful if you could include the error message, if any, and any details about your operating system.

Output format

Passer's output goes to stdout, and if you give the command line parameter -l /path/to/logfile, to that file as well. Here's the format:

Type IPAddr Proto State Optional description (may be empty)
'IP' IPaddr 'IP' dead or live p0f OS description
'MA' IPaddr 'Ethernet' MacAddr ManufDescription
'TC' IPaddr 'TCP_'Port closed or open client description
'TS' IPaddr 'TCP_'Port closed or listening server description
'UC' IPaddr 'UDP_'Port open or closed udp client port description
'US' IPaddr 'UDP_'Port open or closed udp server port description
'DN' IPaddr 'A' or 'PTR' hostname possible extra info
'RO' IPaddr 'TTLEx' router possible extra info
  • Column 1: A 2 letter code for the record type
  • Column 2: The IP address being characterized.
  • Column 3: The protocol involved, or "how do we know this?"
  • Column 4: the state of the object being described
  • Column 5: additional information about the object

Lines are comma separated for easy loading into a spreadsheet or SQL import. None of the fields should contain commas. There is a sample output file at http://www.stearns.org/passer/passer-sample-log.txt .

Here are some examples of how to get the data you want out of these lines:

  1. Remove duplicate lines:
cat /var/tmp/passer-log | sort -u | less
  1. Remove duplicate lines and group all records for a given IP together:
cat /var/tmp/passer-log | sort -t, -k2 -u | less
  1. Grab just the DNS and Router records:
cat /var/tmp/passer-log | sort -u | egrep '(^DN|^RO)' | less
  1. See all records for a particular IP address:
cat /var/tmp/passer-log | sort -u | grep ',192\.168\.0\.17,' | less
  1. See all records for a particular network:
cat /var/tmp/passer-log | sort -u | grep ',192\.168\.' | less
  1. See all machines that are listing on TCP port 25 (smtp servers):
cat /var/tmp/passer-log | sort -u | grep ',TCP_25,listening,' | less
  1. Don't display closed ports:
cat /var/tmp/passer-log | sort -u | grep -v ',closed,' | less
  1. Only display closed ports:
cat /var/tmp/passer-log | sort -u | grep ',closed,' | less
  1. Show all DNS records in the "google.com" domains:
cat /var/tmp/passer-log | sort -u | grep -i 'google\.com\.,' | less
  1. Grab all the DNS address records and create a hosts-like file:
/path/to/make-hosts /var/tmp/passer-log | /path/to/mergehosts.pl >/var/tmp/passer-hosts

"make-hosts" and merge-hosts are separate shell and perl scripts at http://www.stearns.org/passer/make-hosts .

This is just a start! Send in your favourite ways to extract items of interest and I'll add them and give you credit.

BPF filters

The individual record types need certain types of packets to give them their raw data. If you want to include or exclude these, use the accompanying filter.

'IP', IPaddr, 'IP', dead or live, p0f OS description

This needs TCP SYN packets:

'tcp[13] & 0x12 = 0x02'

'MA', IPaddr, 'Ethernet', MacAddr, ManufDescription

These come from arp replies(*1):

'arp'

'TC', IPaddr, 'TCP_'Port, closed or open, client description

These need TCP SYN/ACK's, FIN's and RST's(*2):

'tcp[13] & 0x07 != 0'

'TS', IPaddr, 'TCP_'Port, closed or listening, server description

We need TCP SYN's, SYN/ACK's, and RST's to see if the port is open or closed:

'tcp[13] & 0x06 != 0'

To come up with a server description string, we also need to see the ACK packets that make up the bulk of the traffic on the wire:

'tcp[13] & 0x17 = 0x10'

If you want both, just grab all TCP traffic:

'tcp'

'UC', IPaddr, 'UDP_'Port, open or closed, udp client port description

Easiest to just hand it all udp ports and ICMP port unreachables:

'udp or icmp[0:2] = 0x0303'

'US', IPaddr, 'UDP_'Port, open or closed, udp server port description

Same as above:

'udp or icmp[0:2] = 0x0303'

'DN', IPaddr, A,AAAA,PTR,CNAME hostname, possible extra info

UDP or TCP port 53 (*3):

'udp src port 53 or tcp src port 53'

'RO', IPaddr, 'TTLEx', router, possible extra info

We identify routers because they're sending Time-To-Live-Exceeded or unreachable messages:

'icmp[0:2] = 0x0B00 or icmp[0] = 0x03'

*1 This grabs arp requests too, but these are ignored.

*2 The filter technically includes SYNs as well, but that's a small amount of extra data

*3 As of version 1.16, only UDP port 53 answers are extracted.

Sample BPF filters

  1. To drastically reduce the number of packets to be parsed, losing only the tcp server description strings, don't process ACK-only packets:
'not(tcp[13] & 0x17 = 0x10)'

This chops out 90+% of the number and volume of packets to be handled, letting passer keep up with moderate bandwidth links.

  1. If you're not interested in DNS servers:
'not udp src port 53'

There's quite a bit of work to extract dns records; this may also be a good one to turn off if you're trying to keep up with a fast link.

  1. If you want to focus on packets to or from a particular machine or network:
'host 1.2.3.4'
'net 1.2'

More info

Questions? Bug reports? Issues? Try [email protected] and please include "passer" somewhere in the subject line.

More Repositories

1

rita-legacy

Real Intelligence Threat Analytics (RITA) is a framework for detecting command and control communication through network traffic analysis.
Go
2,500
star
2

BeaKer

Beacon Kibana Executable Report. Aggregates Sysmon Network Events With Elasticsearch and Kibana
Shell
281
star
3

threat-tools

Tools for simulating threats
Python
166
star
4

rita

Real Intelligence Threat Analytics (RITA) is a framework for detecting command and control communication through network traffic analysis.
Go
97
star
5

threat-hunting-labs

Collection of walkthroughs on various threat hunting techniques
HTML
73
star
6

espy

Endpoint detection for remote hosts for consumption by RITA and Elasticsearch
Go
66
star
7

docker-zeek

Run zeek with zeekctl in docker
Shell
44
star
8

smudge

Passive OS detection based on SYN packets without Transmitting any Data
Python
42
star
9

pcap-stats

Learn about a network from a pcap file or reading from an interface
Python
25
star
10

bro-install

An Installation Script for Bro IDS on Debian Based Systems
Shell
19
star
11

zcutter

Extracts fields from zeek logs, compatible with zeek-cut
Python
17
star
12

devprof

Device profile: Define acceptable amounts of traffic for your devices and see a report of outliers.
Python
16
star
13

sniffer-template

Template for building a packet sniffer
Python
15
star
14

zeek-open-connections

Zeek
12
star
15

ipfix-rita

Collect IPFIX / Netflow v9 Records and Ship them to RITA for Analysis
Go
10
star
16

rita-bl

Real Intelligence Threat Analytics -- Blacklist Database
Go
8
star
17

mongo-diff

A Python script for diff'ing mongo databases
Python
8
star
18

zeekcfg

A node.cfg generator for zeekctl
Go
6
star
19

zeek-log-transport

This script ships logs from Zeek to AC-Hunter
Shell
5
star
20

pi_show

Python script/library for displaying text and graphics on Raspberry Pi PiOled Hat
Python
5
star
21

shell-lib

Shell Scripts Used Across ActiveCM Projects
Shell
5
star
22

certificate-issues

Identifies certificate problems from Zeek ssl log files
Shell
5
star
23

pi_project_installer

A support library and set of scripts to simplify installing software on the Raspberry Pi/Raspbian
Shell
5
star
24

rita-blacklist

Real Intelligence Threat Analytics -- Blacklist Database
Go
5
star
25

mgosec

A Small Helper Library For Securing MongoDB Connections with Golang
Go
4
star
26

bro-rita

A bro plugin for writing log data to MongoDB for use with RITA
C++
3
star
27

safelist-tools

Tools for working with the safelist (formerly whitelist)
Go
3
star
28

docker-ca

A Docker Image For OpenSSL Certificate Authorities (For Testing)
Shell
2
star
29

pcap-resources

Support files and tools for pcap analysis and packet capture
2
star
30

zeek-log-clean

Delete Zeek log files until disk usage is under a given threshold
Shell
2
star
31

save_json_stream

JSON TCP stream importer for RITA and AC-Hunter
Python
1
star
32

DBTest

Managed Integration Testing Dependencies via Docker for Go
Go
1
star
33

bro-rita-test

Compares bro-rita against rita's built in parsing
Shell
1
star
34

bro-mongodb

C++
1
star