• Stars
    star
    227
  • Rank 175,900 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created over 6 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Interface to manage and centralize Google Alert information

Chirp

Chirp is a metadata platform built on top of Google Alerts meant to manage monitors, alerts and notifications. It's built on an unofficial abstraction API from the web interface, written in python.

Demo interface of Chirp filled with alerts

Getting Started

Install MongoDB and ensure it's running:

https://www.mongodb.com/

Setup a virtual environment for Python3:

$ sudo virtualenv -p python3 venv3

Activate your container:

$ source venv3/bin/activate

Install the requirements:

$ (venv3) pip install -r requirements.txt

Start redis:

$ redis-server

Start RabbiqMQ:

$ rabbitmq-server

Start the Celery beat:

$ (venv3) sudo celery worker -A celery_worker.celery --loglevel=info -B --concurrency=1

Start the server:

$ (venv3) sudo python server.py run

How Does it Work?

Chirp takes a set of Google credentials and makes use of the python Google alerts abstract library to create, update and delete monitors. Each monitor is turned into an RSS feed and then managed within Chirp. Asynchronous jobs poll the RSS feeds for new articles, summarizing the content, deriving tag data and then presenting them within the interface. All alert data is presented in one location with additional metadata paired from the user.

Why?

Google Alerts by themselves are great for a small number of focused monitors. If you want to direct the Google machine to a larger set of queries, it can quickly become overwhelming for managing and then handling alerts. Chirp was built for cases where Google Alerts was managing a lot of monitors and users needed a way to prioritize the alerting information that came back. Chirp completely abstracts away the Google Alert interface and provides a framework for collecting data on specific topics using Google scale.

State of the Project

This platform is a work in progress and is being developed on an "as needed" basis. In other words, not all functionality is implemented and it's far from polished.

More Repositories

1

google-alerts

Python library for automating the administration of Google Alerts.
Python
84
star
2

netinfo

Simple IP enrichment service and API wrapping PyASN and MaxMind GeoIP.
Python
71
star
3

pdfxray_public

Public version of PDF X-RAY
Python
59
star
4

malpdfobj

Builds json representation of PDF malware sample
Python
49
star
5

heavy_pint

Library used to generate malicious PDF files for testing
PHP
38
star
6

pdfxray_lite

Lite version of PDF X-RAY that uses no backend
Python
34
star
7

pcap_tools

Help summarize a PCAP file
Python
33
star
8

pastycake

Search pastebin for keywords using python
Python
33
star
9

frisbee

Collect email addresses by crawling search engine results.
Python
28
star
10

lazyeye

Chrome extension to extract data from websites surfed inside of chrome
JavaScript
18
star
11

classyPDF

Classify PDF documents based on PDF X-RAY data
Python
18
star
12

boxcar

Process the fortune 1000 domains to identify live typo-sites. Save results into a database for later processing and analysis.
Python
11
star
13

opendns_utils

OpenDNS CLI Utilities
Python
8
star
14

Dirtyhands

Searchs Google for a certain filetype from malicious domains
Python
8
star
15

jetplane

Library to surf through TOR nodes and collect statistics about each trip taken
6
star
16

streamray

Automatically extract artifacts out of a PDF through the command line
6
star
17

Bighands

Grabs files from Google of a certain file type
Python
6
star
18

groundtruth

Data analysis on coffee cupping scores
JavaScript
6
star
19

swarm

Turn browsers into egress points to fetch URL content.
JavaScript
5
star
20

doomsday_encoder

JS Encoder using new techniques
PHP
5
star
21

bookworm

Summarize RSS feeds and store in a database
Python
5
star
22

scripts

Scripts used in research
Python
5
star
23

jooter

Joomla bruteforcer with network concurrency
Python
4
star
24

pynetinfo

Python library wrapper for the Netinfo service
Python
4
star
25

jSneak

Deliver JavaScript Payload on AJAX Call
4
star
26

fatbeagle

FatBeagle provides an interface to send observed URLs to a remote server.
JavaScript
3
star
27

app-strap

Skeleton python2/3 flask application that includes authentication with a mongo backend
Python
3
star
28

Mods

Modifications of other tools
Python
2
star
29

dpacker

Build document packs based on keywords
Python
2
star
30

electron-kqed

Get KQED directly to your Mac with this Electron app!
JavaScript
2
star
31

shinwa

bot to do things for me
Python
2
star
32

bluekeep-marketing

BlueKeep marketing page to organize information around the vulnerability
HTML
1
star
33

openCVSS

Older OpenCVSS classes and code from dueyesterday.net
Python
1
star
34

tf-float-classifier

Classify images of float numbers found within images and guess their true value
1
star
35

weekly_report

JavaScript
1
star
36

picklejar

Simple abstraction over cPickle to help organize pickled items and restore them later on
Python
1
star
37

shortfuse

Generate encoded content based on URL key to be used in Chrome Extensions
Python
1
star
38

angry_jon

HTML5 testing site
PHP
1
star