• Stars
    star
    161
  • Rank 233,470 (Top 5 %)
  • Language
    Python
  • Created over 8 years ago
  • Updated about 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Download streaming tweets that match specific keywords, and dump the results to a file.

Twitter Scrape

Scrape tweets from twitter into a DB. Convert the DB to a CSV file.

Installation

  • pip install -r requirements.txt

Setup

  • Create a file called private.py.
  • Sign up for a Twitter developer account.
  • Create an application here.
  • Set the following keys in private.py. You can get these values from the app you created:
    • TWITTER_KEY
    • TWITTER_SECRET
    • TWITTER_APP_KEY
    • TWITTER_APP_SECRET
  • Set the following key in private.py.
    • CONNECTION_STRING -- use sqlite:///tweets.db as a default if you need to. It's recommended to use postgresql, but not necessary.

Usage

  • python scrape.py to scrape. Use Ctrl + C to stop.
  • python dump.py to generate tweets.csv, which contains all the tweet data that was scraped.
  • If you want to edit behavior, change settings in settings.py.