• Stars
    star
    125
  • Rank 284,721 (Top 6 %)
  • Language
    Python
  • Created about 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

GetOldTweets-Python is a project written in Python to mine old and backdated tweets, It bypasses some limitations/restrictions of the Twitter API. This Repo houses an improvement fork of the original GetOldTweets Library by [Jefferson Herique](https://github.com/Jefferson-Henrique/GetOldTweets-python). The improvement makes running this package on Windows OS seamless with Python 3.x.

Optimized-Modified-GetOldTweets3-OMGOT

GetOldTweets-Python is a project written in Python to mine old and backdated tweets, It bypasses some limitations/restrictions of the Twitter API. This Repo houses an improvement fork of the original GetOldTweets Library by Jefferson Herique. The improvement makes running this package on Windows OS seamless with Python 3.x.

Details

This tool was built out of the need to eliminate the barriers that the Twitter API imposes so as to help researchers, businesses and organizations get all the tweets data they need for various analysis in their different line of operations, with ease and from command line, meaning no prior extensive programming knowledge is required. BigTweet provides a means of getting old/backdated twitter data for analysis, bypassing the rate limits and restrictions from twitter API, giving you unlimited tweet textual data along with some other metadata such as; dates and time information, likes, retweets, replies, and tweet geolocation.

Operating system

OMGOT was tested on Ubuntu distribution of the Linux operating system and on Windows 7, 8, 10.

Command Line Arguments

This package was optimized to work efficiently and seamlessly on both Windows Command prompt (CMD), and on UNIX Terminal. Below are some command line arguments which is by no means exhaustive. run python cli.py --help in terminal to get the full argument options.

  • username (str): An optional specific username from a twitter account. Without "@".
  • since (str. "yyyy-mm-dd"): A lower bound date to restrict search.
  • until (str. "yyyy-mm-dd"): An upper bound date to restrist search.
  • search (str): A query text to be matched.
  • near(str): A reference location area from where tweets were generated
  • csv: Write as a .csv file
  • json: Write as a .json file
  • count: Display the number of tweets scraped at the end of the session
  • year: Filter a tweet before a specified year

Usage - Very Important to Understand.

  1. Clone or download the repo to your local machine.
  2. Unzip it.
  3. cd to the unzipped Optimized-Modified-GetOldTweets3-OMGOT-master folder
  4. cd again to the GetOldTweets3-0.0.10 folder inside unzipped Optimized-Modified-GetOldTweets3-OMGOT-master folder, and fire up command prompt or terminal right there.
  5. then run the codes in the examples below.

Please Feel free to customize, change the parameters/arguments as used in the examples below according to the needs of your project.

Use Cases

Use case 1 - Get all the tweets by a user:

python cli.py --username "irekponorVictor"

Use case 2 - Get all tweets tweeted from a user since 2015-12-20 20:30:15:

python cli.py --username "irekponorVictor" --since "2015-12-20 20:30:15

Use case 3 - Get all tweets tweeted from a user from January 2019 - December 2019 and save in a csv file:

python cli.py --username "irekponorvictor" --since "2019-01-01" --until "2019-12-31" -o user.csv --csv

Use case 4 - Get tweets from a radius of 1km around a place in Lagos, Nigeria and export them to a csv

python cli.py -g="6.465422, 3.406448, 1km" -o Lagos.csv --csv

Use case 5 - Get every tweet containing the word "governance" from every where

python cli.py -s governance

I've added my local enviroment dependencies in a requirements.txt file, so if any body has issues with the dependencies or environments, do a pip install -r requirements.txt.

If you have any question, or need extra help, you are welcome to connect with me on any of my social media below:

connect with me on linkedIn.

Twitter.

Read my latest write-ups on medium.

More Repositories

1

Data-Science-Nigeria-2018-Notebooks-and-practice

This repo houses my notebooks on Exploratory data analysis as well as feature engineering and Modelling of a Supermarket dataset to predict sales provided by the Data science Nigeria body for the intercampus machine learning competition july to september 2018
Jupyter Notebook
6
star
2

PyCon-Nigeria-2019

This repo houses the jupyter notebook presentation for my workshop on Twitter Data Mining, At PyCon NG, Abuja.
Jupyter Notebook
5
star
3

Creating-a-Bias-Free-Testset

this repo houses the code and dataset to my article on medium which is on creating a bias free testset using stratified sampling split, rather than the conventional random split and selection which is broadly known as the train_test_split class in sklearn. link to the article here https://medium.com/@IrekponorVictor/creating-an-unbiased-test-set-for-your-model-using-stratified-sampling-technique-672b778022d5?source=user_profile---------2------------------
Jupyter Notebook
5
star
4

Bsc-Dissertation

Contains the files and codes written for my Bsc dissertation titled A Framework to Determine Ward Level Population Figures Using SOTA Spatially Disaggregation Population Data Estimates, A case study of Lagos Island.
JavaScript
4
star
5

BigTweet

BigTweet: An Open-Source Python-Based Tool for Automatically Querying and Downloading Tweets with Minimal Programming Knowledge
Python
3
star
6

Covid-19-Tracker

JavaScript
2
star
7

DJANGO-TUTORIALS-WEBSITE

Website for tutorials
Python
2
star
8

Frontend-masters-react

JavaScript
2
star
9

Advanced-CSS

Advanced CSS for data scientists hahaha.. yeah Data Scientists!
CSS
2
star
10

Personal-Site

Just a simple personal website to refresh my knowledge on HTML, and CSS. I am not your regular web developer.
CSS
1
star
11

Micro-level-population-estimation

A novel framework to determine micro-level population figures using spatially disaggregated population estimates
Python
1
star
12

Geospatial-Series-DWG-to-Shapefile

1
star
13

Shomolu-EDA

This repo houses expert exploratory analysis carried out on a set of 181 questionnaires administered in Shomolu Local Government Area, Lagos State, Nigeria.
Jupyter Notebook
1
star
14

HK-Twitter-Data-Mining-

This repo houses the codes and methodology, i used to mine twitter data, using a web-scraping script originally written by i modified for this purpose by [Jefferson Herique](https://github.com/Jefferson-Henrique/GetOldTweets-python). The modified repo can be found [here](https://github.com/marquisvictor/Modified-GetOldTweets3).
Jupyter Notebook
1
star