• Stars
    star
    420
  • Rank 103,166 (Top 3 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created almost 8 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Turn Tweet IDs into Twitter JSON & CSV from your desktop!

Hydrator

Twitter's changes to their API which greatly reduce the amount of read-only access means that the Hydrator is no longer a useful application. The application keys, which functioned for the last 7 years, have been rescinded by Twitter.


Build & Release

Hydrator is an Electron based desktop application for hydrating Twitter ID datasets. Twitter's Terms of Service do not allow the full JSON for datasets of tweets to be distributed to third parties. However they do allow datasets of tweet IDs to be shared. Hydrator helps you turn these tweet IDs back into JSON and also CSV from the comfort of your desktop.

If you are interested in learning more please join the DocNow community in Slack, or add an issue ticket here. If you would like to explore tweet identifier datasets please see the DocNow Catalog and GWU's TweetSets.

Install

Important!

It is easiest to download a pre-built version of the Hydrator instead of building it from source. Please see the list of available releases for OS X, Windows and Linux installers.

Note for OS X Users

Since the Hydrator has not been signed (which requires us to pay Apple in order to register as a developer) your initial start up of the Hydrator will be prevented. You can convince OS X to open it anyway by locating the Hydrator app in your Applications folder, control-clicking on it, selecting and then clicking Open (see the screenshot below). From this point on your Hydrator should start normally.

Develop

Get it:

git clone https://github.com/docnow/hydrator
cd hydrator

Configure:

In order to build the Hydrator you will need to get app keys from Twitter and put them in a .env file in your project directory. It should look something like this:

TWITTER_CONSUMER_KEY=CHANGEME
TWITTER_CONSUMER_SECRET=CHANGEMETOO

Next install the dependencies:

yarn install

Start a hot-swappable development server:

yarn run develop

Alternatively, create installers for OS X, Windows and Linux:

yarn run pack:mac
yarn run pack:win
yarn run pack:linux

Hydrator was created using electron-react-redux-boilerplate so check out that documentation for more information about commands that are available.

How to Cite

If you would like to cite this software please use something like the following:

Documenting the Now. (2020). Hydrator [Computer Software]. Retrieved from https://github.com/docnow/hydrator

More Repositories

1

twarc

A command line tool (and Python library) for archiving Twitter JSON
Python
1,320
star
2

diffengine

track changes to the news, where news is anything with an RSS feed
Python
172
star
3

docnow

A Twitter data collection and appraisal application.
JavaScript
48
star
4

twarc-csv

A plugin for twarc2 for converting tweet JSON into DataFrames and exporting to CSV.
Python
31
star
5

unshrtn

A LevelDB backed URL unshortening microservice written in JavaScript
JavaScript
31
star
6

catalog

A simple catalog of Twitter ID Datasets
JavaScript
28
star
7

twarc-network

Generate network visualizations from Twitter data.
Python
18
star
8

awesome-social-media-archiving

Tools for helping you work with web platform archive downloads.
17
star
9

tweet-archive

A tool for working with tweet archives.
JavaScript
15
star
10

dnflow

A design prototype for DocNow to learn with
Python
14
star
11

code-of-conduct

Code of Conduct for the Documenting the Now Community
13
star
12

waybackprov

utility to fetch provenance information from Internet Archive's Wayback Machine
Python
13
star
13

tweet-viewer

Generates a single "infinite scroll" page to show tweets collected with the DocNow App
JavaScript
7
star
14

storified

archive Storify stories
Python
6
star
15

foaf

a microservice for generating friend-of-a-friend networks for Twitter
Python
5
star
16

twarc-timeline-archive

Download timelines for a set of users
Python
4
star
17

notebooks

sketches, ideas and experiments
Jupyter Notebook
4
star
18

twitter-archive-unshorten

Convert the t.co URLs in your Twitter archive back to their full form
JavaScript
4
star
19

social-humans

social media labels for more ethical archives
HTML
3
star
20

twarc-edits

Add a twarc2 command to get edited tweets
Python
3
star
21

docnow-ansible

Ansible Installer for Docnow App
Jinja
3
star
22

twarc-hashtags

Report on hashtags in tweet data.
Python
3
star
23

docnow.github.io

docnow.io website
HTML
3
star
24

twarc-ids

A plugin for twarc2 to extract tweet ids from tweet JSON.
Python
2
star
25

twarc-text

A twarc plugin to print tweets to the console
Python
2
star
26

flatten-tweet

Make Twitter v2 API response data easier to work with
JavaScript
2
star
27

roadmap

A roadmap of development on the Documenting the Now project.
1
star
28

twarcd

a microservice for twarc
Python
1
star
29

sfm-ansible

Ansible playbooks for setting up Social Feed Manager
1
star
30

tweetgresql

A little playground for testing modeling tweets in Node & PostgreSQL
JavaScript
1
star
31

twarc-statistics

A plugin for getting counts and basic statistics from a dataset
1
star
32

twarc-videos

Python
1
star
33

dnflow-ansible

Ansible playbooks for setting up dnflow
Shell
1
star