• Stars
    star
    411
  • Rank 105,247 (Top 3 %)
  • Language
    Rust
  • License
    Mozilla Public Li...
  • Created about 4 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Tools for fighting abuse on Twitter

Overview

Build status Coverage status

This repository contains some low-tech tools designed to help you make Twitter a nicer place for yourself. Some of these tools might also be useful in other ways, like for example if some litigious person with a long history of making common cause with white supremacists and misogynists threatens to sue you for defamation.

See this related project for an example of the kind of use case cancel-culture is designed to support (an archive of around 35 million deleted tweets associated with Gamergate, LambdaConf, Stop the Steal, etc.), or this project focused on tracking ban evasion by far-right accounts, or this recent project by Salish Coast Anti-Fascist Action.

Testimonials

A Twitter user:

Still, he somehow has access to everything you’ve ever posted & deleted, and can seemingly immediately find your new alt/resurrect/punished accounts.

Terms of service compliance

This software is designed to promote use that is compliant with the Twitter API Developer Agreement and the Internet Archive's Terms of Use.

Text and metadata for Twitter statuses are retrieved from the Wayback Machine, not the Twitter API, which is primarily used here to list follower relationships and to allow users to import and export block lists.

In theory it's possible that there are ways you could violate the Developer Agreement with the help of this software (for example by using "information obtained from the Twitter API to target people with advertising outside of the Twitter platform"). Don't do that.

Examples

One of the things this project provides is a command-line tool that takes a Twitter screen name and outputs a list of all of the accounts you've blocked that that account follows (sorted here by follower count):

$ cargo build --release
    Finished release [optimized] target(s) in 0.06s

$ target/release/twcc blocked-follows sfscala
@jdegoes             11807
@rolandkuhn           9238
@propensive           8323
@etorreborre          7045
@ChiefScientist       5450
@dibblego             3587
@nuttycom             3307
@kubukoz              2808
@scalaworldconf       2495
...

I get a lot of hate-follows, and this tool makes it much easier for me to decide which new followers I need to block. It's like a version of Twitter's "Followed by… and 123 others you follow" that's actually useful.

I sometimes work in a certain programming language community where prominent community members have a tendency to say abusive or exclusionary things and then delete and deny everything when they're confronted, so the CLI also provides a way to search the Wayback Machine for deleted tweets by a specified user:

$ target/release/twcc deleted-tweets --limit 100 jdegoes
https://web.archive.org/web/20190922222236/https://twitter.com/jdegoes/status/1170420726400212997
https://web.archive.org/web/20190923221242/https://twitter.com/jdegoes/status/1170711737361940481
https://web.archive.org/web/20200526150339/https://twitter.com/jdegoes/status/1265251872048320513

In this case we've limited the search to the 100 tweets most recently archived by the Wayback Machine.

You can also use this command to generate a Markdown-formatted report instead of a simple list of links:

$ target/release/twcc deleted-tweets --report ChiefScientist

Which currently generates this document.

It can also print a list of everyone you currently block, follow, or are followed by, it can get the URL of a deleted tweet from the URL of a reply, and it can partition a list of tweet IDs by their deleted status.

twcc 0.1.0
Travis Brown <[email protected]>

USAGE:
    twcc [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -v, --verbose    Level of verbosity
    -V, --version    Prints version information

OPTIONS:
    -k, --key-file <key-file>    TOML file containing Twitter API keys [default: keys.toml]

SUBCOMMANDS:
    blocked-follows    For a given user, list everyone they follow who you block
    check-existence    Checks whether a list of status IDs (from stdin) still exist
    deleted-tweets     Lists Wayback Machine URLs for all deleted tweets by a user
    follower-report    For a given user, print a report about their followers
    help               Prints this message or the help of the given subcommand(s)
    import-blocks      Blocks a list of user IDs (from stdin)
    list-blocks        Print a list of all users you've blocked
    list-followers     Print a list of all users who follow you (or someone else)
    list-friends       Print a list of all users you (or someone else) follows
    list-tweets        Print a list of (up to approximately 3200) tweet IDs for a user
    lookup-reply       Get the URL of a tweet given the URL or status ID of a reply

Tweet screenshots

The twshoot command-line tool can take a screenshot of a tweet, given either a URL or status ID:

$ cargo build --release
    Finished release [optimized] target(s) in 0.29s

$ target/release/twshoot https://twitter.com/travisbrown/status/1291256191641952256

And then you have a 1291256191641952256.png file in the current directory that looks like this:

Liking Scala is not a personality but it does mean you're racist / Do I think this is 100% accurate or fair: no … Do I think the Scala community is capable of coming to terms with the behavior that got it this reputation: also no

The application also generates a -full.png image showing the entire browser screen. The image sizes, output directory, etc. are configurable (see twshoot --help for details).

This tool doesn't require a Twitter API account, but you do have to have ChromeDriver running (it also works with GeckoDriver, but the results don't look as nice).

Setup

You'll need to install Rust and Cargo.

Once you've got those, you can run cargo build --release and the binaries will be available in the target/release directory.

For the main twcc application, you'll need Twitter API access for your Twitter account, and you'll need to provide the necessary keys in a file (by default keys.toml):

[twitter]
consumerKey = "****"
consumerSecret = "****"
accessToken = "****"
accessTokenSecret = "****"

Some of the other tools require a WebDriver server instead of API access. These should work with either ChromeDriver or GeckoDriver.

Other

The project also contains some other miscellaneous stuff, including a way to export your Twitter block list even if you don't have a Twitter API account, and a way to search the Wayback Machine even if the CDX server isn't working.

It doesn't currently include a few related tools I use regularly, including a way to block everyone who retweeted or favorited a given tweet, a bunch of stuff related to Wayback Machine ingestion and downloading, and some scripts that bundle some of the follower functionality into daily reports.

Most of these things are excluded for one of the following reasons:

  • They're even more fragile than the stuff that's here now.
  • I haven't ported them from Scala to Rust yet.

I might add some of them eventually.

Nothing here is very polished or robust. These applications don't keep track of rate limits in all cases, for example, so if you run out of requests for an endpoint, they may just crash, and you'll have to wait. I might try to smooth out some of these rough edges at some point, but it's unlikely.

License

This project is licensed under the Mozilla Public License, version 2.0. See the LICENSE file for details.

More Repositories

1

memory.lol

memory.lol
Rust
471
star
2

iteratee

Iteratees for Cats
Scala
184
star
3

dhallj

Dhall for Java
Java
174
star
4

blue

Twitter Blue data
122
star
5

octocrabby

Tools for managing GitHub block lists
Rust
98
star
6

type-provider-examples

Macro-based type providers for Scala (examples)
Scala
85
star
7

twitter-watch

Tracking the far right on Twitter
Rust
58
star
8

abstracted

Forget your methods
Scala
41
star
9

expressier

A regular expression type provider demo
Scala
40
star
10

sized

Scala
35
star
11

hassreden-tracker

Hassreden-Tracker
Rust
34
star
12

unsuspensions

Elon Musk's suspension reversals
32
star
13

metaplasm

meta.plasm.us
HTML
32
star
14

scala-quickstart

Getting started in the Scala REPL
Shell
28
star
15

deleted-tweets

Deleted tweet archive metadata and code
26
star
16

typelevel-tour

A tour of some Typelevel libraries
Scala
25
star
17

itm

Interactive topic modeling
Java
24
star
18

stop-the-steal

Stop the Steal / J6 Twitter user profiles
20
star
19

evasion

Tracking far-right ban evasion on Twitter
19
star
20

circe-derivation

Scala
17
star
21

mstparser

Scala
17
star
22

circe-algebra

Experimental decoding algebra for circe
Scala
13
star
23

wayback-rs

Tools for working with the Wayback Machine in Rust
Rust
13
star
24

orcrs

An ORC reader for Rust
Rust
12
star
25

scala-java-interop

Scala-Java interoperability examples
Java
12
star
26

dotty-experiments

Scala
11
star
27

incompletes

Derivation for incomplete type class instances
Scala
9
star
28

instancez

Scalaz 7 type class instances for various Scala and Java libraries.
Scala
9
star
29

names

Named-entity recognition with Finagle
Scala
8
star
30

coverages

A sandbox for experimenting with Scala code coverage tools
Scala
7
star
31

rotation-rs

Document rotation detection
Rust
7
star
32

tsg-metadata

Metadata related to the Twitter Stream Grab from Archive Team
Rust
7
star
33

macavity

Faster stuff for Cats
Scala
7
star
34

memory-lol

memory.lol
Java
7
star
35

rust-jvm-demo

Rust on the JVM
Scala
6
star
36

concurrent

Scala
6
star
37

syzygist

Various utilities for Scalaz streams
Scala
6
star
38

scrooge-circe-demo

Circe codecs for Scrooge-generated code
Scala
6
star
39

at-twitter-stream

Tools for working with Twitter JSON data
Rust
5
star
40

haskell-cpython

CPython bindings for Haskell
Haskell
5
star
41

woodchipper

Woodchipper
JavaScript
5
star
42

deleted-tweets-archive

Deleted Tweets Archive - These tweets display several bad actors' most divisive uses of the Twitter platform.
5
star
43

iteratee-twitter

iteratee.io module for Twitter Util
Scala
5
star
44

twitter-model

JSON Schema definitions for the Twitter API
Rust
4
star
45

hkvdb

hkvdb
Rust
4
star
46

sbt-javacc

An sbt plugin for JavaCC
Scala
4
star
47

parquetry

Rust
4
star
48

sbt-opinions

Standard SBT project plugin
Scala
4
star
49

twprs

Twitter profile tools for Rust
Rust
4
star
50

mallet

MALLET
Java
3
star
51

egg-mode-extras

Rate-limit-aware streams and other helpers for egg-mode
Rust
3
star
52

twpis

Rust
2
star
53

sentlex

Haskell
2
star
54

sbt-jacc

An sbt plugin for JFlex and Jacc
Java
2
star
55

xmlunit

XmlUnit
Java
2
star
56

tesseract

Tesseract
C++
2
star
57

shapeless-twitter

Twitter + Shapeless experiments
Scala
2
star
58

findmyfrens

Tracking findmyfrens.net
Rust
2
star
59

cli-helpers

Rust
1
star
60

misccli

Miscellaneous command-line tools
Rust
1
star
61

relate

A prosopography visualization tool
Java
1
star
62

scala-visitor

Scala
1
star
63

smoothlife

A simple version of SmoothLife in Haskell
Haskell
1
star
64

morphadorner

Java
1
star
65

dotfiles

My configuration
Vim Script
1
star
66

euler

Project Euler framework and solutions
Scala
1
star
67

abbot

TEI conversion utilities
Shell
1
star
68

disinfo-notes

1
star
69

rocksdb-table

Some helpers for working with RocksDB databases in Rust
Rust
1
star
70

rearranger

Rust
1
star
71

gothic

Analysis of a collection of Gothic texts
1
star
72

scardf

Scala RDF API
Scala
1
star
73

corporacamp-site

PHP
1
star
74

travisbrown

1
star
75

oscon-challenge

Twitter Challenge at OSCON 2014
JavaScript
1
star
76

circles

Miscellaneous files for Romantic Circles
1
star
77

archivindex-builder

Archivindex Builder
HTML
1
star
78

gistlist

List your gists
Rust
1
star
79

retryable-error

Rust
1
star
80

json-schema-tools

JSON Schema tools
Rust
1
star