Explore @ArchiveTeam Open Source projects

Archive Team (@ArchiveTeam)

ArchiveTeam

Stars
3,519
Global Org. Rank 6,153 (Top 2 %)
Registered over 13 years ago
Most used languages

Lua
43.7 %

Python
34.7 %

Shell
11.1 %

Ruby
2.6 %

Dockerfile
1.6 %

HTML
1.6 %

JavaScript
1.1 %

C
1.1 %

Others 2.5 %
Location 🇺🇸 United States
Country Total Rank 2,949
Country Ranking

NewLisp 10

Lua
67

DIGITAL Command Language 72

Dockerfile
76

Roff 179

HTML
427

Python
756

Shell
770

C
2,069

Haskell
2,912

Ruby
8,492

grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

wpull

Wget-compatible web downloader and crawler.

ArchiveBot

ArchiveBot, an IRC bot for archiving websites

warrior-dockerfile

A Dockerfile for the ArchiveTeam Warrior

parler-grab

Archiving Parler.

Ubuntu-Warrior

Scripts to build and boot warrior virtual machine containing Docker

wget-lua

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

IA.BAK

We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.

seesaw-kit

Making a reusable toolkit for writing seesaw scripts

terroroftinytown

URLTeam's second generation of URL shortener archiving tools

reddit-grab

Grabbing everything from reddit.

NewsGrabber

Grabbing all news.

yahooanswers-grab

Saving all questions and answers from Yahoo! Answers.

tumblr-grab

Archiving all to-be-deleted NSFW tumblr blogs.

imgur-grab

Archiving imgur.

universal-tracker

A configurable, reusable tracker with dashboard

googleplus-grab

Archiving Google+.

terroroftinytown-client-grab

The Seesaw pipeline grab script for the URLTeam (terroroftinytown) project

ludios_wpull

wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved

warrior-code2

Boot scripts for the ArchiveTeam Warrior 2

ftp-gov-grab

Archiving government FTPs.

warrior-code

WebArchiver

Decentralized web archiving

soundcloud-grab

500px-grab

Archiving https://500px.com/creativecommons

tinyback

A tiny web scraper

gamemaker-sandbox-items

Gamemaker Sandbox Tracker items

youtube-grab

Archiving all metadata from YouTube (everything except videos themselves due to size)

youtube-dislikes-grab

Archiving general youtube video metadata through innertube for dislikes removal.

youtube-dislikes-items

Managing items for youtube-dislikes-grab.

VideoBot

Specialised bot for periodical grabs and video/audio/etc. webpage scrapes.

urlteam-stuff

Urlteam website, code, ... also, PONIES

urls-grab

Archiving URLs (outlinks) from a variety of sources.

NewsGrabber-Warrior

google-sites-grab

Archiving Google Sites Classic.

flickr-grab

Grabbing Flickr images.

pastebin-grab

Archiving pastebin

youtube-items

Managing items for youtube-grab

wget-lua-forum-scripts

Downloading forums posts with Wget+Lua

greader-grab

http://www.archiveteam.org/index.php?title=Google_Reader

ftp-grab

Save all FTP sites!

mediafire-items

Managing items for mediafire-grab.

citeseerxpdf-grab

Grabbing all sources of CiteSeerX.

twitchtv-grab

Grabbing twitch.tv videos

mobileme-grab

Downloading MobileMe

warrior-preseed

Constructing a new warrior VM

ftp-nab

Thinger to download FTP sites

coursera-grab

Saving courses from Coursera.

tinyarchive

Software behind tracker.tinyarchive.org - Warning: Very hacky code

formspring-grab

Downloading Formspring

yahoomessages-grab

Archiving Yahoo Messages

splinder-grab

telegram-grab

Archiving public telegram messages.

ffnet-grab

archiveteam-megawarc-factory

Some scripts to process ArchiveTeam uploads

roblox-grab

Archiving roblox forums.

gamemaker-sandbox-grab

Grabbing sandbox.yoyogames.com

justintv-grab

Grabbing as much of justin.tv's archives as possible

sourceforge-grab

Archiving SourceForge.

grab-base-df

Base Dockerfile for warrior project grab scripts

wikis-grab

Grabbing all wikis.

liveleak-grab

Archiving liveleak.com

tencent-weibo-grab

Archiving Tencent Weibo (t.qq.com), 腾讯微博

imdb-grab

Archiving IMDb.

reddit-items

Managing items for reddit-grab.

flashdomains-grab

Copy of domains-grab for Flash sites.

ftp-queue

Create queue items for ftp-grab.

tumblr-grab-test

Archiving Tumblr blogs (an ArchiveTeam Warrior testing project)

heroku-buildpack-archiveteam-warrior

Heroku buildpack with the Archive Team Warrior

mobileme-index

An index of the MobileMe downloads

twitchtv-items

Managing twitch.tv items.

Universal-tracker-2

A better tracker with more features for ArchiveTeam

eroshare-grab

panoramio-grab

Grabbing everything from panoramio

blingee-grab

Saving all images and content from Blingee.

vidme-grab

Archiving all videos from vid.me.

livejournal-discovery

Discovering items for livejournal-grab.

github-grab

Archiving GitHub

mediafire-grab

Archiving mediafire.com URLs.

furaffinity-grab

Grabbing all images and other stuff from Fur Affinity.

yahoogroups-grab

Archiving Yahoo! Groups.

webs-grab

Archiving webs.com

puush-grab

twitchtv-discovery-grab

Discovering twitch.tv content

vlive-grab

Archiving vlive.tv.

NewsGrabber-Services

The services for NewsGrabber.

parler-items

Managing items for parler-grab.

furaffinity-items

standalone-readme-template

Readme instructions template for manually running pipeline grab scripts outside the warrior

ArchiveBot-agents

Site-specific agents that work with ArchiveBot

ua-grab

Archiving all of .ua.

googlecode-grab

Saving the full Google Code site!

pixiv-2-grab

Archiving pixiv2 images

miiverse-grab

Archiving miiverse

orkut-grab

Download all of Orkut

dpreview-grab

Archiving DPReview

bottle

A statistics monitor for the listerine download project @ Archive Team. Massive hack, no tests.

halo-new-grab

Archiving Halo (round 2)

googleplus-items

Managing items for googleplus-grab and googleplus2-grab.

furaffinity-discovery