Explore @internetarchive Open Source projects

Internet Archive (@internetarchive)

internetarchive

Stars
12,897
Global Org. Rank 1,785 (Top 0.6 %)
Registered almost 15 years ago
Most used languages

Python
40.6 %

JavaScript
25.8 %

TypeScript
12.5 %

Shell
3.9 %

Java
3.9 %

Go
2.3 %

HTML
2.3 %

PHP
2.3 %

Swift
1.6 %

Scala
1.6 %

Others 3.2 %
Location 🇺🇸 United States
Country Total Rank 769
Country Ranking

Python
191

Java
196

Scala
715

JavaScript
919

PHP
1,220

Kotlin
1,383

Shell
2,350

Go
3,102

HTML
4,281

TypeScript
4,468

Swift
7,674

CSS
8,459

openlibrary

One webpage for every book ever published!

heritrix3

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

bookreader

The Internet Archive BookReader

wayback-machine-webextension

A web browser extension for Chrome, Firefox, Edge, and Safari 14.

brozzler

brozzler - distributed browser-based web crawler

warcprox

WARC writing MITM HTTP/S proxy

openlibrary-client

Python Client Library for the Archive.org OpenLibrary API

warc

Python library for reading and writing warc files

dweb-mirror

Offline Internet Archive project

warctools

Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)

bookserver

Archive.org OPDS Bookserver - A standard for digital book distribution

internetarchivebot

fatcat

Perpetual Access To The Scholarly Record

archive-pdf-tools

Fast PDF generation and compression. Deals with millions of pages daily.

fatcat-scholar

search interface for scholarly works

iaux

Monorepo for Archive.org UX development and prototyping.

umbra

A queue-controlled browser automation tool for improving web crawl quality

openlibrary-bots

A repository of cleanup bots implementing the openlibrary-client

dweb-archive

hind

Hashistack-IN-Docker (single container with nomad + consul + caddy)

wayback-machine-firefox

Reduce annoying 404 pages by automatically checking for an archived copy in the Wayback Machine. Learn more about this Test Pilot experiment at https://testpilot.firefox.com/

cdx-summary

Summarize web archive capture index (CDX) files.

internet-archive-voice-apps

Voice Apps (Actions on Google, Alexa Skill) of Internet Archive. Just say: "Ok Google, Ask Internet Archive to Play Jazz" or "Alexa, Ask Internet Internet Archive to play Instrumental Music"

liveweb

Liveweb proxy of the Wayback Machine project

Zeno

State-of-the-art web crawler 🔱

epub

For code related to making ePub files

trough

Trough: Big data, small databases.

dweb-transport

Internet Archive Decentralized Web Common API

archive-hocr-tools

Efficient hOCR tooling

wayback-diff

React components to render differences between captures at the Wayback Machine

dweb-transports

sandcrawler

Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki

crawling-for-nomore404

snakebite-py3

Pure python HDFS client: python3.x version

ia-hadoop-tools

iiif

The official Internet Archive IIIF service

dweb-gateway

Decentralized web Gateway for Internet Archive

newsum

Daily TV News Summary using GPT

arklet

ARK minter, binder, resolver

xfetch

Cache stampede test harness. Code accompanies the presentation made at RedisConf 2017, 30 May to 1 June, 2017, in San Francisco.

openlibrary-librarians

Coordination between the OpenLibrary.org Librarian community

scrapy-warcio

Support for writing WARC files with Scrapy

arch

Web application for distributed compute analysis of Archive-It web archive collections.

iacopilot

Summarize and ask questions about items in the Internet Archive

iari

Import workflows for the Wikipedia Citations Database

doublethink

rethinkdb python library

s3_loader

Watch for local files to appear and move them into S3

wayback-machine-android

Sparkling

Internet Archive's Sparkling Data Processing Library

archive-commons

draintasker

a tool for continuously ingesting w/arc files into the archive

ias3

Internet Archive S3-like connector

cicd

build & test using github registry; deploy to nomad clusters

wayback-radial-tree

chocula

journal-level metadata munging. part of fatcat project

read_api_extras

Demo code for the Open Library Read API

web_collection_search

An API wrapper to the Elasticsearch index of web archival collections and a web UI to explore those indexes.

dweb-archivecontroller

epub-labs

ia

A JS interface to archive.org

iaux-typescript-wc-template

IAUX Typescript WebComponent Template

wikibase-patcher

Python library for interacting with the Wikibase REST API

offlinesolr

Tool to build solr index offline

ia-bin-tools

Internet Archive Command-line Utilities

dweb-objects

archive-ocr-tools

iare

An interactive IARI JSON viewer

wayback-machine-safari

collections-cleaners

trendmachine

A mathematical model to calculate a normalized score to quantify the temporal resilience of a web page as a time-series data based on the historical observations of the page in web archives.

acs4_py

Python interface to ACS4

esbuild_es5

minify JS/TS files using `esbuild` and `swc` down to ES5 (uses `deno`)

iaux-search-service

map-of-the-web

eventer

Eventer is a simple event dispatching library in Python

iaux-donation-form

The Internet Archive Donation Form

internetarchive.github.com

Internet Archive Open Source Blog

isodos

Go module to interact with Internet Archive's Isodos API

internet-archive-alexa-skill

mediawiki-extension-archive-leaf

A MediaWiki extension that supports importing of Archive.org palm leaf items

hashitalksdemo

openlibrary-api

API documentation for https://github.com/internetarchive/openlibrary

iaux-collection-browser

btget

Command line retrieval of torrents using transmission-daemon (via transmission-remote)

strainer

Heritrix frontier files manipulation tool.

httpd

Fast and easy-to-use web server, using the Deno native http server (hyper in rust). It serves static files & dirs, with arbitrary handling using an optional `handler` argument.

file_server_plus

`deno` static file webserver, clone of `file_server.ts`, PLUS an additional final "404 handler" to run arbitrary JS/TS

dyno

tarb_insights

A Streamlit application to visualize Wikipedia IABot statistics

rulesengine-client

Python client package for the playback rules engine

coderunr

deploy saved changes to website unique hostnames instantly -- can skip commits, pushes & full CI/CD

archiveorg-e2e-playwright

deferred

Redis promises & futures library for Predis / PHP

hello-js

an example of full CI/CD from GitHub to a nomad cluster

wiki-references-db

Data models and scripts to build a database of references (broadly defined) appearing on Wikipedia and other wikis

maisy

Project Gutenberg collection importation via IAS3 interface

kohacon2011-presentation

Presentation for KohaCon 2011

rulesengine

model and front-end for rules for managing wayback playback

deploy

GitHub Action to deploy to a nomad cluster

iaux-democracys-library

A web component that highlights Democracy's Library