• Stars
    star
    377
  • Rank 113,535 (Top 3 %)
  • Language
    Python
  • License
    GNU Affero Genera...
  • Created over 9 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Python Client Library for the Archive.org OpenLibrary API

openlibrary-client

pre-commit test_python

A reference client library for the Open Library API. Tested with Python 3.7, 3.8, 3.9, and 3.10.

Installation

To install the openlibrary-client package:

$ pipx install git+https://github.com/internetarchive/openlibrary-client.git

-- or --

$ git clone https://github.com/internetarchive/openlibrary-client.git
$ cd openlibrary-client
$ pip install .
-- or --
$ pipx install git+https://github.com/internetarchive/openlibrary-client.git

Configuration

Many Open Library actions (like creating Works and Editions) require authentication, i.e. certain requests must be provided a valid cookie of a user which has been logged in with their openlibrary account credentials. The openlibrary-client can be configured to "remember you" so you don't have to provide credentials with each request.

First time users may run the following command to enable the "remember me" feature. This process will ask for an Archive.org email and password, will authenticate the credentials, and then store the account's corresponding s3 keys in ~/.config/ol.ini (or whichever config location the user has specified):

$ ol --configure --email [email protected]
password: ***********
Successfully configured

Using Keys Directly

The ol.ini has two variables, access and secret. If you have both of them, you can manually initialise them

from olclient import OpenLibrary, config
ol = OpenLibrary(credentials=config.Credentials(access='<access>', secret='<secret>'))

This way, access and secret can be pulled from environment variables at runtime!

Usage

Python Library

For more examples, you can take a look at our examples directory on Python scripts for specific use cases that are needed.

Google Colab

You can view interactive documentation of openlibrary-client at this Google Colab document.

Adding a new Book

Fun things you can do to add a new book to Open Library

>>> from olclient.openlibrary import OpenLibrary
>>> import olclient.common as common
>>> ol = OpenLibrary()
>>> book = common.Book(title=u"Warlight: A novel", authors=[common.Author(name=u"Michael Ondaatje")], publisher=u"Deckle Edge", publish_date=u"2018")
>>> book.add_id(u'isbn_10', u'0525521194')
>>> book.add_id(u'isbn_13', u'978-0525521198')
>>> new_book = ol.create_book(book)
>>> new_book.add_bookcover('https://images-na.ssl-images-amazon.com/images/I/51kmM%2BvVRJL._SX337_BO1,204,203,200_.jpg')

Works

Fun things you can do with an Work:

>>> from olclient.openlibrary import OpenLibrary
>>> ol = OpenLibrary()
>>> work = ol.Work.get(u'OL12938932W')
>>> editions = work.editions

One thing to consider in the snippet above is that work.editions is a @property which makes several http requests to OpenLibrary in order to populate results. Once a call has been made to work.editions, its editions are saved/cached as work.editions.

Editions

Fun things you can do with an Edition:

>>> from olclient.openlibrary import OpenLibrary
>>> ol = OpenLibrary()
>>> edition = ol.Edition.get(u'OL25952968M')
>>> authors = edition.authors
>>> work = edition.work
>>> work.add_bookcover(u'https://covers.openlibrary.org/b/id/7451891-L.jpg')
>>> edition.add_bookcover(u'https://covers.openlibrary.org/b/id/7451891-L.jpg')

Authors

Author Information for existing authors can be done in the following manner.

>>> from olclient.openlibrary import OpenLibrary
>>> ol = OpenLibrary()
>>> author_olid = ol.Author.get_olid_by_name('Dan Brown')
>>> author_obj = ol.get(author_olid)

Command Line Tool

Installing the openlibrary-client library will also install the ol command line utility.

$ ol

usage: ol [-h] [-v] [--configure] [--get-work] [--get-author-works]
          [--get-book] [--get-olid] [--olid OLID] [--isbn ISBN]
          [--create CREATE] [--title TITLE] [--author-name AUTHOR_NAME]
          [--baseurl BASEURL] [--email EMAIL]

olclient

optional arguments:
  -h, --help            show this help message and exit
  -v                    Displays the currently installed version of ol
  --configure           Configure ol client with credentials
  --get-work            Get a work by --title, --olid
  --get-author-works    Get a works of an author providing author's --olid,
                        --author-name
  --get-book            Get a book by --isbn, --olid
  --get-olid            Get an olid by --title or --isbn
  --olid OLID           Specify an olid as an argument
  --isbn ISBN           Specify an isbn as an argument
  --create CREATE       Create a new work from json
  --title TITLE         Specify a title as an argument
  --author-name AUTHOR_NAME
                        Specify an author as an argument
  --baseurl BASEURL     Which OL backend to use
  --email EMAIL         An IA email for requests which require authentication.
                        You will be prompted discretely for a password

You can create a new work from the command line using the following syntax. It's almost identical to the olclient.common.Book object construction, except instead of providing an Author object, you instead pass a key for "author" and a corresponding value:

> ol --create '{"title": "The Cartoon Guide to Calculus", "publisher": "Teach Yourself", "publish_date": "2013", "identifiers": {"isbn_10": ["144419111X"]}, "cover": "https://images-na.ssl-images-amazon.com/images/I/51aJdEGttLL._SX328_BO1,204,203,200_.jpg", "author": "Hugh Neill"}'
OL26194598M

Successful creation of a new Work results in the return of its Open Library edition ID.

Testing

To run test cases (from the openlibrary-client directory):

$ pytest

Other Client Libraries

Other Open Library client libraries include:

More Repositories

1

openlibrary

One webpage for every book ever published!
Python
5,180
star
2

heritrix3

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
Java
2,821
star
3

bookreader

The Internet Archive BookReader
JavaScript
975
star
4

wayback-machine-webextension

A web browser extension for Chrome, Firefox, Edge, and Safari 14.
JavaScript
657
star
5

brozzler

brozzler - distributed browser-based web crawler
Python
657
star
6

warcprox

WARC writing MITM HTTP/S proxy
Python
377
star
7

warc

Python library for reading and writing warc files
Python
237
star
8

dweb-mirror

Offline Internet Archive project
JavaScript
232
star
9

warctools

Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
Python
149
star
10

internetarchivebot

PHP
120
star
11

bookserver

Archive.org OPDS Bookserver - A standard for digital book distribution
Python
119
star
12

fatcat

Perpetual Access To The Scholarly Record
Python
114
star
13

archive-pdf-tools

Fast PDF generation and compression. Deals with millions of pages daily.
Python
97
star
14

fatcat-scholar

search interface for scholarly works
Python
78
star
15

Zeno

State-of-the-art web crawler πŸ”±
HTML
70
star
16

iaux

Monorepo for Archive.org UX development and prototyping.
JavaScript
63
star
17

openlibrary-bots

A repository of cleanup bots implementing the openlibrary-client
Python
62
star
18

umbra

A queue-controlled browser automation tool for improving web crawl quality
Python
60
star
19

dweb-archive

JavaScript
54
star
20

hind

Hashistack-IN-Docker (single container with nomad + consul + caddy)
Shell
53
star
21

wayback-machine-firefox

Reduce annoying 404 pages by automatically checking for an archived copy in the Wayback Machine. Learn more about this Test Pilot experiment at https://testpilot.firefox.com/
JavaScript
53
star
22

cdx-summary

Summarize web archive capture index (CDX) files.
Python
52
star
23

internet-archive-voice-apps

Voice Apps (Actions on Google, Alexa Skill) of Internet Archive. Just say: "Ok Google, Ask Internet Archive to Play Jazz" or "Alexa, Ask Internet Internet Archive to play Instrumental Music"
JavaScript
46
star
24

liveweb

Liveweb proxy of the Wayback Machine project
Python
44
star
25

epub

For code related to making ePub files
Python
40
star
26

surt

Sort-friendly URI Reordering Transform (SURT) python module
Python
40
star
27

archive-hocr-tools

Efficient hOCR tooling
Python
39
star
28

trough

Trough: Big data, small databases.
Python
36
star
29

dweb-transport

Internet Archive Decentralized Web Common API
36
star
30

wayback-diff

React components to render differences between captures at the Wayback Machine
JavaScript
31
star
31

dweb-transports

JavaScript
25
star
32

sandcrawler

Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki
HTML
24
star
33

iiif

The official Internet Archive IIIF service
JavaScript
22
star
34

crawling-for-nomore404

Python
22
star
35

snakebite-py3

Pure python HDFS client: python3.x version
Python
22
star
36

newsum

Daily TV News Summary using GPT
Python
21
star
37

ia-hadoop-tools

Java
21
star
38

arklet

ARK minter, binder, resolver
Python
21
star
39

dweb-gateway

Decentralized web Gateway for Internet Archive
Python
21
star
40

xfetch

Cache stampede test harness. Code accompanies the presentation made at RedisConf 2017, 30 May to 1 June, 2017, in San Francisco.
PHP
18
star
41

openlibrary-librarians

Coordination between the OpenLibrary.org Librarian community
18
star
42

arch

Web application for distributed compute analysis of Archive-It web archive collections.
Scala
15
star
43

cicd

build & test using github registry; deploy to nomad clusters
13
star
44

scrapy-warcio

Support for writing WARC files with Scrapy
Python
13
star
45

iacopilot

Summarize and ask questions about items in the Internet Archive
Python
13
star
46

iari

Import workflows for the Wikipedia Citations Database
Python
12
star
47

doublethink

rethinkdb python library
Python
11
star
48

s3_loader

Watch for local files to appear and move them into S3
Python
11
star
49

Sparkling

Internet Archive's Sparkling Data Processing Library
Scala
11
star
50

wayback-machine-android

Kotlin
10
star
51

archive-commons

Java
10
star
52

draintasker

a tool for continuously ingesting w/arc files into the archive
Python
9
star
53

ias3

Internet Archive S3-like connector
Python
8
star
54

wayback-radial-tree

JavaScript
7
star
55

chocula

journal-level metadata munging. part of fatcat project
Python
7
star
56

read_api_extras

Demo code for the Open Library Read API
7
star
57

wikibase-patcher

Python library for interacting with the Wikibase REST API
Python
7
star
58

dweb-archivecontroller

JavaScript
7
star
59

web_collection_search

An API wrapper to the Elasticsearch index of web archival collections and a web UI to explore those indexes.
Python
7
star
60

epub-labs

epub-labs
6
star
61

iaux-typescript-wc-template

IAUX Typescript WebComponent Template
TypeScript
6
star
62

ia

A JS interface to archive.org
JavaScript
6
star
63

archive-ocr-tools

Python
6
star
64

offlinesolr

Tool to build solr index offline
Java
6
star
65

ia-bin-tools

Internet Archive Command-line Utilities
C
6
star
66

dweb-objects

JavaScript
5
star
67

iare

An interactive IARI JSON viewer
JavaScript
5
star
68

iaux-collection-browser

TypeScript
5
star
69

wayback-machine-safari

JavaScript
5
star
70

collections-cleaners

Shell
5
star
71

trendmachine

A mathematical model to calculate a normalized score to quantify the temporal resilience of a web page as a time-series data based on the historical observations of the page in web archives.
Python
5
star
72

acs4_py

Python interface to ACS4
Python
4
star
73

esbuild_es5

minify JS/TS files using `esbuild` and `swc` down to ES5 (uses `deno`)
TypeScript
4
star
74

iaux-search-service

TypeScript
4
star
75

map-of-the-web

Python
4
star
76

eventer

Eventer is a simple event dispatching library in Python
Python
4
star
77

iaux-donation-form

The Internet Archive Donation Form
TypeScript
4
star
78

internetarchive.github.com

Internet Archive Open Source Blog
CSS
4
star
79

isodos

Go module to interact with Internet Archive's Isodos API
Go
4
star
80

strainer

Heritrix frontier files manipulation tool.
Go
4
star
81

internet-archive-alexa-skill

JavaScript
3
star
82

btget

Command line retrieval of torrents using transmission-daemon (via transmission-remote)
Python
3
star
83

mediawiki-extension-archive-leaf

A MediaWiki extension that supports importing of Archive.org palm leaf items
JavaScript
3
star
84

hashitalksdemo

JavaScript
3
star
85

openlibrary-api

API documentation for https://github.com/internetarchive/openlibrary
HTML
3
star
86

httpd

Fast and easy-to-use web server, using the Deno native http server (hyper in rust). It serves static files & dirs, with arbitrary handling using an optional `handler` argument.
JavaScript
3
star
87

wbm_ai_kg

Google Summer of Code (GSoC) 2024 Wayback Machine GenAI Knowledge Graph project
HTML
3
star
88

file_server_plus

`deno` static file webserver, clone of `file_server.ts`, PLUS an additional final "404 handler" to run arbitrary JS/TS
TypeScript
2
star
89

dyno

JavaScript
2
star
90

archiveorg-e2e-playwright

TypeScript
2
star
91

tarb_insights

A Streamlit application to visualize Wikipedia IABot statistics
Python
2
star
92

rulesengine-client

Python client package for the playback rules engine
Python
2
star
93

coderunr

deploy saved changes to website unique hostnames instantly -- can skip commits, pushes & full CI/CD
Shell
2
star
94

deferred

Redis promises & futures library for Predis / PHP
PHP
2
star
95

hello-js

an example of full CI/CD from GitHub to a nomad cluster
JavaScript
2
star
96

wiki-references-db

Data models and scripts to build a database of references (broadly defined) appearing on Wikipedia and other wikis
Python
2
star
97

maisy

Project Gutenberg collection importation via IAS3 interface
Python
2
star
98

kohacon2011-presentation

Presentation for KohaCon 2011
Shell
2
star
99

rulesengine

model and front-end for rules for managing wayback playback
Python
2
star
100

deploy

GitHub Action to deploy to a nomad cluster
2
star