• Stars
    star
    945
  • Rank 47,212 (Top 1.0 %)
  • Language
    JavaScript
  • License
    GNU Affero Genera...
  • Created over 15 years ago
  • Updated 26 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The Internet Archive BookReader

Internet Archive BookReader

Build Status codecov

Disclaimer: BookReader v5 is currently in beta. It's stable enough for production use (and is what is being used on archive.org), but there will be some breaking changes in the next ~month or so to public BookReader APIs.

Internet Archive BookReader full logo

BookReader v5 interface screenshot

The Internet Archive BookReader is used to view books from the Internet Archive online and can also be used to view other books.

See live examples:

Demos

See BookReaderDemo directory. These can be tested by building the source files (make sure Node.js is installed):

npm run build

and starting a simple web server in the root directory:

npm run serve

And then open http://localhost:8000/BookReaderDemo/demo-simple.html.

Usage

Here is a short example.

// Create the BookReader object
var options = {
  data: [
    [
      { width: 800, height: 1200,
        uri: '//archive.org/download/BookReader/img/page001.jpg' },
    ],
    [
      { width: 800, height: 1200,
        uri: '//archive.org/download/BookReader/img/page002.jpg' },
      { width: 800, height: 1200,
        uri: '//archive.org/download/BookReader/img/page003.jpg' },
    ],
    [
      { width: 800, height: 1200,
        uri: '//archive.org/download/BookReader/img/page004.jpg' },
      { width: 800, height: 1200,
        uri: '//archive.org/download/BookReader/img/page005.jpg' },
    ]
  ],

  bookTitle: 'Simple BookReader Presentation',

  // thumbnail is optional, but it is used in the info dialog
  thumbnail: '//archive.org/download/BookReader/img/page014.jpg',

  // Metadata is optional, but it is used in the info dialog
  metadata: [
    {label: 'Title', value: 'Open Library BookReader Presentation'},
    {label: 'Author', value: 'Internet Archive'},
    {label: 'Demo Info', value: 'This demo shows how one could use BookReader with their own content.'},
  ],

  ui: 'full', // embed, full (responsive)

};
var br = new BookReader(options);

// Let's go!
br.init();

Architecture Overview

Starting at v5, BookReader introduces hybrid architecture that merges the core code written in jQuery closer to its evolution as a web component. As we march toward the future of BookReader as a web component, we are taking an Event Driven approach to connect the two together.

Approach:

  • Event driven
  • Control BR from the outside by using public methods
    • When BookNavigator reacts to BR's events, BookNavigator can directly control BR core using public functions.
      • As we continue to decouple the UI from drawing/calculating logic, these logical methods will become easier to spot, raise as a public method, and create unit tests for them.

Menu panels: Web Components via LitElement

BookReader's side navigation is powered by LitElement flavored web components.

Core: jQuery

BookReader's core functionality is in jQuery. This includes:

  • drawing & resizing the book and the various modes (1up, 2 page spread, gallery view)
  • the horizontal navigation
  • search API service
  • plugins

A peek in how to use/extend core functionality:

  • Properties
  • Plugins
    • A basic plugin system is used. See the examples in the plugins directory. The general idea is that they are mixins that augment the BookReader prototype. See the plugins directory for all the included plugins, but here are some examples:
      • plugin.autoplay.js - autoplay mode. Flips pages at set intervals.
      • plugin.chapters.js - render chapter markers
      • plugin.search.js - add search ui, and callbacks
      • plugin.tts.js - add tts (read aloud) ui, sound library, and callbacks
      • plugin.url.js - automatically updates the browser url
      • plugin.resume.js - uses cookies to remember the current page
      • plugin.vendor-fullscreen.js - replaces fullscreen mode with vendor native fullscreen
      • see plugin directory for current plugin files

Embedding BookReader in an iFrame

BookReader can be embedded within an <iframe>. If you use the IFrame Plugin inside the <iframe>, the reader will send notifications about changes in the state of the reader via window.postMessage(). The parent window can send messages of its own (also via window.postMessage()) and the IFrame Plugin will handle updating the reader to match.

Message Events

Fragment Change

The Fragment Change message is sent to the parent window when the embedded BookReader moves between pages/modes. When the <iframe> receives this message, it moves to the specified page/mode. The “fragment” is formatted in accordance with the BookReader URL spec.

{
  "type": "bookReaderFragmentChange",
  "fragment": "page/n1/mode/2up"
}

Development

(updates?)

The source JavaScript is written in ES6 (located in the src/js directory) and in ES5 (located in BookReader). npm run serve-dev starts an auto-reloading dev server, that builds js/css that has been edited at localhost:8000.

Until the next major version bump, we have to store the build files inside the repo to maintain backwards compatibility. Please DO NOT include these files in your PR. Anything in the BookReader/ directory should not be committed.

Developing icons

To see local icon package changes in bookreader, you'll need to install core-js into the icon package and link into bookreader.

Let's use icon-share as an example.

  1. Confirm your icon package is working properly in the iaux-icons demo
  2. Navigate to your icon package (iaux-icons/packages/icon-share) and run command: npm install core-js
    • You shouldn't need to commit any of these core-js changes
  3. From within your icon package directory run command: npm link
    • You can use the command npm ls -g to confirm your local package now appears in the registry
  4. Navigate to /bookreader and run command: npm link @internetarchive/icon-share
    • You can use the command npm ls |grep icon-share to confirm icon-share is now a link to your local directory
  5. You may now start a local server to see your changes by running command: npm run serve-dev

Releases

To version bump the repo and prepare a release, run npm version major|minor|patch (following semver), then (something like) git push origin HEAD --tags. It'll automatically update the version number where it appears, build the files, and ask you to update the CHANGELOG.

We release BookReader in-repo as tags & also as a node module @internetarchive/bookreader

Tests

We would like to get to 100% test coverage and are tracking our progress in this project: BookReader Fidelity

End to end tests

We also have end to end tests using Testcafe. We write tests for the repo itself and also for our use on archive.org. You can read about them in here. These are relatively easy to do, and a fantastic way of getting introduced to the wonders of BookReader. Check the project board for open tickets to work on. And if you don't see a test for something you spotted, feel free to make an issue.

To run all local end to end tests, run command: npm run test:e2e

To keep end to end test server on while developing, run command: npm run test:e2e:dev

Unit tests

We have unit tests and use Jest to run them. For mocks, we use Jest's internal mocking mechanism and Sinon to set spies.

To run all local unit tests, run command: npm run test

Ways to contribute

We can always use a hand building BookReader. Check out the issues and see what interests you. If you have an idea for an improvement, open an issue.

More info

Developer documentation: https://openlibrary.org/dev/docs/bookreader

Hosted source code: https://github.com/internetarchive/bookreader

IIIF (http://iiif.io) See BookReaderDemo/demo-iiif.html to see an example of how to load an IIIF manifest in BookReader.

Target Devices

Note that BookReader is a core part of Archive.org's mission of Universal Access to All Knowledge. Therefore, care must be taken to support legacy browsers. It should still work and be useable on old devices.

Areas for improvement

  • Change libraries to be NPM dependencies rather than included in the source code

See CHANGELOG.md for history of the project.

License

The source code license is AGPL v3, as described in the LICENSE file.

Other credits

The ability to test on multiple devices is provided courtesy of Browser Stack.

More Repositories

1

openlibrary

One webpage for every book ever published!
Python
4,933
star
2

heritrix3

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
Java
2,718
star
3

brozzler

brozzler - distributed browser-based web crawler
Python
635
star
4

wayback-machine-webextension

A web browser extension for Chrome, Firefox, Edge, and Safari 14.
JavaScript
613
star
5

warcprox

WARC writing MITM HTTP/S proxy
Python
366
star
6

openlibrary-client

Python Client Library for the Archive.org OpenLibrary API
Python
352
star
7

warc

Python library for reading and writing warc files
Python
233
star
8

dweb-mirror

Offline Internet Archive project
JavaScript
232
star
9

warctools

Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
Python
142
star
10

bookserver

Archive.org OPDS Bookserver - A standard for digital book distribution
Python
115
star
11

internetarchivebot

PHP
113
star
12

fatcat

Perpetual Access To The Scholarly Record
Python
109
star
13

archive-pdf-tools

Fast PDF generation and compression. Deals with millions of pages daily.
Python
79
star
14

fatcat-scholar

search interface for scholarly works
Python
76
star
15

iaux

Monorepo for Archive.org UX development and prototyping.
JavaScript
63
star
16

umbra

A queue-controlled browser automation tool for improving web crawl quality
Python
58
star
17

openlibrary-bots

A repository of cleanup bots implementing the openlibrary-client
Python
57
star
18

dweb-archive

JavaScript
54
star
19

hind

Hashistack-IN-Docker (single container with nomad + consul + caddy)
Shell
53
star
20

wayback-machine-firefox

Reduce annoying 404 pages by automatically checking for an archived copy in the Wayback Machine. Learn more about this Test Pilot experiment at https://testpilot.firefox.com/
JavaScript
52
star
21

cdx-summary

Summarize web archive capture index (CDX) files.
Python
47
star
22

Zeno

State-of-the-art web crawler 🔱
Go
45
star
23

internet-archive-voice-apps

Voice Apps (Actions on Google, Alexa Skill) of Internet Archive. Just say: "Ok Google, Ask Internet Archive to Play Jazz" or "Alexa, Ask Internet Internet Archive to play Instrumental Music"
JavaScript
45
star
24

liveweb

Liveweb proxy of the Wayback Machine project
Python
44
star
25

epub

For code related to making ePub files
Python
40
star
26

trough

Trough: Big data, small databases.
Python
36
star
27

dweb-transport

Internet Archive Decentralized Web Common API
36
star
28

archive-hocr-tools

Efficient hOCR tooling
Python
32
star
29

wayback-diff

React components to render differences between captures at the Wayback Machine
JavaScript
30
star
30

dweb-transports

JavaScript
25
star
31

sandcrawler

Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki
HTML
24
star
32

crawling-for-nomore404

Python
23
star
33

snakebite-py3

Pure python HDFS client: python3.x version
Python
22
star
34

ia-hadoop-tools

Java
21
star
35

iiif

The official Internet Archive IIIF service
JavaScript
21
star
36

dweb-gateway

Decentralized web Gateway for Internet Archive
Python
21
star
37

newsum

Daily TV News Summary using GPT
Python
20
star
38

arklet

ARK minter, binder, resolver
Python
18
star
39

xfetch

Cache stampede test harness. Code accompanies the presentation made at RedisConf 2017, 30 May to 1 June, 2017, in San Francisco.
PHP
18
star
40

openlibrary-librarians

Coordination between the OpenLibrary.org Librarian community
16
star
41

scrapy-warcio

Support for writing WARC files with Scrapy
Python
13
star
42

arch

Web application for distributed compute analysis of Archive-It web archive collections.
Scala
13
star
43

iacopilot

Summarize and ask questions about items in the Internet Archive
Python
13
star
44

iari

Import workflows for the Wikipedia Citations Database
Python
12
star
45

doublethink

rethinkdb python library
Python
11
star
46

s3_loader

Watch for local files to appear and move them into S3
Python
11
star
47

wayback-machine-android

Kotlin
10
star
48

Sparkling

Internet Archive's Sparkling Data Processing Library
Scala
10
star
49

archive-commons

Java
10
star
50

draintasker

a tool for continuously ingesting w/arc files into the archive
Python
9
star
51

ias3

Internet Archive S3-like connector
Python
8
star
52

cicd

build & test using github registry; deploy to nomad clusters
7
star
53

wayback-radial-tree

JavaScript
7
star
54

chocula

journal-level metadata munging. part of fatcat project
Python
7
star
55

read_api_extras

Demo code for the Open Library Read API
7
star
56

web_collection_search

An API wrapper to the Elasticsearch index of web archival collections and a web UI to explore those indexes.
Python
7
star
57

dweb-archivecontroller

JavaScript
7
star
58

epub-labs

epub-labs
6
star
59

ia

A JS interface to archive.org
JavaScript
6
star
60

iaux-typescript-wc-template

IAUX Typescript WebComponent Template
TypeScript
6
star
61

wikibase-patcher

Python library for interacting with the Wikibase REST API
Python
6
star
62

offlinesolr

Tool to build solr index offline
Java
6
star
63

ia-bin-tools

Internet Archive Command-line Utilities
C
6
star
64

dweb-objects

JavaScript
5
star
65

archive-ocr-tools

Python
5
star
66

iare

An interactive IARI JSON viewer
JavaScript
5
star
67

wayback-machine-safari

JavaScript
5
star
68

collections-cleaners

Shell
5
star
69

trendmachine

A mathematical model to calculate a normalized score to quantify the temporal resilience of a web page as a time-series data based on the historical observations of the page in web archives.
Python
5
star
70

acs4_py

Python interface to ACS4
Python
4
star
71

esbuild_es5

minify JS/TS files using `esbuild` and `swc` down to ES5 (uses `deno`)
TypeScript
4
star
72

iaux-search-service

TypeScript
4
star
73

map-of-the-web

Python
4
star
74

eventer

Eventer is a simple event dispatching library in Python
Python
4
star
75

iaux-donation-form

The Internet Archive Donation Form
TypeScript
4
star
76

internetarchive.github.com

Internet Archive Open Source Blog
CSS
4
star
77

isodos

Go module to interact with Internet Archive's Isodos API
Go
4
star
78

internet-archive-alexa-skill

JavaScript
3
star
79

mediawiki-extension-archive-leaf

A MediaWiki extension that supports importing of Archive.org palm leaf items
JavaScript
3
star
80

hashitalksdemo

JavaScript
3
star
81

openlibrary-api

API documentation for https://github.com/internetarchive/openlibrary
HTML
3
star
82

iaux-collection-browser

TypeScript
3
star
83

btget

Command line retrieval of torrents using transmission-daemon (via transmission-remote)
Python
3
star
84

strainer

Heritrix frontier files manipulation tool.
Go
3
star
85

httpd

Fast and easy-to-use web server, using the Deno native http server (hyper in rust). It serves static files & dirs, with arbitrary handling using an optional `handler` argument.
JavaScript
3
star
86

file_server_plus

`deno` static file webserver, clone of `file_server.ts`, PLUS an additional final "404 handler" to run arbitrary JS/TS
TypeScript
2
star
87

dyno

JavaScript
2
star
88

tarb_insights

A Streamlit application to visualize Wikipedia IABot statistics
Python
2
star
89

rulesengine-client

Python client package for the playback rules engine
Python
2
star
90

coderunr

deploy saved changes to website unique hostnames instantly -- can skip commits, pushes & full CI/CD
Shell
2
star
91

archiveorg-e2e-playwright

TypeScript
2
star
92

deferred

Redis promises & futures library for Predis / PHP
PHP
2
star
93

hello-js

an example of full CI/CD from GitHub to a nomad cluster
JavaScript
2
star
94

wiki-references-db

Data models and scripts to build a database of references (broadly defined) appearing on Wikipedia and other wikis
Python
2
star
95

maisy

Project Gutenberg collection importation via IAS3 interface
Python
2
star
96

kohacon2011-presentation

Presentation for KohaCon 2011
Shell
2
star
97

rulesengine

model and front-end for rules for managing wayback playback
Python
2
star
98

deploy

GitHub Action to deploy to a nomad cluster
2
star
99

iaux-democracys-library

A web component that highlights Democracy's Library
TypeScript
2
star
100

cgraphbot

Wikibase bot for updating identifiers and citation relationships
Python
2
star