• Stars
    star
    655
  • Rank 66,114 (Top 2 %)
  • Language
    Python
  • License
    BSD 3-Clause "New...
  • Created about 8 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Reference deployment of JupyterHub with docker

jupyterhub-deploy-docker

jupyterhub-deploy-docker provides a reference deployment of JupyterHub, a multi-user Jupyter Notebook environment, on a single host using Docker.

Possible use cases include:

  • Creating a JupyterHub demo environment that you can spin up relatively quickly.
  • Providing a multi-user Jupyter Notebook environment for small classes, teams or departments.

Disclaimer

This deployment is NOT intended for a production environment. It is a reference implementation that does not meet traditional requirements in terms of availability, scalability, or security.

If you are looking for a more robust solution to host JupyterHub, or you require scaling beyond a single host, please check out the excellent zero-to-jupyterhub-k8s project.

Technical Overview

Key components of this reference deployment are:

  • Host: Runs the JupyterHub components in a Docker container on the host.

  • Authenticator: Uses Native Authenticator to authenticate users. Any user will be allowed to sign up.

  • Spawner: Uses DockerSpawner to spawn single-user Jupyter Notebook servers in separate Docker containers on the same host.

  • Persistence of Hub data: Persists JupyterHub data in a Docker volume on the host.

  • Persistence of user notebook directories: Persists user notebook directories in Docker volumes on the host.

Prerequisites

Docker

This deployment uses Docker, via Docker Compose, for all the things.

  1. Use Docker's installation instructions to set up Docker for your environment.

Authenticator setup

This deployment uses JupyterHub Native Authenticator to authenticate users.

  1. An single admin user will be enabled by default. Any user will be allowed to sign up.

Build the JupyterHub Docker image

  1. Use docker compose to build the JupyterHub Docker image:

    docker compose build

Customisation: Jupyter Notebook Image

You can configure JupyterHub to spawn Notebook servers from any Docker image, as long as the image's ENTRYPOINT and/or CMD starts a single-user instance of Jupyter Notebook server that is compatible with JupyterHub.

To specify which Notebook image to spawn for users, you set the value of the DOCKER_NOTEBOOK_IMAGE environment variable to the desired container image.

Whether you build a custom Notebook image or pull an image from a public or private Docker registry, the image must reside on the host.

If the Notebook image does not exist on the host, Docker will attempt to pull the image the first time a user attempts to start his or her server. In such cases, JupyterHub may timeout if the image being pulled is large, so it is better to pull the image to the host before running JupyterHub.

This deployment defaults to the quay.io/jupyter/base-notebook Notebook image, which is built from the base-notebook Docker stacks.

You can pull the image using the following command:

docker pull quay.io/jupyter/base-notebook:latest

Run JupyterHub

Run the JupyterHub container on the host.

To run the JupyterHub container in detached mode:

docker compose up -d

Once the container is running, you should be able to access the JupyterHub console at http://localhost:8000.

To bring down the JupyterHub container:

docker compose down

FAQ

How can I view the logs for JupyterHub or users' Notebook servers?

Use docker logs <container>. For example, to view the logs of the jupyterhub container

docker logs jupyterhub

How do I specify the Notebook server image to spawn for users?

In this deployment, JupyterHub uses DockerSpawner to spawn single-user Notebook servers. You set the desired Notebook server image in a DOCKER_NOTEBOOK_IMAGE environment variable.

JupyterHub reads the Notebook image name from jupyterhub_config.py, which reads the Notebook image name from the DOCKER_NOTEBOOK_IMAGE environment variable:

# DockerSpawner setting in jupyterhub_config.py
c.DockerSpawner.image = os.environ['DOCKER_NOTEBOOK_IMAGE']

If I change the name of the Notebook server image to spawn, do I need to restart JupyterHub?

Yes. JupyterHub reads its configuration, which includes the container image name for DockerSpawner. JupyterHub uses this configuration to determine the Notebook server image to spawn during startup.

If you change DockerSpawner's name of the Docker image to spawn, you will need to restart the JupyterHub container for changes to occur.

In this reference deployment, cookies are persisted to a Docker volume on the Hub's host. Restarting JupyterHub might cause a temporary blip in user service as the JupyterHub container restarts. Users will not have to login again to their individual notebook servers. However, users may need to refresh their browser to re-establish connections to the running Notebook kernels.

How can I back up a user's notebook directory?

There are multiple ways to Back up and restore data in Docker containers.

Suppose you have the following running containers:

    docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"

    CONTAINER ID        IMAGE                    NAMES
    bc02dd6bb91b        quay.io/jupyter/minimal-notebook jupyter-jtyberg
    7b48a0b33389        quay.io/jupyterhub               jupyterhub

In this deployment, the user's notebook directories (/home/jovyan/work) are backed by Docker volumes.

    docker inspect -f '{{ .Mounts }}' jupyter-jtyberg

    [{jtyberg /var/lib/docker/volumes/jtyberg/_data /home/jovyan/work local rw true rprivate}]

We can back up the user's notebook directory by running a separate container that mounts the user's volume and creates a tarball of the directory.

docker run --rm \
  -u root \
  -v /tmp:/backups \
  -v jtyberg:/notebooks \
  quay.io/jupyter/minimal-notebook \
  tar cvf /backups/jtyberg-backup.tar /notebooks

The above command creates a tarball in the /tmp directory on the host.

More Repositories

1

jupyterhub

Multi-user server for Jupyter notebooks
Python
7,506
star
2

binderhub

Run your code in the cloud, with technology so advanced, it feels like magic!
Python
2,465
star
3

repo2docker

Turn repositories into Jupyter-enabled Docker images
Python
1,577
star
4

zero-to-jupyterhub-k8s

Helm Chart & Documentation for deploying JupyterHub on Kubernetes
Python
1,433
star
5

the-littlest-jupyterhub

Simple JupyterHub distribution for 1-100 users on a single server
Python
978
star
6

kubespawner

Kubernetes spawner for JupyterHub
Python
511
star
7

dockerspawner

Spawns JupyterHub single user servers in Docker containers
Python
472
star
8

oauthenticator

OAuth + JupyterHub Authenticator = OAuthenticator
Python
392
star
9

jupyter-server-proxy

Jupyter notebook server extension to proxy web services.
Python
319
star
10

jupyterhub-deploy-teaching

Reference deployment of JupyterHub and nbgrader on a single server
Python
240
star
11

configurable-http-proxy

node-http-proxy plus a REST API
JavaScript
224
star
12

nbgitpuller

Jupyter server extension to sync a git repository one-way to a local path
Python
199
star
13

ldapauthenticator

LDAP Authenticator Plugin for Jupyter
Python
195
star
14

jupyterhub-tutorial

Tutorial materials for deploying JupyterHub
Shell
192
star
15

jupyterlab-hub

Deprecated: JupyterLab extension for running JupyterLab with JupyterHub
TypeScript
181
star
16

helm-chart

A store of Helm chart tarballs for deploying JupyterHub and BinderHub on a Kubernetes cluster
173
star
17

batchspawner

Custom Spawner for Jupyterhub to start servers in batch scheduled systems
Python
169
star
18

repo2docker-action

A GitHub action to build data science environment images with repo2docker and push them to registries.
Shell
137
star
19

mybinder.org-user-guide

Turn a Git repo into a collection of interactive notebooks. This is Binder's user documentation repository.
Python
137
star
20

jupyter-rsession-proxy

Jupyter extensions for running an RStudio rsession proxy
Python
110
star
21

jupyter-remote-desktop-proxy

Run a Linux Desktop on a JupyterHub
Python
92
star
22

systemdspawner

Spawn JupyterHub single-user notebook servers with systemd
Python
88
star
23

jupyterhub-idle-culler

JupyterHub service to cull idle servers and users
Python
88
star
24

jupyterhub-the-hard-way

Understanding JupyterHub by deploying it step by step
77
star
25

mybinder.org-deploy

Deployment config files for mybinder.org
Jupyter Notebook
74
star
26

nativeauthenticator

JupyterHub-native User Authenticator
Python
69
star
27

ltiauthenticator

A JupyterHub authenticator for LTI
Python
62
star
28

team-compass

A repository for team interaction, syncing, and handling meeting notes across the JupyterHub ecosystem.
Jupyter Notebook
62
star
29

wrapspawner

Mechanism for runtime configuration of spawners for JupyterHub
Python
60
star
30

hubshare

A directory sharing service for JupyterHub
Python
57
star
31

chartpress

automate building and publishing images for helm charts
Python
53
star
32

traefik-proxy

JupyterHub proxy implementation with traefik
Jupyter Notebook
51
star
33

sudospawner

Spawn JupyterHub single-user servers with sudo
Python
49
star
34

firstuseauthenticator

JupyterHub Authenticator that lets users set passwords when they first log in
Python
48
star
35

grafana-dashboards

Grafana Dashboards useful for k8s + JupyterHub
Jsonnet
45
star
36

jupyterhub-deploy-hpc

Documented examples of Jupyterhub deployment in HPC settings
Python
35
star
37

jupyterhub-example-kerberos

Example for experimenting with how JupyterHub can be configured to work with Kerberos
Python
33
star
38

outreachy

Tasks, management and documentation for Outreachy Internships in JupyterHub
Python
32
star
39

dummyauthenticator

A Dummy JupyterHub Authenticator to make testing easy
Python
28
star
40

pebble-helm-chart

To deploy a server to replace Let's Encrypt in externally unreachable CI environment for testing purposes.
Shell
27
star
41

gh-scoped-creds

Provide fine-grained push access to GitHub from a JupyterHub
Python
23
star
42

action-k3s-helm

A GitHub action to install K3S, Calico, and Helm.
21
star
43

tmpauthenticator

JupyterHub authenticator that hands out temporary accounts for everyone. For use in tmpnb.org
Python
21
star
44

yarnspawner

Spawn JupyterHub single user notebook servers in Hadoop/YARN containers.
Python
19
star
45

jupyterhub-on-hadoop

Documentation and resources for deploying JupyterHub on Hadoop
Shell
18
star
46

simpervisor

Simple Python3 Supervisor library
Python
11
star
47

kerberosauthenticator

A JupyterHub authenticator using Kerberos
Python
11
star
48

nullauthenticator

Null Authenticator for JupyterHub instances that should have no login mechanism
Python
9
star
49

binder-data

A place to store data for Binder
Jupyter Notebook
9
star
50

docker-image-cleaner

Python script to cleanup unused docker images
Python
8
star
51

action-k8s-await-workloads

A Github Action to await k8s workloads to become Ready
TypeScript
8
star
52

research-facilities

resources for developing, deploying, and using Jupyter at Research Facilities
8
star
53

simplespawner

A simple variant of LocalProcessSpawner for JupyterHub that doesn't require users exist on the system
Python
6
star
54

action-k8s-namespace-report

GitHub Action to report info and logs from the current namespace.
Shell
6
star
55

pytest-jupyterhub

Reusable pytest plugin for testing JupyterHub's components
Python
5
star
56

binder-billing

A repository to explore billing data for mybinder.org
Jupyter Notebook
3
star
57

autodoc-traits

Sphinx extension to autodocument traitlets
Python
3
star
58

action-major-minor-tag-calculator

GitHub action to calculate major and minor semver tags, e.g. for tagging containers
JavaScript
3
star
59

mybinder-tools

Some tools for munging and utilizing kubectl
Jupyter Notebook
2
star
60

jhub-proposals

Repo for collecting conference proposals, proceedings, &c.
2
star
61

design

A collection of design material for JupyterHub and related projects
2
star
62

alabaster-jupyterhub

A slight modification of the Alabaster theme for use with JupyterHub projects
CSS
2
star
63

katacoda-scenarios

https://katacoda.com/ is a playground for learning, and these are our learning scenarios
1
star
64

nbgitpuller-downloader-googledrive

Python
1
star
65

.github

1
star
66

jupyterhub-python-repo-template

A template for generating Python repositories in the `jupyterhub` organization
Python
1
star
67

nbgitpuller-downloader-plugins

jupyterhub/nbgitpuller allows for content provider plugins, this python package provides downloader plugins
Python
1
star