• Stars
    star
    136
  • Rank 261,396 (Top 6 %)
  • Language
    Shell
  • Created almost 4 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Dirty hack to run a read-only, public Docker registry on almost any static file hosting service (e.g. NGINX, Netlify, S3...)

Registrish

This is kind of a Docker registry, but with many restrictions:

  • it's read-only (you can pull but you cannot push)
  • it only supports public access (no authentication)
  • it only supports a subset of the Docker Distribution API

The last point means that pulls from registrish will hopefully work, but might break in unexpected ways. See Limitations below for more info.

On the bright side, registrish can be deployed without running the registry code, using almost any static file hosting service. For instance:

  • a plain NGINX server (without LUA, JSX, or whatever custom module)
  • the Netlify CDN
  • an object store like S3, or a compatible one like R2 or Scaleway

Example

The following commands will run an Alpine image hosted on various locations, thanks to registrish.

Netlify:

docker run registrish.netlify.app/alpine echo hello there

R2:

docker run pub-a8f6c50314a0467b8f862a261a950bcf.r2.dev/alpine echo hello there

S3:

docker run registrish.s3.amazonaws.com/alpine echo hello there

Scaleway object store:

docker run registrish.s3.fr-par.scw.cloud/alpine echo hello there

Hosting your images with registrish

In the following example, we are going to host the official image alpine:latest with registrish.

Let's set a couple of env vars for convenience:

export DIR=tmp IMAGE=alpine TAG=latest

Let's obtain the manifests and blobs of the image. This requires Skopeo.

skopeo copy --all docker://$IMAGE:$TAG dir:$DIR

The --all flag means that we want to obtain a manifest list (i.e a multi-arch image), if one is available.

Then, we're going to move the files downloaded by Skopeo to their respective directories. Blobs go to the blobs directory, and manifests go to the manifests directory. All files get renamed to sha256:xxx where xxx is their SHA256 checksum. The top-level manifest also gets copied to the tag name to allow pulling by tag.

./dir2reg.sh

You can check that everything looks fine with the tree command:

tree v2

There should be:

  • a bunch of sha256:xxx files in blobs,
  • a bunch of sha256:xxx files in manifests,
  • one tag file (e.g. latest) in manifests.

Then, pick a registrish back-end and follow its specific instructions.

NGINX server

Generate the NGINX configration file. The configuration file will set content-type HTTP headers.

./gen-nginx.sh

Start NGINX in a local Docker container. It will be on port 5555.

docker-compose up

The image will be available as localhost:5555/$IMAGE:$TAG.

Netlify

Generate the Netlify headers file.

./gen-netlify.sh

To deploy to Netlify, it's better to deploy only the v2 directory and the _headers file:

TEMP=$(mktemp -d)
cp -a v2 _headers $TEMP
npx netlify deploy --dir $TEMP --prod

S3 bucket or compatible

The following assumes that you have configured your S3 credentials, for instance by setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Update this variable with your bucket name.

export BUCKET=registrish

If you are using an S3-compatible API, you need to set a few additional variables (or set the corresponding parameters in your profile).

  • R2:

    export CLOUDFLARE_ACCOUNT_ID=c4c9...
    export ENDPOINT="--endpoint-url https://$CLOUDFLARE_ACCOUNT_ID.r2.cloudflarestorage.com"
  • Scaleway:

    export AWS_DEFAULT_REGION=fr-par
    export ENDPOINT="--endpoint-url https://s3.fr-par.scw.cloud"

Sync files to the bucket.

./reg2bucket.sh

The image will be available as $BUCKET.s3.amazonaws.com/$IMAGE:$TAG (for S3) or $BUCKETNAME.$ENDPOINT/$IMAGE:$TAG (for compatible APIs).

Testing

When testing registrish, we want to be sure that the entire image will be pulled correctly with all its layers. If we try on our local container engine, we might already have some manifests and layers.

One way to test that is to use a throwaway Docker-in-Docker container.

docker run --name dind -d --privileged --net host docker:dind
docker exec dind docker pull REGISTRISH-IMAGE
docker rm -f dind

How it works

The Docker Registry is almost a static web server. The main trick is to handle Content-Type headers correctly.

As far as I understand, this is what happens when a container engine tries to pull an image by tag.

  • First, the engine wants to know the hash of the manifest that we're trying to pull.
  • To learn that hash, the engine makes a HEAD request on /v2/<image>/manifests/<tag>.
    • If the registry sends a Docker-Content-Digest header (which will look like sha256:<xxx>), that header is the hash, so the engine can go to the next step.
    • If the registry doesn't send a Docker-Content-Digest header, the engine makes a GET request on /v2/<image>/manifests/<tag>, computes the SHA256 checksum of the response body (let's say it's <xxx> to match the previous example). Now the engine has the hash (at the cost of an extra HTTP request).
  • The engine makes a request to /v2/<image>/manifests/sha256:<xxx>. The Content-Type will indicate if we're dealing with a v2 manifest (single-arch image) or a v2 manifest list (multi-arch image).
    • If it's a v2 manifest, we can use it directly.
    • If it's a manifest list, it contains a list of manifests. Each entry has a platform (e.g. linux/amd64) and a hash (for instance sha256:<yyy>). The engine picks the entry that it deems appropriate (because it matches its architecture, or one that it's compatible with) and it requests the corresponding manifest, on /v2/<image>/manifests/sha256:<yyy>. That manifest should be a v2 manifest.
  • The engine now has a v2 manifest. In the v2 manifest, there is a list of blobs. One of these blobs will be a JSON file containing the configuration of the image (entry point, command, volumes, etc.) and the other ones will be the layers of the images. The blobs are stored in /v2/<image>/blobs/sha256:<sha-of-the-blob>.

As long as we use the correct Content-Type when serving image manifests, the container engine should be happy. Unless...

Limitations

Unless the container engine explicitly asks a specific type of manifests, which it can do by using Accept request headers. If the engine asks for a v2 manifest (single-arch) and we serve a v2 manifest list (multi-arch), I expect that it will complain loudly.

I don't understand why my former colleagues at Docker decided to go with this scheme, instead of e.g. keeping v2 manifests in /manifests and storing the multi-arch manifest lists in e.g. /lists. It would have avoided using HTTP headers to alter the content served by the registry. ๐Ÿคท๐Ÿป

I also don't understand the point of that HEAD request and custom Docker-Content-Digest HTTP header. The same result could have been achieved with an explicit HTTP route to resolve tags to hashes. ๐Ÿคท๐Ÿป

Notes

Netlify is very fast to serve web pages, but not so much to serve binary blobs. (I suspect that they throttle them on purpose to prevent abuse, but that's just an intuition.)

Their terms of service state the following (in August 2020):

Users must exercise caution when hosting large downloads (>10MB). Netlify reserves the right to refuse to host any large downloadable files.

Providers and object stores that might not work

I've tried to use OVHcloud object storage, but it doesn't seem to be easy to do so. OVHcloud storage buckets have very long URLs like https://storage.gra.cloud.ovh.net/v1/AUTH_xxx-long-tenant-id-xxx/bucketname and the Docker registry protocol doesn't support uppercase characters in image names.

It's possible to use custom domain names, but then there are certificate issues. If you know an easy way to make it work, let me know!

TODO

  • add 404 page on Netlify
  • try CloudFlare R2 as soon as it's out :)

Similar work and prior art

More Repositories

1

pipework

Software-Defined Networking tools for LXC (LinuX Containers)
Shell
4,139
star
2

container.training

Slides and code samples for training, tutorials, and workshops about Docker, containers, and Kubernetes.
Shell
3,559
star
3

nsenter

Shell
2,582
star
4

ampernetacle

HCL
2,519
star
5

dind

Docker in Docker
Shell
2,474
star
6

dockvpn

Recipe to build an OpenVPN image for Docker
Shell
833
star
7

squid-in-a-can

Python
357
star
8

pxe

Dockerfile to build a PXE server in a Docker container
Shell
250
star
9

minimage

Minimal Docker images: a collection of Dockerfiles illustrating how to reduce container image size.
Shell
206
star
10

griode

Griode + Novation Launchpad + Raspberry Pi = a music instrument!
Python
139
star
11

shpod

Container image to get a consistent training environment to work on Kubernetes.
Dockerfile
134
star
12

critmux

Docker + CRIU + tmux = magic!
Dockerfile
117
star
13

dockercoins

Python
98
star
14

docker-busybox

Busybox for Stackbrew
Shell
94
star
15

sekexe

Separate Kernel Execution: execute a process within user-mode-linux and retrieve its output and status code
Shell
79
star
16

dessine-moi-un-cluster

Instructions to build a Kubernetes control plane one piece at a time, for learning purposes.
Shell
78
star
17

gunsub

Get your github notifications under control!
Python
74
star
18

syslogdocker

70
star
19

stevedore

Containerize your development environments
Shell
68
star
20

hamba

Shell
67
star
21

obs-docker

OBS-Studio (and a few extra tools) in containers
Python
62
star
22

docker2docker

Shell
26
star
23

wordsmith

Java
23
star
24

intro-to-docker

CSS
23
star
25

jpetazzo.github.io

HTML
22
star
26

go-docker-

20
star
27

trainingwheels

HTML
18
star
28

snakedeck

Elgato StreamDeck controller for Linux, in Python.
Python
17
star
29

nsplease

Tiny Kubernetes operator to create Namespaces on demand (for CI/CD, for instance)
Shell
16
star
30

django

Django on DotCloud tutorial
Python
16
star
31

whisperfiles

A bunch of Dockerfiles for OpenAI Whisper, to illustrate various image optimization techniques
Shell
15
star
32

foundation-example

Shell
14
star
33

httpenv

Tiny HTTP server showing the environment variables
Go
14
star
34

buildkit-demos

Dockerfile
13
star
35

dctrl

Shell
13
star
36

orchestration-workshop

We have moved! We are now at โ†’ https://github.com/jpetazzo/container.training
HTML
12
star
37

decoup

Python
11
star
38

layeremove

Surgically remove layers from a Docker image (with a chainsaw)
Python
11
star
39

znc-on-dotcloud

Shell
11
star
40

tilestream-on-dotcloud

Python
10
star
41

littleboxes

Just for fun scripts to manage local cloud-like VMs with KVM
Shell
10
star
42

django-and-mongodb-on-dotcloud

Django on DotCloud tutorial, using MongoDB to store objects!
Python
10
star
43

kubercoins

8
star
44

sstk

Shell
8
star
45

geodjango-on-dotcloud

Python
7
star
46

meteor-on-dotcloud

7
star
47

solr-on-dotcloud

JavaScript
7
star
48

seleterm

Selenium for terminal applications
Python
6
star
49

snap-on-dotcloud

Shell
6
star
50

httplat

Minimalist Prometheus exporter to collect the latency of an HTTP target
Go
6
star
51

boggle

Solver for the Boggle Word Game
Python
6
star
52

mume

Python
6
star
53

gitorial

(Ab)use git history to write tutorials!
Python
6
star
54

postgresql-on-dotcloud

Python
5
star
55

django-on-gpaas

Django on GANDI PAAS
Python
5
star
56

scangraph

Retrieve point coordinates from a raster plot
JavaScript
5
star
57

hano

Online IDE for Node.js on dotCloud
Shell
5
star
58

pyramid-on-dotcloud

Python
5
star
59

traefik-compose

Quick demo showing how to run web sites (like Wordpress) on Docker with Traefik
4
star
60

plumber

Shell
4
star
61

consul

jpetazzo's Consul image
Shell
4
star
62

zwave-exporter

Prometheus exporter for Z-Wave sensors
JavaScript
4
star
63

jenkins-on-dotcloud

Shell
4
star
64

color

Go
4
star
65

busyhttp

A trivial HTTP server that eats CPU cycles at each request.
Python
4
star
66

tinydocgen

Tiny document generator using Jinja2, Markdown, and WeasyPrint.
Makefile
3
star
67

ngrok

3
star
68

charliebot

Python
3
star
69

prettypictures

3
star
70

usb-webcam-analyzer

Python
3
star
71

rickroll-in-docker

HTML
3
star
72

dnc

Domain Name Command-line tool
Python
3
star
73

python-simple-logging

Python
3
star
74

replay.container.training

Shell
3
star
75

riak-on-dotcloud-ALPHA

Shell
3
star
76

ucengine-on-dotcloud-ALPHA

Shell
3
star
77

django-r2d2

R2D2 (RRDDashboard) is a Django application to draw graphs from metrics coming from e.g. collectd.
Python
3
star
78

pieuvre

Distributed HTTP proxy in Node.js
JavaScript
2
star
79

tmp-sealedsecret-juin-2022

2
star
80

couchdb-on-dotcloud-ALPHA

Shell
2
star
81

escapehash

Python
2
star
82

dockerhubratelimit

Shell
2
star
83

workflows

2
star
84

python-worker-on-dotcloud

Shell
2
star
85

tcl-on-dotcloud-ALPHA

Shell
2
star
86

elastic-gke

HCL
2
star
87

dampmam

Docker-Apache-MySQL-PHP but without Apache and MySQL
JavaScript
2
star
88

watchdns

Shell
2
star
89

highfive

Dockerfile
2
star
90

memcached-on-dotcloud

2
star
91

pawd

PulseAudio Web Daemon
2
star
92

pingr

HTTP server to ping other servers and report their status
Go
2
star
93

dotfiles-old

Config files for various environments
Shell
2
star
94

pgpool-II-on-dotcloud

2
star
95

ls

An ls image for the Docker Fundamentals training
Shell
2
star
96

jetty-on-dotcloud

Reimplementation of dotCloud java service using the custom build API
Shell
2
star
97

dockage

Shell
2
star
98

jira-on-dotcloud

Shell
2
star
99

tornado-on-dotcloud

Python
2
star
100

aiguillage

Nginx
2
star