• This repository has been archived on 11/Mar/2020
  • Stars
    star
    160
  • Rank 234,703 (Top 5 %)
  • Language
    Shell
  • License
    Apache License 2.0
  • Created over 9 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Tool to migrate Docker images from Docker Hub or v1 registry to a v2 registry

docker/migrator

Tool to migrate Docker images from Docker Hub or v1 registry to a v2 registry including Amazon Elastic Container Registry (ECR)

https://hub.docker.com/r/docker/migrator/

Usage

docker run -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e V1_REGISTRY=v1.registry.fqdn \
    -e V2_REGISTRY=v2.registry.fqdn \
    docker/migrator

Environment Variables

The following environment variables can be set:

Required

  • V1_REGISTRY - DNS hostname of your v1 registry or Docker Hub (Do not include https://)
    • If migrating images from Docker Hub, use docker.io
  • V2_REGISTRY - DNS hostname of your v2 registry (Do not include https://)

Optional

  • AWS_ACCESS_KEY_ID - AWS Access Key supplied as either an environment variable or as a part of your credentials file.
  • AWS_REGION - AWS Region, must be specified if using ECR
  • AWS_SECRET_ACCESS_KEY - AWS Secret Access Key supplied as either an environment variable or as a part of your credentials file.
  • ERROR_ACTION - Sets the default action on error for pushes and pulls
    • prompt - (Default) Prompt for user input as to what action to take on error
    • retry - Retry the failed action on error (may cause infinite loop of failure)
    • skip - Log the error and continue migration on error
    • abort - Abort the migration on error
  • MIGRATION_INCREMENT - Breaks up migration in chunks of n images
    • Defaults to migrating all images at once if not specified
    • Must be a positive integer
    • Only works if source and destination are not the same FQDN
  • USER_PROMPT - Sets the default action for user prompts (non-error)
    • true - (Default) Prompts user for input/validation
    • false - Skips user prompt and automatically proceeds
  • NO_LOGIN
    • true - Skips docker login for both the v1 and v2 registries
    • false - (Default) Prompts user to login to the v1 and v2 registries
  • V1_NO_LOGIN
    • true - Skips docker login for the v1 registry
    • false - (Default) Prompts user to login to the v1 registry
  • V2_NO_LOGIN
    • true - Skips docker login for the v2 registry
    • false - (Default) Prompts user to login to the v2 registry
  • USE_INSECURE_CURL
    • true - Allows curl to perform insecure SSL connections for querying APIs
    • false - (Default) Require curl to perform secure SSL connections for querying APIs
  • USE_HTTP
    • true - Allows curl to connect to both the v1 and v2 registries over HTTP
      • Note: daemon must also have --insecure-registry option set
    • false - (Default) Requires curl to connect to v1 and v2 registries over HTTPS
  • V1_USE_HTTP
    • true - Allows curl to connect to v1 registry running over HTTP
      • Note: daemon must also have --insecure-registry option set
    • false - (Default) Requires curl to connect to v1 registry over HTTPS
  • V2_USE_HTTP
    • true - Allows curl to connect to v2 registry running over HTTP
      • Note: daemon must also have --insecure-registry option set
    • false - (Default) Requires curl to connect to v2 registry over HTTPS
  • DOCKER_HUB_ORG - Docker Hub organization name to migrate images from
    • Defaults to the username used to login to Docker Hub if not provided
  • V1_FULL_REPO_LIST
    • If provided, this allows the user to provide a whitespace separated list of repos for migration. This allows skipping the V1 call to _search (some setups might have search disabled)
  • V1_REPO_FILTER - Search filter to limit the scope of the repositories to migrate (uses grep basic regular expression interpretation)
    • Note: This only filters the repositories returned from the source registry search API, not the individual tags
  • V1_TAG_FILTER - Search filter to limit the scope of the tags to migrate (Plain text matching).
  • LIBRARY_NAMESPACE - Sets option to migrate official namespaces (images where there is no namespace provided) to the library/ namespace (Note: must be set to true for DTR 1.4 or greater)
    • true - (Default) Adds library namespace to image names
    • false - Keeps images as they are without a namespace
  • SKIP_EXISTING_TAGS - Option to skip tags that exist at the target repository
    • true - Do not migrate tags that exist at the target repository
    • false - (Default) Do not skip any tags
  • Custom CA certificate and Client certificate support - for custom CA and/or client certificate support to your v1 and/or v2 registries, you should utilize a volume to share them into the container by adding the following to your run command:
    • -v /etc/docker/certs.d:/etc/docker/certs.d:ro
  • V1_USERNAME - Username used for docker login to the v1 registry
  • V1_PASSWORD - Password used for docker login to the v1 registry
  • V1_EMAIL - Email used for docker login to the v1 registry
  • V2_USERNAME - Username used for docker login to the v2 registry
  • V2_PASSWORD - Password used for docker login to the v2 registry
  • V2_EMAIL - Email used for docker login to the v2 registry

Note: You must use all three variables (V1_USERNAME, V1_PASSWORD, and V1_EMAIL or V2_USERNAME, V2_PASSWORD, and V2_EMAIL) for the given automated docker login to function properly. Omitting one will prompt the user for input of all three.

Prerequisites

This migration tool assumes the following:

  • You have a v1 registry (or Docker Hub) and you are planning on migrating to a v2 registry
  • The new v2 registry can either be running using a different DNS name or the same DNS name as the v1 registry - both scenarios work in this case. If you are utilizing the same DNS name for your new v2 registry, set both V1_REGISTRY and V2_REGISTRY to the same value.

It is suggested that you run this container on a Docker engine that is located near your registry as you will need to pull down every image from your v1 registry (or Docker Hub) and push them to the v2 registry to complete the migration. This also means that you will need enough disk space on your local Docker engine to temporarily store all of the images. If you have limited disk space, it is suggested that you use the MIGRATION_INCREMENT option to migrate n number of images at a time.

If you're interested in migrating to an Amazon Elastic Container Registry (ECR) you will additionally need to supply your AWS API keys to the migrator tool. This can be accomplished in one of the two following ways:

docker run -it \
    -v ~/.aws:/root/.aws:ro \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e V1_REGISTRY=v1.registry.fqdn \
    -e V2_REGISTRY=v2.registry.fqdn \
docker/migrator

docker run -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e AWS_ACCESS_KEY_ID=<key> \
    -e AWS_SECRET_ACCESS_KEY=<secret> \
    -e V1_REGISTRY=v1.registry.fqdn \
    -e V2_REGISTRY=v2.registry.fqdn \
docker/migrator

How Migration Works

The migration occurs using an automated script inside of the Docker container. Running using the above usage will work as expected.

  1. Login to the v1 registry or Docker Hub (Optional)
    • If you do not have authentication enabled, leave the username blank when prompted
  2. Query the v1 registry or Docker Hub for a list of all repositories
  3. With the list of images, query the v1 registry or Docker Hub for all tags for each repository. This becomes the list of all images with tags that you need to migrate
  4. Using a Docker engine, pull all images (including each tag)
  5. Once all images are pulled, there are a few options for next steps:
    1. If the same DNS record will be used for the v1 and v2 registries:
      • Have user switch the DNS record over to the new server's IP or if same box to be used, stop the v1 registry and start the v2 registry
    2. If a different DNS record will be used for the v1 and v2 registries:
      • Re-tag all images to change the tagging from the old DNS record to the new one
  6. Login to the v2 registry (Optional)
    • If you do not have authentication enabled, leave the username blank when prompted
  7. Push all images and tags to the v2 registry
  8. Verify v1 to v2 image migration was successful (not yet implemented)
  9. Cleanup local docker engine to remove images

asciicast

Logging Migration Output

If you need to log the output from migrator, add 2>&1 | tee migration.log to the end of the command shown above to capture the output to a file of your choice.

More Repositories

1

classicswarm

Swarm Classic: a container clustering system. Not to be confused with Docker Swarm which is at https://github.com/docker/swarmkit
Go
5,759
star
2

docker-ce

⚠️ This repository is deprecated and will be archived (Docker CE itself is NOT deprecated) see the https://github.com/docker/docker-ce/blob/master/README.md ⚠️
Go
5,747
star
3

toolbox

The Docker Toolbox
Inno Setup
3,215
star
4

docker-registry

This is **DEPRECATED**! Please go to https://github.com/docker/distribution
Python
2,885
star
5

deploykit

A toolkit for creating and managing declarative, self-healing infrastructure.
Go
2,249
star
6

libcontainer

PROJECT MOVED TO RUNC
Go
1,189
star
7

compose-cli

Easily run your Compose application to the cloud with compose-cli
Go
959
star
8

dockercloud-haproxy

HAproxy image that autoreconfigures itself when used in Docker Cloud
Python
651
star
9

dockerlite

Lightweight virtualization system based on LXC and BTRFS. See dotcloud/docker.
Shell
496
star
10

docker-birthday-3

480
star
11

ecs-plugin

See http://github.com/docker/compose-cli
Go
411
star
12

go-redis-server

A Go implementation of the Redis server protocol
Go
404
star
13

communitytools-image2docker-win

PowerShell
304
star
14

openstack-docker

Nova driver and Glance backend to use Docker inside OpenStack
Python
293
star
15

go-p9p

A modern, performant 9P library for Go.
Go
206
star
16

jenkins-pipeline-scripts

Groovy
189
star
17

swarm-frontends

Deploy Kubernetes with Swarm
Shell
182
star
18

leeroy

Jenkins integration with GitHub pull requests
Go
177
star
19

leadership

Distributed Leader Election using docker/libkv
Go
160
star
20

go-healthcheck

Go
146
star
21

global-hack-day-3

Participant final submissions for the 3rd edition of the Docker Global Hack Day
Go
126
star
22

infra-container_exporter

Prometheus exporter exposing container metrics
Go
126
star
23

docker-snap

Shell
120
star
24

communitytools-image2docker-linux

Go
116
star
25

boatyard

[DEPRECATED] Web UI to docker build
Go
114
star
26

docker-tutorial

JavaScript
101
star
27

for-aws

92
star
28

dnsserver

A small DNS service for communicating A and SRV records.
Go
82
star
29

swarm-microservice-demo-v1

Swarm Web App Demo Cluster: Voting App
JavaScript
81
star
30

golem

Integration test framework using Docker
Go
76
star
31

dockercloud-cli

CLI for Docker Cloud
Python
74
star
32

dcus-hol-2017

Dockercon 2017 Hands-On Labs
JavaScript
61
star
33

infra-docker-lloyd

Shipment insurance for docker containers
Shell
55
star
34

docker-network

Experimental network control tool for Docker
Go
48
star
35

ucp_lab

Lab for UCP at DockerCon EU
48
star
36

ee-workshop

43
star
37

bender

IRC bot to manage scrum-like standup meetings
Python
43
star
38

dockercloud-authorizedkeys

Adds a user public SSH key to the host's ~/.ssh/authorized_keys via docker container
Shell
42
star
39

infra-reefer

Managing a stable environment in your container.
Go
41
star
40

dcus-hol-2016

Python
41
star
41

minecraft

Minecraft Dockerfile used in Kitematic
Dockerfile
39
star
42

oscalkit

NIST OSCAL SDK and CLI
Go
36
star
43

gordon-bot

Gordon the turtle IRC bot to rebuild Jenkins builds.
Go
34
star
44

dockercloud-hello-world

Hello world!
PHP
34
star
45

dockercloud-agent

Agent to manage docker in nodes controlled by Docker Cloud
34
star
46

openstack-heat-docker

Orchestrate Docker containers with OpenStack Heat
Python
32
star
47

cloud-integration-beta

Docker CLI with ACI integration (beta)
Shell
32
star
48

modernizing_apps_for_java_devs

Java
31
star
49

swarm-library-image

Official Image packaging for Classic Swarm, now archived
Shell
31
star
50

kitematic-html

Shell
30
star
51

serverlessbenchmark

Deploying and benchmarking open source Function-As-A-Service frameworks
HCL
29
star
52

infra-docker-proper

OS. Proper swabbing the deck - Cleanup of old Docker containers and images.
Go
29
star
53

dotnet-musicstore

Music Store web app written in .NET Core
C#
28
star
54

docker-bb

Docker binary builder, triggered from nsq messages.
Go
28
star
55

python-dockercloud

Python library for Docker Cloud
Python
28
star
56

for-azure

27
star
57

docker-status

Python
26
star
58

v1.10-migrator

Prepare docker graph directory for content addressability
Go
24
star
59

registration-docker

Tutorial for debugging Java applications in Docker
Java
22
star
60

infrakit.aws

Infrakit plugins for AWS.
Go
21
star
61

stacks

Go
21
star
62

hybrid-workshop

HTML
21
star
63

infra-prometheus-consul

Prometheus with dynamic configuration by consul and consul-template
Shell
20
star
64

dockercloud-node

Docker Cloud node as a docker-in-docker container
Shell
19
star
65

pulpo

Configuration mechanism for Node
TypeScript
18
star
66

mta-netfx-dev

Sample code from video series: Modernizing .NET Apps with Docker - for Developers
16
star
67

go-dockercloud

Go library for Docker Cloud API
Go
15
star
68

nginx-gohttp

Go
15
star
69

web-application-firewall

Sample cross-platform Web Application Firewall, used as a security proxy for existing web apps
C#
15
star
70

docker-for-oracle-weblogic

Shell
14
star
71

windows-container-utility

Windows Container Utility
C++
13
star
72

mta_java_workshop

MTA Java Workshop for Dockercon 2018
Java
12
star
73

docker-pets

Docker Pets is a simple application that's useful for testing out features of Docker Enterprise Edition.
Python
12
star
74

infrakit.gcp

Infrakit plugins for Google Cloud Platform.
Go
12
star
75

markdownlint

Docker projects markdown linter
Go
11
star
76

infrakit.digitalocean

DEPRECATED: Infrakit plugin(s) for Digitalocean
Go
11
star
77

varnish

Dockerized Varnish server image
Shell
10
star
78

birthdaysite

Docker's Birthday Site
CSS
10
star
79

dockercloud-network-daemon

[Docker Cloud System Image] Provides a secure overlay network between nodes
10
star
80

irc-minutes

Docker-dev irc meeting Organization and minutes.
10
star
81

buildx-packaging

Linux distro and static packaging for buildx
HCL
10
star
82

dnsutils

Docker image with dnsutils installed
9
star
83

tomcat_cluster

Java
8
star
84

dockercloud-quickstart-python

Getting started in Docker Cloud with Python
Python
8
star
85

ee-exercises

7
star
86

moby-extras

for stitching together docker-ce
Go
7
star
87

dockercloud-quickstart-go

Getting started in Docker Cloud with Go
Go
6
star
88

quickstart-node

Getting started in Tutum with Node.js
JavaScript
6
star
89

quickstart-go

Getting started in Tutum with Go
Go
6
star
90

dockercloud-events

[Docker Cloud System Image] Monitors docker events and sends updates to Docker Cloud's API
Go
6
star
91

infra-heka_dashboard_exporter

prometheus exporter for heka logging daemon
Go
5
star
92

vol-test

Go
5
star
93

licensing

Library for interacting with Docker issued product licenses
Go
5
star
94

homebrew-oscalkit

Homebrew recipe for oscalkit
Ruby
5
star
95

app-e2e

Go
4
star
96

mulesoft-docker-demo

C#
3
star
97

hybrid-app

Java
2
star
98

dctx

A context package for use across docker projects
Go
2
star
99

virtualbox

Mirror of VirtualBox
2
star
100

notary-signer-image

2
star