docker-ipv6nat

This project mimics the way Docker does NAT for IPv4 and applies it to IPv6. Jump to Usage to get started right away.

Why would I need this?

Unfortunately, initially Docker was not created with IPv6 in mind. It was added later and, while it has come a long way, is still not as usable as one would want. Much discussion is still going on as to how IPv6 should be used in a containerized world; see the various GitHub issues linked below.

Currently, you can let Docker give each container an IPv6 address from your (public) pool, but this has disadvantages:

• Giving each container a publicly routable address means all ports (even unexposed / unpublished ports) are suddenly reachable by everyone, if no additional filtering is done (docker/docker#21614)
• By default, each container gets a random IPv6, making it impossible to do properly do DNS; the alternative is to assign a specific IPv6 address to each container, still an administrative hassle (docker/docker#13481)
• Published ports won't work on IPv6, unless you have the userland proxy enabled (which, for now, is enabled by default in Docker)
• The userland proxy, however, seems to be on its way out (docker/docker#14856) and has various issues, like:
  • It can use a lot of RAM (docker/docker#11185)
  • Source IP addresses are rewritten, making it completely unusable for many purposes, e.g. mail servers (docker/docker#17666, docker/libnetwork#1099)

Special mention of @JonasT who submitted the majority of the above issues, pointing out some of the practical issues when using IPv6 with Docker.

So basically, IPv6 for Docker can (depending on your setup) be pretty unusable (docker/docker#13481) and completely inconsistent with the way how IPv4 works (docker/docker#21951). Docker images are mostly designed with IPv4 NAT in mind, having NAT provide a layer of security allowing only published ports through, and letting container linking or user-defined networks provide inter-container communication. This does not go hand in hand with the way Docker IPv6 works, requiring image maintainers to rethink/adapt their images with IPv6 in mind.

Welcome IPv6 NAT

So what does this repo do? It attempts to resolve all of the above issues by managing ip6tables to setup IPv6 NAT for your containers, similar to how it's done by the Docker daemon for IPv4.

• A ULA range (RFC 4193) is used for containers; this automatically means the containers will NOT be publicly routable
• Published ports are forwarded to the corresponding containers, similar to IPv4
• The original IPv6 source addresses are maintained, again, just like with IPv4
• Userland proxy can be turned off and IPv6 will still work

This makes a transition to IPv6 completely painless, without needing to make changes to your images.

Please note:

• The Docker network API is required, so at least Docker 1.9.0
• It triggers only on ULA ranges (so within fc00::/7), e.g. fd00:dead:beef::/48
• Only networks with driver bridge are supported; this includes Docker's default network ("bridge"), as well as user-defined bridge networks

NAT on IPv6, are you insane?

First of all, thank you for questioning my sanity! I'm aware NAT on IPv6 is almost always a no-go, since the huge number of available addresses removes the need for it. However, considering all the above issues related to IPv6 are fixed with IPv6 NAT, I thought: why not? The concepts of working with Docker images/containers rely heavily on IPv4 NAT, so if this makes IPv6 with Docker usable in the same way, be happy. I'm in no way "pro IPv6 NAT" in the general case; I'm just "pro working shit".

Probably IPv6 NAT will never make it into Docker, just because it's not "the right way". This is fine; when a better alternative is found, I'd be happy to use it and get rid of this repo. However, since the IPv6 support just isn't there yet, and discussions are still ongoing, this repo can be used in the meantime.

Still think IPv6 NAT is a bad idea? That's fine, you're absolutely free to NOT use this repo.

Usage

Docker Container

The recommended way is to run the Docker image:

docker run -d --name ipv6nat --privileged --network host --restart unless-stopped -v /var/run/docker.sock:/var/run/docker.sock:ro -v /lib/modules:/lib/modules:ro robbertkl/ipv6nat

The flags --privileged and --network host are necessary because docker-ipv6nat manages the hosts IPv6 firewall using ip6tables.

To limit runtime privileges as a security precaution, the --privileged flag can be replaced with --cap-add NET_ADMIN --cap-add SYS_MODULE.

If you're a security fan (it's not bad), you can drop all capabilities --cap-drop ALL and leave only --cap-add NET_ADMIN --cap-add NET_RAW --cap-add SYS_MODULE. About it you can read in a good article from RedHat.

You may omit the -v /lib/modules:/lib/modules:ro bind mount and --cap-add SYS_MODULE if your distro already loaded ip6_tables kernel module on boot. If you have a systemd-based distro, you can ensure that on next boot by echo ip6_tables >/etc/modules-load.d/ipv6nat.conf, see modules-load.d(5).

By default, the docker-ipv6nat command runs with the --retry flag. See the "Usage Flags" section below for more options. Add them to the end of the docker run command.

AUR Package

If you are running ArchLinux, you can install the latest version by getting the package from the AUR.

trizen -S docker-ipv6nat

For running docker-ipv6nat on system starup, you can simply enable (and start) the systemd service:

systemctl enable docker-ipv6nat.service
systemctl start docker-ipv6nat.service

Standalone Binary

Alternatively, you can download the latest release from the release page and run it directly on your host. See below for usage flags.

Usage Flags

Output from docker-ipv6nat --help:

Usage: docker-ipv6 [options]

Automatically configure IPv6 NAT for running docker containers

Options:
  -cleanup
    	remove rules when shutting down
  -debug
    	log ruleset changes to stdout
  -retry
    	keep retrying to reconnect after a disconnect
  -version
    	show version

Environment Variables:
  DOCKER_HOST - default value for -endpoint
  DOCKER_CERT_PATH - directory path containing key.pem, cert.pem and ca.pem
  DOCKER_TLS_VERIFY - enable client TLS verification

Docker IPv6 configuration

Instructions below show ways to enable IPv6 and are not specific to docker-ipv6nat. Just make sure to use a ULA range in order for docker-ipv6nat to pick them up.

Option A: default bridge network

To use IPv6, make sure your Docker daemon is started with --ipv6 and specifies a ULA range with --fixed-cidr-v6 (e.g. --fixed-cidr-v6 fd00:dead:beef::/48).

Option B: user-defined network

To try it out without messing with your Docker daemon flags, or if you're already using user-defined networks, you can create a IPv6-enabled network with:

docker network create --ipv6 --subnet fd00:dead:beef::/48 mynetwork

Then start all of your other containers with --network mynetwork. Please note the robbertkl/ipv6nat container still needs to run with --network host to access the host firewall.

Docker-ipv6nat respects all supported com.docker.network.bridge.* options (pass them with -o) and adds 1 additional option:

• com.docker.network.bridge.host_binding_ipv6: Default IPv6 address when binding container ports (do not include subnet/prefixlen; defaults to ::, i.e. all IPv6 addresses)

Please note this option can only be set on user-defined networks, as the default bridge network is controlled by the Docker daemon.

Swarm mode support

As mentioned above, docker-ipv6nat ip6tables changes affects only bridge type networks, so overlay networks are out of the window. Despite of that fact, in order to NAT outgoing traffic from a container to the outside world we can use the swarm docker_gwbridge which is a bridge network that every container in your swarm will get a 'leg' in.

When you run docker swarm init a default bridge for the swarm is created named docker_gwbridge which is equivalent to docker0 for standalone docker engines. the thing is that it's configured by default to prevent ip_forwarding

So the workaround is to create the docker_gwbridge with ip_forwarding enabled before you run the docker swarm init in this way we managed to access the outside world components and yet keep the container within the swarm overlay network to enjoy all the benefits of swarm.

Example:

docker network create \
 --ipv6 \
 --subnet 172.20.0.0/20 \
 --gateway 172.20.0.1 \
 --gateway fd00:3984:3989::1 \
 --subnet fd00:3984:3989::/64 \
 --opt com.docker.network.bridge.name=docker_gwbridge \
 --opt com.docker.network.bridge.enable_icc=true \
 --opt com.docker.network.bridge.enable_ip_forwarding=true \
 --opt com.docker.network.bridge.enable_ip_masquerade=true \
 docker_gwbridge

Troubleshooting

If you can see the added ip6tables rules, but it's still not working, it might be that forwarding is not enabled for IPv6. This is usually the case if you're using router advertisements (e.g. having net.ipv6.conf.eth0.accept_ra=1). Enabling forwarding in such a case will break router advertisements. To overcome this, use the following in your /etc/sysctl.conf:

net.ipv6.conf.eth0.accept_ra = 2
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1

The special value of 2 will allow accepting router advertisements even if forwarding is enabled.

Setting the -debug flag for docker-ipv6nat will log all ruleset changes to stdout so you can check your logs how docker-ipv6nat is modifing your ip6tables rulesets.

Authors

• Robbert Klarenbeek, [email protected]

Big thanks to all GitHub contributors for testing, reporting issues and PRs!

Special thanks to Elias Werberich @bephinix for his many contributions, mainly keeping up with recent docker / libnetwork changes and porting them to docker-ipv6nat!

License

This repo is published under the MIT License.