Concerning Containers' Connections: on Docker Networking

Containers can be considered the third wave in service provision after physical boxes (the first wave) and virtual machines (the second wave). Instead of working with complete servers (hardware or virtual), you have virtual operating systems, which are far more lightweight. Instead of carrying around complete environments, you just move applications, with their configuration, from one server to another, where it will consume its resources, without any virtual layers. Shipping over projects from development to operations also is simplified—another boon. Of course, you'll face new and different challenges, as with any technology, but the possible risks and problems don't seem to be insurmountable, and the final rewards appear to be great.

Docker is an open-source project based on Linux containers that is showing high rates of adoption. Docker's first release was only a couple years ago, so the technology isn't yet considered mature, but it shows much promise. The combination of lower costs, simpler deployment and faster start times certainly helps.

In this article, I go over some details of setting up a system based on several independent containers, each providing a distinct, separate role, and I explain some aspects of the underlying network configuration. You can't think about production deployment without being aware of how connections are made, how ports are used and how bridges and routing are set up, so I examine those points as well, while putting a simple Web database query application in place.

Basic Container Networking

Let's start by considering how Docker configures network aspects. When the Docker service dæmon starts, it configures a virtual bridge, docker0, on the host system (Figure 1). Docker picks a subnet not in use on the host and assigns a free IP address to the bridge. The first try is, but that could be different if there are conflicts. This virtual bridge handles all host-containers communications.

When Docker starts a container, by default, it creates a virtual interface on the host with a unique name, such as veth220960a, and an address within the same subnet. This new interface will be connected to the eth0 interface on the container itself. In order to allow connections, iptables rules are added, using a DOCKER-named chain. Network address translation (NAT) is used to forward traffic to external hosts, and the host machine must be set up to forward IP packets.

Figure 1. Docker uses a bridge to connect all containers on the same host to the local network.

The standard way to connect a container is in "bridged" mode, as described previously. However, for special cases, there are more ways to do this, which depend on the -net option for the docker run command. Here's a list of all available modes:

  • -net=bridge — The new container uses a bridge to connect to the rest of the network. Only its exported public ports will be accessible from the outside.

  • -net=container:ANOTHER.ONE — The new container will use the network stack of a previously defined container. It will share its IP address and port numbers.

  • -net=host — This is a dangerous option. Docker won't separate the container's network from the host's. The new container will have full access to the host's network stack. This can cause problems and security risks!

  • -net=none — Docker won't configure the container network at all. If you want, you can set up your own iptables rules (see Resources if you're interested in this). Even without the network, the container could contact the world by shared directories, for example.

Docker also sets up each container so it will have DNS resolution information. Run findmnt inside a container to produce something along the lines of Listing 1. By default, Docker uses the host's /etc/resolv.conf data for DNS resolution. You can use different nameservers and search lists with the --dns and --dns-search options.

Listing 1. The last three lines show Docker's special mount trick, so containers get information from Docker-managed host files.

root@4de393bdbd36:/var/www/html# findmnt -o TARGET,SOURCE
TARGET                  SOURCE
/                       /dev/mapper/docker-8:2-25824189-4de...822[/rootfs]
|-/proc                 proc
| |-/proc/sys           proc[/sys]
| |-/proc/sysrq-trigger proc[/sysrq-trigger]
| |-/proc/irq           proc[/irq]
| |-/proc/bus           proc[/bus]
| `-/proc/kcore         tmpfs[/null]
|-/dev                  tmpfs
| |-/dev/shm            shm
| |-/dev/mqueue         mqueue
| |-/dev/pts            devpts
| `-/dev/console        devpts[/2]
|-/sys                  sysfs
|-/etc/resolv.conf      /dev/sda2[/var/lib/docker/containers/4de...822/resolv.conf]
|-/etc/hostname         /dev/sda2[/var/lib/docker/containers/4de...822/hostname]
`-/etc/hosts            /dev/sda2[/var/lib/docker/containers/4de...822/hosts]

Now that you have an idea about how Docker sets up networking for individual containers, let's develop a small system that will be deployed via containers and then finish by working out how to connect all the pieces together.