Managing Docker Instances with Puppet

Changing a Linux Hostname

When changing the hostname on a Linux system, it's important to understand that the sudo utility will complain loudly and often if a number of information sources don't agree on the the current hostname. In particular, on an Ubuntu system, the following should all agree:

  1. The hostname stored in /etc/hostname.

  2. The hostname defined for 127.0.1.1 in /etc/hosts.

  3. The hostname reported by /bin/hostname.

If they don't all match, you may see errors such as:


> sudo: unable to resolve host quux

And in extreme cases, you even may lose the ability to run the sudo command. It's best to avoid the situation by ensuring that you update all three data sources to the same value when changing your hostname.

In order to avoid errors with the sudo command, you actually need to change the hostname of your virtual machine in several places. In addition, the hostname reported by the PS1 prompt will not be updated until you start a new shell. The following commands, when run inside the Ubuntu guest, will make the necessary changes:


# Must be exported to use in sudo's environment.
export new_hostname="foo-alpine33"

# Preserve the environment or sudo will lose the
# exported variable. Also, we must explicitly
# execute on localhost rather than relying on
# whatever sudo thinks the current hostname is to
# avoid "sudo: unable to resolve host" errors.
sudo \
    --preserve-env \
    --host=localhost \
    -- \
    sed --in-place \
        "s/${HOSTNAME}/${new_hostname}/g" \
        /etc/hostname /etc/hosts
sudo \
    --preserve-env \
    --host=localhost \
    -- \
    hostname "$new_hostname"

# Replace the current shell in order to pick up the
# new hostname in the PS1 prompt.
exec "$SHELL"

Your prompt now should show that the hostname has changed. When you re-run the Puppet manifest, it will match the node list because you've defined a rule for hosts that include "alpine33" in the hostname. Puppet then will apply role::alpine33 for you, simply because the hostname matches the node definition! For example:


# Apply the manifest from inside the Ubuntu guest.
sudo puppet apply \
    --modulepath ~/.puppet/modules \
    /vagrant/roles_and_profiles.pp

# Verify that the role has been correctly applied.
docker images alpine

REPOSITORY   TAG       IMAGE ID        CREATED         SIZE
alpine       3.3       6c2aa2137d97    7 weeks ago     4.805MB

Ignore "update_docker_image.sh" Errors

When running the Puppet manifest in the example, you may see several errors that contain the following substring:


> update_docker_image.sh alpine:3.4 returned 3 instead of one of [0,1]

These errors currently are caused by upstream bugs in the Puppet Docker modules used in the examples. Bugs have been filed upstream, but can safely be ignored for the immediate purposes of this article. Despite the reported error, the Docker images actually still are being properly installed, which you can verify yourself inside the virtual machine with docker images alpine.

If you want to track the progress of these bugs, please see:

To apply this role to an entire cluster of machines, all you need to do is ensure they have hostnames that match your defined criteria. For example, say you have five hosts with the following names:

  1. foo-alpine33

  2. bar-alpine33

  3. baz-alpine33

  4. abc-alpine33

  5. xyz-alpine33

Then, the single node definition for /alpine33/ would apply to all of them, because the regular expression matches each of their hostnames. By assigning roles to patterns of hostnames, you can configure large segments of your data center simply by setting the proper hostnames! What could be easier?

Reassigning Roles at Runtime

Well, now you have a way to assign a role to thousands of boxes at a time. That's impressive all by itself, but the magic doesn't stop there. What if you need to reassign a system to a different role?

Imagine that you have a box with the Alpine 3.3 image installed, and you want to upgrade that box so it hosts the Alpine 3.4 image instead. In reality, hosting multiple images isn't a problem, and these images aren't mutually exclusive. However, it's illustrative to show how you can use Puppet to add, remove, update and replace images and containers.

Given the existing node definitions, all you need to do is update the hostname to include "alpine34" and let Puppet pick up the new role:


# Define a new hostname that includes "alpine34"
# instead of "alpine33".
export new_hostname="foo-alpine34"

sudo \
    --preserve-env \
    --host=localhost \
    -- \
    sed --in-place \
        "s/${HOSTNAME}/${new_hostname}/g" \
        /etc/hostname /etc/hosts
sudo \
    --preserve-env \
    --host=localhost \
    -- \
    hostname "$new_hostname"
exec "$SHELL"

# Rerun the manifest using the new node name.
sudo puppet apply \
    --modulepath ~/.puppet/modules \
    /vagrant/roles_and_profiles.pp

# Show the Alpine images installed.
docker images alpine

REPOSITORY   TAG       IMAGE ID        CREATED         SIZE
alpine       3.4       baa5d63471ea    7 weeks ago     4.803MB

As you can see from the output, Puppet has removed the Alpine 3.3 image, and installed Alpine 3.4 instead! How did this happen? Let's break it down into steps:

  1. You renamed the host to include the substring "alpine34" in the hostname.

  2. Puppet matched the substring using a regular expression in its node definition list.

  3. Puppet applied the Alpine 3.4 role (role::alpine34) assigned to nodes that matched the "alpine34" substring.

  4. The Alpine 3.4 role called its component profiles (which are actually parameterized classes) using "present" and "absent" arguments to declare the intended state of each image.

  5. Puppet applied the image management declarations inside the Alpine 3.3 and Alpine 3.4 profiles (profile::alpine33 and profile::alpine34, respectively) to install or remove each image.

Although hostname-based role assignment is just one of the many ways to manage the configuration of multiple systems, it's a very powerful one, and certainly one of the easiest to demonstrate. Puppet supports a large number of ways to specify what configurations should apply to a given host. The ability to configure systems dynamically based on discoverable criteria makes Puppet a wonderful complement to Docker's versioned images and containerization.

Other Puppet Options for Node Assignment

Puppet can assign roles, profiles and classes to nodes in a number of ways, including the following:

  • Classifying nodes with the Puppet Enterprise Console.

  • Defining nodes in the main site manifest—for example, site.pp.

  • Implementing an External Node Classifier (ENC), which is an external tool that replaces or supplements the main site manifest.

  • Storing hierarchical data in a Hiera YAML configuration file.

  • Using Puppet Lookup, which merges Hiera information with environment and module data.

  • Crafting conditional configurations based on facts known to the server or client at runtime.

Each option represents a set of trade-offs in expressive power, hierarchical inheritance and maintainability. A thorough discussion of these trade-offs is outside the scope of this article. Nevertheless, it's important to understand that Puppet gives you a great deal of flexibility in how you classify and manage nodes at scale. This article focuses on the common use case of name-based classification, but there are certainly other valid approaches.

Conclusion

In this article, I took a close look at managing Docker images and containers with docker::image and docker::run, but the Puppet Docker module supports a lot more features that I didn't have room to cover this time around. Some of those additional features include:

  • Building images from a Dockerfile with the docker::image class.

  • Managing Docker networks with the docker::networks class.

  • Using Docker Compose with the docker::compose class.

  • Implementing private image registries using the docker::registry class.

  • Running arbitrary commands inside containers with the docker::exec class.

When taken together, this powerful collection of features allows you to compose extremely powerful roles and profiles for managing Docker instances across infrastructure of almost any scale. In addition, by leveraging Puppet's declarative syntax and its ability to automate role assignment, it's possible to add, remove and modify your Docker instances on multiple hosts without having to manage each instance directly, which is typically a huge win in enterprise automation. And finally, the standardization and repeatability of Puppet-driven container management makes systems more reliable when compared to hand-tuned, hand-crafted nodes that can "drift" from the ideal state over time.

In short, Docker provides a powerful tool for creating lightweight golden images and containerized services, while Puppet provides the means to orchestrate those images and containers in the cloud or data center. Like strawberries and chocolate, neither is "better" than the other; combine them though, and you get something greater than the sum of its parts.

Resources

Key Files from This Article, Available on GitHub: https://github.com/CodeGnome/MDIWP-Examples

Docker: https://www.docker.co

Puppet Home Page (Docs and Commercial Versions): https://puppet.com

Puppet Ruby Gem (Open-Source Version): https://rubygems.org/gems/puppet

Puppet Labs docker_platform Module: https://forge.puppet.com/puppetlabs/docker_platform

The garethr-docker Module Wrapped by docker_platform: https://github.com/garethr/garethr-docker

Official Apache HTTP Server Docker Images: https://hub.docker.com/_/http

Oracle VirtualBox: https://www.virtualbox.org

Vagrant by HashiCorp: https://www.vagrantup.com

Ubuntu Images on HashiCorp Atlas: https://atlas.hashicorp.com/ubuntu

Puppet Documentation on the "Roles and Profiles" Pattern: https://docs.puppet.com/pe/2016.4/r_n_p_intro.html

______________________

Todd A. Jacobs is a veteran IT consultant with a passion for all things Linux. He spends entirely too much time making systems do things they were never designed to do.