Managing Docker Instances with Puppet

Controlling Docker with Puppet Roles and Profiles

It may seem like a lot of work to automate the configuration of a single machine. However, even when dealing with only a single machine, the consistency and repeatability of a managed configuration is a big win. In addition, this work lays the foundation for automating an unlimited number of machines, which is essential for scaling configuration management to hundreds or thousands of servers. Puppet makes this possible through the "roles and profiles" workflow.

In the Puppet world, roles and profiles are just special cases of Puppet manifests. It's a way to express the desired configuration through composition, where profiles are composed of component modules and then one or more profiles comprise a role. Roles are then assigned to nodes dynamically or statically, often through a site.pp file or an External Node Classifier (ENC).

Let's walk through a simplified example of what a roles-and-profiles workflow looks like. First, you'll create a new manifest in the same directory as your Vagrantfile named roles_and_profiles.pp. Listing 4 shows a useful example.

Listing 4. roles_and_profiles.pp


####################################################
# Profiles
####################################################
# The "dockerd" profile uses a forge module to
# install and manage the Docker daemon. The only
# difference between this and the "docker" class
# from the earlier docker.pp example is that we're
# wrapping it inside a profile.
class profile::dockerd {
    class { 'docker':
      package_name => 'docker.io',
      docker_users => ['ubuntu'],
    }
}

# The "alpine33" profile manages the presence or
# absence of the Alpine 3.3 Docker image using a
# parameterized class. By default, it will remove
# the image.
class profile::alpine33 ($status = 'absent') {
    docker::image { 'alpine_33':
        image     => 'alpine',
        image_tag => '3.3',
        ensure    => $status,
    }
}


# The "alpine34" profile manages the presence or
# absence of the Alpine 3.4 Docker image. By
# default, it will remove the image.
class profile::alpine34 ($status = 'absent') {
    docker::image { 'alpine_34':
        image     => 'alpine',
        image_tag => '3.4',
        ensure    => $status,
    }
}

####################################################
# Roles
####################################################
# This role combines two profiles, passing
# parameters to add or remove the specified images.
# This particular profile ensures the Alpine 3.3
# image is installed, and removes Alpine 3.4 if
# present.
class role::alpine33 {
    class { 'profile::alpine33':
        status => 'present',
    }

    class { 'profile::alpine34':
        status => 'absent',
    }
}

# This role is the inverse of role::alpine33. It
# calls the same parameterized profiles, but
# installs Alpine 3.4 and removes Alpine 3.3.
class role::alpine34 {
    class { 'profile::alpine33':
        status => 'absent',
    }

    class { 'profile::alpine34':
        status => 'present',
    }
}

####################################################
# Nodes
####################################################
# Apply role::alpine33 to any host with "alpine33"
# in its hostname.
node /alpine33/ {
    include ::role::alpine33
}

# Apply role::alpine34 to any host with "alpine34"
# in its hostname.
node /alpine34/ {
    include ::role::alpine34
}

Note that all the profiles, roles and nodes are placed into a single Puppet manifest. On a production system, those should all be separate manifests located in appropriate locations on the Puppet Master. Although this example is illustrative and extremely useful for working with masterless Puppet, be aware that a few rules are broken here for the sake of convenience.

Let me briefly discuss each section of the manifest. Profiles are the reusable building blocks of a well organized Puppet environment. Each profile should have exactly one responsibility, although you can allow the profile to take optional arguments that make it more flexible. In this case, the Alpine profiles allow you to add or remove a given Docker image depending on the value of the $status variable you pass in as an argument.

A role is the "reason for being" that you're assigning to a node. A node can have more than one role at a time, but each role should describe a singular purpose regardless of how many component parts are needed to implement that purpose. In the wild, some common roles assigned to a node might include:

  • role::ruby_on_rails

  • role::jenkins_ci

  • role::monitored_host

  • role::bastion_host

Each role is composed of one or more profiles, which together describe the purpose or function of the node as a whole. For this example, you define the alpine34 role as the presence of the Docker dæmon with Alpine 3.4 and the absence of an Alpine 3.3 image, but you could just as easily have described a more complex role composed of profiles for NTP, SSH, Ruby on Rails, Java and a Splunk forwarder.

This separation of concerns is borrowed from object-oriented programming, where you try to define nodes through composition in order to isolate the implementation details from the user-visible behavior. A less programmatic way to think of this is that profiles generally describe the features of a node, such as its packages, files or services, while roles describe the node's function within your data center.

Nodes, which are generally defined in a Puppet Master's site.pp file or an external node classifier, are where roles are statically or dynamically assigned to each node. This is where the real scaling power of Puppet becomes obvious. In this example, you define two different types of nodes. Each node definition uses a string or regular expression that is matched against the hostname (or certname in a client/server configuration) to determine what roles should be applied to that node.

In the node section of the example manifest, you tell Puppet to assign role::alpine33 to any node that includes "alpine33" as part of its hostname. Likewise, any node that includes "alpine34" in the hostname gets role::alpine34 instead. Using pattern-matching in this way means that you could have any number of hosts in your data center, and each will pick up the correct configuration based on the hostname that it's been assigned. For example, say you have five hosts with the following names:

  1. foo-alpine33

  2. bar-alpine33

  3. baz-alpine33

  4. abc-alpine34

  5. xyz-alpine34

Then, the first three will pick up the Alpine 3.3 role when they contact the Puppet Master, and the last two will pick up the Alpine 3.4 role instead. This is almost magical in its simplicity. Let's see how this type of dynamic role assignment works in practice.

Dynamic Role Assignments

Assuming that you've already placed roles_and_profiles.pp into the directory containing your Vagrantfile, you're able to access the manifest within the Ubuntu virtual machine. Let's log in to the VM and test it out (Listing 5).

Listing 5. Logging in to the Ubuntu Virtual Machine


# Ensure we're in the right directory on our Vagrant
# host.
cd ~/Documents/puppet-docker

# Ensure that the virtual machine is active. There's
# no harm in running this command multiple times,
# even if the machine is already up.
vagrant up

# Login to the Ubuntu guest.
vagrant ssh

Next, run the roles_and_profiles.pp Puppet manifest to see what happens. Hint: it's going to fail, and then you're going to explore why that's a good thing. Here's what happens:


ubuntu@ubuntu-xenial:~$ sudo puppet apply --modulepath
 ↪~/.puppet/modules /vagrant/roles_and_profiles.pp
Error: Could not find default node or by name with
 ↪'ubuntu-xenial.localdomain, ubuntu-xenial' on node
 ↪ubuntu-xenial.localdomain
Error: Could not find default node or by name with
↪'ubuntu-xenial.localdomain, ubuntu-xenial' on node
 ↪ubuntu-xenial.localdomain

Why did the manifest fail to apply? There are actually several reasons for this. The first reason is that you did not define any nodes that matched the current hostname of "ubuntu-xenial". The second reason is that you did not define a default to be applied when no other match is found. Puppet allows you to define a default, but in many cases, it's better to raise an error than to get a configuration you weren't expecting.

In this test environment, you want to show that Puppet is able to assign roles dynamically based on the hostname of the node where the Puppet agent is running. With that in mind, let's modify the hostname of the Ubuntu guest to see how a site manifest can be used to configure large clusters of machines appropriately based solely on each machine's hostname.

______________________

Todd A. Jacobs is a veteran IT consultant with a passion for all things Linux. He spends entirely too much time making systems do things they were never designed to do.