Using Hiera with Puppet

Perhaps Hiera's most powerful feature is the ability to pull data from a variety of back-end storage technologies. Hiera back ends are too numerous to list, but they include JSON, Redis, MongoDB and even HTTP to create a URL-driven Puppet value API. Let's take a look at two useful back ends: Postgres and hiera-eyaml.

To start with the psql back end, you need to install the hiera-psql gem on your Puppet master (or each node if you're using masterless Puppet runs with Puppet apply), with a simple hiera.yaml file of:

* 'environment/%{env}'
* default 
* psql 
    :dbname: hiera 
    :host: localhost 
    :user: root 
    :password: password

You can do lookups on a local Postgres installation with a single database called hiera with a single table called config with three columns: Path, Key and Value.

path                   key                               value

'environment/prod'   'nginx::credentials::basic_auth'   'password'

This is extremely useful if you want to expose your Hiera data to custom in-house applications outside Puppet, or if you want to create a DevOps Web console or reports.

Storing credentials in Puppet modules is a bad idea. If you store credentials in Puppet and your manifests on an external code repository, you're not only unable to share those manifests with developers with less-secure access, but you're obviously exposing vital security data outside the organization, and possibly in violation of various types of compliance. So how do you encrypt sensitive data in Puppet while keeping your manifests relevant and sharable? The answer is with hiera-eyaml.

Tom Poulton created hiera-eyaml to allow engineers to do just that: encrypt only the sensitive string of data inside the actual file rather than encrypting the entire file, which also can be done with hiera-gpg (a very useful encryption gem but not covered in this article).

To get started, install the hiera-eyaml gem, and generate a keypair on the Puppet master:

$ eyaml createkeys

Then move the keys to a secure location, like /etc/puppet/secure/keys. Your hiera.yaml configuration should look something like this:

  - eyaml
  - yaml
  :datadir: /etc/puppet/hieradata
  :datadir: /etc/puppet/hieradata
  :extension: 'yaml'          # <- so all files can be named .yaml
  :pkcs7_private_key: /path/to/private_key.pkcs7.pem
  :pkcs7_public_key: /path/to/public_key.pkcs7.pem
  - "node/%{::fqdn}"
  - "environment/%{::env}/main"
  - "environment/%{::env}/%{calling_module}"
* defaults

To encrypt values, you need only the public key, so distribute it to anyone who needs to create encrypted values:

$ eyaml encrypt -s 'password'

This will generate an encrypted block that you can add as the value in any yaml file:

nginx::credentials::user: slackey       #cleartext example value
nginx::credentials::basic_auth : >      #encrypted example value