Paranoid Penguin - Managing SSH for Scripts and cron Jobs

Using insecure protocols leaves your data and connected machines vulnerable to attack. Remote server management requirements demand that security be given a top priority. This article explains my process for using OpenSSH in unattended scripts and cron jobs.

Most readers are familiar with the secure shell (SSH) protocol, which creates a secure tunnel for commands, data and passwords to travel across the network. Recently, my workplace was faced with some challenges in securely setting up scripts through cron. We use SSH because it resolves the major problems with rsh; rsh sends clear text over the network and has weak host-based authentication. But our challenge became how to deal with password prompts when using SSH in unattended jobs. For example, we run df -k, top and swapon -s to get remote server statistics and alert our team if problems exist.

If you also still are using rsh, the SSH client, ssh, makes a perfect replacement in scripts. Modifications typically are minor. For example:

for host in $servers
do
rsh $host df -k
done

simply becomes:

for host in $servers
do
ssh $host df -k
done

SSH supports a wide range of authentication systems, the most common being Kerberos, Rhosts, Public-key and Password. Because we didn't have a Kerberos infrastructure in place, our readily available options to solve this problem were to echo the password in the script, use Rhosts, use ssh-agent or use public keys.

Our first options presented some challenges and weaknesses. First, simply echoing the password in a script is not a simple task, as SSH does not read from standard input. To make it do so would require advanced scripting techniques. More important, you would need to put your password either in the script itself or in a file on the filesystem. Although you could set proper permissions, getting access to the password would be a relatively easy task for a determined intruder. It could be as simple as restoring data from a backup or even viewing the password on-screen. This method was not an option for us.

Second, we considered host-based authentication, which is how users executing the rsh command are granted access. Because users are granted or denied access based on the host they are logging in from, no password is needed. This solution may work in some situations where security concerns are light, but the ability to pretend to be another host, to IP spoof and to disrupt DNS does exist. Also, due to the fact that once a host has been impersonated successfully, all users have been compromised on the remote host, we decided we needed something more secure.

Our third option was to use ssh-agent. Before we discuss this option here, though, we need to cover public keys and their use. Instead of using a plain-text password, SSH has the ability to use public key cryptography. This means that when a client connects to a server, it has a conversation with the server and proves its identity based on advanced mathematical computations.

Let's walk through the setup to generate a set of public and private SSH keys to allow a user named scripts to log in from hostA to hostB, assuming the user exists on both hosts:

1) Generate the keys:

[scripts@hostA]$ssh-keygen  -t dsa
Generating public/private dsa key pair.

Enter file in which to save the key (/home/scripts/.ssh/id_dsa):
Created directory '/home/scripts/.ssh'.
Enter passphrase (empty for no passphrase): XXXX
Enter same passphrase again: XXXX
Your identification has been saved in /home/scripts/.ssh/id_dsa.
Your public key has been saved in /home/scripts/.ssh/id_dsa.pub.
The key fingerprint is:
41:03:aa:dc:cc:b9:39:50:65:bc:ee:7b:36:d2:64:7a scripts@hostA

2) Copy public key to hostB from hostA:

scp /home/scripts/.ssh/id_dsa.pub \
hostB:/home/scripts/.ssh/authorized_keys

scp is an encrypted replacement for rcp and simply copies files in a secured manner.

The authorized_keys file is a file that contains the public identities, or public keys, of users who can log in to your account by using public key authentication. All users maintain their own authorized_keys file, which typically lives in the hidden .ssh directory in a user's home directory. Users also may configure security restrictions to public keys here as well, which we review below.

The authorized_keys file is not created when you first run ssh-keygen to create your public and private keys. As a best practice, we recommend permissions of 600 for this file.

At this point, userA should be able to log in to hostB without a password using public key technology. Now, of course, we still have the same problem with echoing the passphrase into a script. As I mentioned, SSH does not take input from standard input, so this represents the same scripting challenge as before. To eliminate the need to retype your password continually, SSH comes with ssh-agent. You use ssh-agent as follows, in combination with ssh-add:

[scripts@hostA scripts]$ ssh-agent bash
[scripts@hostA .ssh]$ ssh-add  id_dsa
Identity added: id_dsa (id_dsa)

We pass our shell to ssh-agent, and it inherits the keys we add with ssh-add. Now we need only type our passphrase once and we can use our key default key, id_dsa.pub, to be authenticated. An important note about using multiple keys in an interactive session with SSH is how you need to call SSH. For example, if you have created three private keys—your default key, id_dsa, and two other keys called backup and monitor to use for different tasks—you simply would call SSH with the -i parameter. This is done to make sure you're using your new key while logging in to the remote SSH server:

[scripts@hostA]$ssh -i backupkey hostB

When you are using ssh-agent, you may believe you simply type in ssh -i backup to use your backup identity. This is not quite the case, though, as the ssh-agent typically uses the key that is on the top of its key list. To get a listing of all the keys you have loaded in ssh-agent, run ssh-add -l for a listing of fingerprints of all identities currently loaded in the agent:

[scripts@hostA scripts]$ssh-add -l
1024 df:ab:8e:d7:e4:bd:35:f6:b3:2e:76:6b:dd:71:2f:fe monitor (DSA)
1024 4e:4c:00:ba:1e:5d:60:08:f2:b8:2e:d4:59:1e:ff:2f id_dsa (DSA)
1024 0a:72:24:9e:0a:cd:e2:e4:5f:93:cb:ac:66:78:03:f6 backup (DSA)

Because ssh-agent typically favors the key listed first, it favors the monitor key. To be able to use the backup key, you need to unset the shell variable SSH_AUTH_SOCK and then point SSH to the identity you want to use, as follows:

[scripts@hostA scripts]env -u SSH_AUTH_SOCK \
  ssh -i backup  hostB