Flat File Encryption with OpenSSL and GPG

The Pretty Good Privacy (PGP) application, which has long been known as a primary tool for file encryption, commonly focused on email. It has management tools for exchanging credentials with peers and creating secure communication channels over untrusted networks. GNU Privacy Guard (GPG) has carried on this legacy with a free and open implementation included in most major Linux distributions. PGP/GPG has proven highly resistant to cryptographic attack and is a preeminent tool for secure communications.

OpenSSL is more known for network security, but it also has tools useful for most aspects of encrypting flat files. Although using OpenSSL requires more knowledge of specific algorithms and methods, it can be more flexible in a number of scenarios than other approaches. OpenSSH keys can be used transparently for flat file encryption with OpenSSL, allowing user and/or host SSH keys to pervade any number of unrelated services.

OpenSSL is also useful for illustrating the sequence of encryption techniques that create secure channels. This knowledge is applicable in many other situations, so the material is worth study even if there is no immediate need for the tools.

OpenSSL Flat File Processing

Many common programs in UNIX have implementations within the OpenSSL command-line utility. These include digest/checksum tools (md5sum, sha1sum, sha256sum), "ASCII-Armor" tools (base64/uuencode/uudecode), "safe" random number generation and MIME functions in addition to a suite of cipher and key management utilities. Because OpenSSL often is found on non-UNIX platforms, those utilities can provide a familiar interface on unfamiliar systems for UNIX administrators.

Let's begin with a complete script for flat file encryption with OpenSSL, using asymmetric exchange of a session key, SHA-256 digest checksums and the use of a symmetric cipher. This entire exchange, both to encode and decode, is presented in the following text for the Korn shell (GNU Bash also may be used with no required changes):


set -euo pipefail
IFS=$'\n\t' #http://redsymbol.net/articles/unofficial-bash-strict-mode/

# openssl genrsa -aes256 -out ~/.prv.key 2868        # Make private key
# openssl rsa -in ~/.prv.key -pubout -out ~/.pub.key # Make public key

PVK=~/.prv.key; PBK=~/.pub.key
SKEY=$(mktemp -t crypter-session_key-XXXXXX)          # Symmetric key

case $(basename "${0}") in

encrypter) ####sender needs only public key - not .pas or .prv.key#####
 openssl rand -base64 48 -out "${SKEY}"              # Generate sesskey
 openssl rsautl -encrypt -pubin -inkey "${PBK}" -in "${SKEY}" |
  openssl base64;
 echo __:
 for F                                                # Generate digest
 do echo $(openssl dgst -sha256 "${F}" | sed 's/^[^ ]*[ ]//') "${F}"
 done | openssl enc -aes-128-cbc -salt -a -e -pass "file:${SKEY}"
 echo __:
 for F                                                # Encrypt files
 do openssl enc -aes-128-cbc -salt -a -e -pass "file:${SKEY}" -in "${F}"
    echo __:
 done ;;

decrypter) #####receiver###############################################
 TMP=$(mktemp -t crypter-tmp-XXXXXX); PW=${HOME}/.pas; unset IFS
 DGST=$(mktemp -t crypter-dgst-XXXXXX); #cd ${HOME}/dest #unpack dest
 while read Z
 do if [[ ${Z%%:*} == '__' ]]
    then if [[ -s "${SKEY}" ]]
         then if [[ -s "${DGST}" ]]
              then openssl enc -aes-128-cbc -d -salt -a -in "${TMP}" \
                    -pass "file:${SKEY}" -out "${NAME[I]}"
                   ((I+=1))                           # Decrypt files
              else openssl enc -aes-128-cbc -d -salt -a -in "${TMP}" \
                    -pass "file:${SKEY}" -out "${DGST}"
                   date +%Y/%m/%d:%H:%M:%S
                   while read hash file
                   do echo "${hash} ${file}"
                      NAME[I]=$(basename "${file}")  # Unpack only @dest
                   done < "${DGST}"
         else openssl base64 -d -in "${TMP}" |       # Extract sesskey
               openssl rsautl -decrypt -inkey "${PVK}" \
                -passin "file:${PW}" -out "${SKEY}"

             #Older OpenSSL: decrypt PVK; c/sha256/sha1/; no strict
             #openssl rsa -in "${PVK}" -passin "file:${PW}" -out "$DGST"
             #openssl base64 -d -in "${TMP}" |        # Extract sesskey
             # openssl rsautl -decrypt -inkey "${DGST}" -out "${SKEY}"
             #> "${DGST}"
         > "${TMP}"                                   # Erase tempfile
    else echo "${Z}" >> ${TMP}
 while [[ ${I} -lt ${#NAME[*]} ]]                     # Verify digest
 do F=$(openssl dgst -sha256 "${NAME[I]}" | sed 's/^[^ ]*[ ]//')
    if [[ "${F}" = "${HASH[I]}" ]]
    then echo "${NAME[I]}: ok"; else echo "${NAME[I]}: **SHA CORRUPT**"
 rm "${TMP}" "${DGST}" ;;

rm "${SKEY}"

I will specifically cover everything above to the end of the encrypter case block, as this succinctly addresses the major cryptographic components of most encryption tools—that is, SSH, TLS, PGP and so on.

First, I include a well known strict mode for Korn/Bash published by Aaron Maxwell that can prevent coding errors, as documented at the URL near the top of the script.

Next, I generate an RSA private key. RSA, as an "asymmetric cipher", uses pairs of keys for communication and was developed by Ron Rivest, Adi Shamir and Leonard Adleman in 1977. Other asymmetric ciphers in common use are Diffie-Hellman key exchange and Elliptic Curve, but OpenSSL's support for RSA is more thorough, complete and widespread (a bug listed in OpenSSL's dhparam manual page indicates "There should be a way to generate and manipulate DH keys."). With an asymmetric cipher, content encrypted by one key can only be read in clear text by the other. You can use such keypairs not only to communicate securely, but also to prove authenticity. Below is an example of the generation of an RSA private key of a non-standard size of 2868 bits:

$ openssl genrsa -aes256 -out ~/.prv.key 2868
Generating RSA private key, 2868 bit long modulus
e is 65537 (0x10001)
Enter pass phrase for /home/ol7_user/.prv.key:
Verifying - Enter pass phrase for /home/ol7_user/.prv.key:

$ chmod 400 .prv.key

$ cat .prv.key
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,16846D1D37C82C834E65B518C456DE2F


I have chosen this non-standard key size based upon recommendations from the US National Institute of Standards (NIST). NIST uses a "general number field sieve" on page 92 of its implementation guidance document to determine minimum RSA key size. You can implement this formula with the GNU bc utility (part of GNU Coreutils):

$ cat keysize-NIST.bc
#!/usr/bin/bc -l
l = read()
scale = 14; a = 1/3; b = 2/3; t = l * l(2); m = l(t) # a^b == e(l(a) * b)
n = e( l(m) * b ); o = e( l(t) * a ); p = (1.923 * o * n - 4.69) / l(2)
print "Strength: ", p, "\n"

$ echo 2868 | ./keysize-NIST.bc
Strength: 128.01675571278223
$ echo 7295 | ./keysize-NIST.bc
Strength: 192.00346260354399
$ echo 14446 | ./keysize-NIST.bc
Strength: 256.00032964845911

$ echo 2048 | ./keysize-NIST.bc
Strength: 110.11760837749330
$ echo 2127 | ./keysize-NIST.bc
Strength: 112.01273358822347

In general, asymmetric ciphers are slower and weaker than "symmetric ciphers" (which are defined as using only one key to both encrypt and decrypt). Later I will be using a 128-bit symmetric cipher to communicate the bulk of my data, so I will use an RSA key of (strictly) comparable strength of 2868 bits. Note that many people feel that RSA key sizes over 2048 bits are a waste. Still, the most forward thinkers in cryptography conjecture that there may be "...some mathematical breakthrough that affects one or more public-key algorithms. There are a lot of mathematical tricks involved in public-key cryptanalysis, and absolutely no theory that provides any limits on how powerful those tricks can be....The fix is easy: increase the key lengths." In any case, I am strictly following NIST's recommended guidelines as I generate the key.

I have listed 192- and 256-bit equivalences because this symmetric cipher is not approved for "top secret" use at 128 bits. For highly sensitive information that must be kept secret, consider an RSA key size of 7295 or 14446 bits as (strictly) recommended by NIST's formula. Note that an RSA key size of 2048 bits computes to 110 bits of equivalent strength. This is below the requirement of RFC-7525 of a minimum of 112 bits of security (128 recommended)—2127-bit RSA keys satisfy this mandate.

A corresponding public key can be generated for use with the script:

$ openssl rsa -in ~/.prv.key -pubout -out ~/.pub.key
Enter pass phrase for /home/ol7_user/.prv.key:
writing RSA key

$ cat .pub.key
-----END PUBLIC KEY-----

The private key is compatible with the OpenSSH Protocol 2 RSA format, and you can generate what normally would be stored as the id_rsa.pub file with a simple keygen command:

$ ssh-keygen -y -f ~/.prv.key
Enter passphrase:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABZwqkByyctfGgfj553YGh/Yi6k4TBNh3P43lH7

An SSH server also runs with several types of host keys (which do not normally use a password), usually of 2048 bits in size. A host's private RSA key can be used with my crypter script by generating a compatible public key with this command:

# openssl rsa -in /etc/ssh/ssh_host_rsa_key -pubout
writing RSA key
-----END PUBLIC KEY-----

Since I will be using this RSA keypair for batch transfers, I will be recording the clear-text password for this key in the ~/.pas file. Because of this, the RSA key likely should not be used for SSH. OpenSSL is able to read passwords from a variety of other sources, so if you remove the ~/.pas file and supply the password from a more secure source, the use of a single RSA key for both SSH network sessions and OpenSSL flat file encryption becomes more of an option. Alternately, use a key without a password, and dispense with the ${PW} clauses above.

You cannot use the RSA keypair for the bulk of the encryption of a large amount of data, because RSA can encode only small amounts of information, as is detailed in the manual page:

$ man rsautl | col -b | awk '/NOTES/,/^$/'
  rsautl, because it uses the RSA algorithm directly, can only be used
  to sign or verify small pieces of data.

Echoing a string of zeros to a text file, the maximum size of the clear-text input for RSA encryption with my 2868-bit RSA key is 348 bytes:

$ for((x=0;x<348;x+=1));do echo -n 0 >> bar;done

$ ll bar
-rw-rw-r--. 1 ol7_user ol7_user 348 Jul  7 17:49 bar

$ openssl rsautl -encrypt -pubin -inkey ~/.pub.key -in bar |
        openssl base64

$ echo -n 0 >> bar

$ ll bar
-rw-rw-r--. 1 ol7_user ol7_user 349 Jul  7 17:49 bar

$ openssl rsautl -encrypt -pubin -inkey ~/.pub.key -in bar |
        openssl base64
RSA operation error
139936549824416:error:0406D06E:rsa routines:
RSA_padding_add_PKCS1_type_2:data too large for key size:

Similar testing with a 2048-bit RSA key yields a maximum of 245 bytes (slightly smaller than the size of the key).

Because of this limitation, I will generate a random 64-character password with OpenSSL's random number generator, encrypt a copy of it in the output with the public key, then use it for symmetric encryption for the bulk of the data as the "session key". For this example, I will use the following password obtained from OpenSSL in this manner:

$ openssl rand -base64 48

If I store this file as /tmp/skey, I can see the encryption take place:

$ openssl rsautl -encrypt -pubin -inkey ~/.pub.key -in /tmp/skey |
        openssl base64

Note above the call to the base64 function—the encrypted output is a binary file, and it cannot be displayed directly with the standard ASCII seven-bit character set. The base64 encoder changes binary data into a stream of printable characters suitable for transmission over seven-bit channels—email, for example. This performs the same function as uuencode, using different ASCII symbols.

If I record the output of the encryption in the /tmp/ekey file, I can decrypt it with the private key:

$ openssl base64 -d < /tmp/ekey |
        openssl rsautl -decrypt -inkey ~/.prv.key
Enter pass phrase for .prv.key:

Note above in the decryption section that very old versions of the OpenSSL rsautl command did not allow passwords to be specified on the command line. Therefore, an unencrypted copy of the key must be created before RSA decryption of the session key can take place. That procedure is documented in the comments for legacy systems and versions.

With the session key in hand, I next compute SHA-256 digest checksums of all the input files and record the encrypted results in the output. OpenSSL's version of the sha256sum utility differs slightly in formatting from the conventional version. Also included are SHA-1, RIPEMD-160 and MD5 checksums below:

$ sha256sum /etc/resolv.conf
04655aaa80ee78632d616c1...4bd61c70b7550eacd5d10e8961a70  /etc/resolv.conf

$ openssl dgst -sha256 /etc/resolv.conf
SHA256(/etc/resolv.conf)= 04655aaa80ee78632d6...1c70b7550eacd5d10e8961a70

$ openssl dgst -sha1 /etc/resolv.conf
SHA1(/etc/resolv.conf)= adffc1b0f9620b6709e299299d2ea98414adca2c
$ openssl dgst -ripemd160 /etc/resolv.conf
RIPEMD160(/etc/resolv.conf)= 9929f6385e3260e52ba8ef58a0000ad1261f4f31
$ openssl dgst -md5 /etc/resolv.conf
MD5(/etc/resolv.conf)= 6ce7764fb66a70f6414e9f56a7e1d15b

The SHA-family of digests were all created by the NSA, to whom we owe a great debt for their publication. The RIPEMD-160 digest was developed by researchers in Belgium and is an open alternative to SHA with no known flaws, but it is slower than SHA-1 and was released afterwards, so it is not used as often. MD5 digests should not be used beyond basic media error detection as they are vulnerable to tampering.

The script adjusts the format produced by OpenSSL to more closely mimic the standard utility, then uses the AES128-CBC symmetric cipher to code the digest for all the input files after printing a delimiter (__:). Very old versions of the OpenSSL utility might lack SHA-256—notes in the script detail downgrading to the weaker SHA-1 when using legacy systems (MD5 never should be used). The man dgst command will give full details on OpenSSL's digest options if the manual pages are available.

Finally, the script enters the main encryption loop where each file is processed with AES128-CBC, encoded with base64, separated by delimiters, then sent to STDOUT under the intention that the script be redirected/piped to a file or program for further processing. Information on OpenSSL's various symmetric ciphers can be found with the man enc command when the manual pages are accessibly installed. An informative and amusing cartoon has been published online covering AES' history and theory of operation, for those who have a deeper interest in our chosen symmetric cipher. The GPG website currently advocates Camellia and Twofish in addition to AES, and Camellia can be found in OpenSSL.

OpenSSL can be called to encrypt a file to the standard output with AES like so:

openssl enc -aes-128-cbc -salt -a -e -pass file:pw.txt
 ↪-in file.txt > file.aes

The encryption is undone like so:

openssl enc -aes-128-cbc -d -salt -a -pass file:pw.txt -in file.aes

Here is an example of a complete run of the script:

$ ln -s crypter.sh encrypter
$ ln -s crypter.sh decrypter
$ chmod 755 crypter.sh

$ ./encrypter /etc/resolv.conf /etc/hostname > foo

$ ./decrypter < foo
04655aaa80ee78632d616c1...4bd61c70b7550eacd5d10e8961a70 /etc/resolv.conf
4796631793e89e4d6b5b203...37a4168b139ecdaee6a4a55b03468 /etc/hostname
resolv.conf: ok
hostname: ok

To use this script, or otherwise use the OpenSSL utility for secure communication, it is only necessary to send a public key to a distant party. Assuming that the integrity of the public key is verified between the sender and receiver (that is, via an SHA-256 sum over the phone or another trusted channel), the sender can create a session key, then use it to encode and send arbitrary amounts of data through any untrusted yet reliable transfer medium with reasonable confidence of secrecy.

Note that the decryption block uses shell arrays, which are limited to 1024 elements in some versions (ksh88, pdksh). That will be a hard file limit in those cases.

This entire script can be worked into an email system for automated transfers. To do this on Oracle Linux 7 with the default Postfix SMTP server, ensure that the following two lines are set in /etc/postfix/main.cf:

inet_interfaces = $myhostname, localhost
default_privs = nobody

Here I will place a copy of the SSH private RSA host key in the /etc/postfix directory, set the configuration and permissions, open firewall port 25, then generate a public key as outlined below:

cd /etc/postfix
cp /etc/ssh/ssh_host_rsa_key .prv.key
chown nobody:nobody .prv.key
chmod 400 .prv.key
chcon system_u:object_r:postfix_etc_t:s0 .prv.key
iptables -I INPUT -p tcp --dport 25 --syn -j ACCEPT
openssl rsa -in .prv.key -pubout -out .pub.key

Notice that I'm using the nobody user with the system host key. If you are not comfortable with this security, note that the key file is in the ssh_keys group, and create a separate user for postfix to handle the keypair.

Next, place a copy of decrypter in /etc/postfix. The script must be modified to do the following: 1) skip the email header, 2) remove the password clause from the host key processing, 3) set /tmp as the unpack directory and 4) define new locations for the keypair. Below, sed is used with in-place editing to accomplish this:

sed -i.old '/^ while read Z/s:^:sed '"'"'1,/^$/d'"'"' |:
        s/^[ ]*-passin "[^"]*"//
        /^ DGST=/s:#.*$:cd /tmp:
        /^PVK=/c \
PVK=/etc/postfix/.prv.key; PBK=/etc/postfix/.pub.key' decrypter

With those changes in place, I create an email alias that will trigger the decrypter:

echo 'crypter: "| /etc/postfix/decrypter >> /tmp/cryp.log 2>&1"' \
        >> /etc/aliases
chcon system_u:object_r:postfix_local_exec_t:s0 decrypter
postfix reload
systemctl restart postfix.service

Now, pipe the encrypter output to the mail client:

cd /etc
encrypter resolv.conf hostname | mail crypter@localhost

The files sent into the mail client should appear in /tmp. Move the public key to a remote server, and automatic encrypted file transfer over SMTP is established.

It is also possible to work RSA encryption in reverse, decrypting with the public key. This is useful in establishing authenticity of data—for example, to encrypt a small amount of clear text (bounded by RSA length limitations) with the private key:

echo 'I have control of the private key.' |
  openssl rsautl -sign -inkey ~/.prv.key -passin "file:$HOME/.pas" |
  openssl base64 > blob

The blob file then can be posted in a public medium (website, file server and so on), and holders of the public key can successfully decrypt the message like so:

openssl base64 -d < blob |
        openssl rsautl -inkey ~/.pub.key -pubin

In doing so, users verify that the private key was involved in the creation of the message, lending some authenticity to the data that has been transferred. The public key is not assumed to be secret, so this establishes data authenticity, not data privacy.

Rather than arbitrary text, you can pipe in the text from an SHA-256 signature program call, and thus "sign" a larger file in a way that proves authenticity:

openssl dgst -sha256 crypter.sh |
  openssl rsautl -sign -inkey ~/.prv.key -passin "file:$HOME/.pas" |
  openssl base64 > csign

You decrypt this text in exactly the same manner as you did before, producing an SHA-256 clear-text digest that you can verify independently. However, OpenSSL can summarize in one step the signed SHA-256 checksum (note that full x.509 keys also can be manipulated to sign a digest):

openssl dgst -sha256 -sign ~/.prv.key \
        -out crypter.sha256 crypter.sh

If the two files above are placed accessibly, holders of the public key can verify that the files have not been altered:

openssl dgst -sha256 -verify ~/.pub.key \
        -signature crypter.sha256 crypter.sh

OpenSSL should output "Verified OK" when the files are intact. The capability of using an encrypted SHA-256 digest to verify a file securely is far beyond the features of the standard sha256sum utility and demonstrates authenticity unambiguously.

Introduction to GPG

GNU Privacy Guard has much more comprehensive tools for the management of keypairs and peer identities. This includes databases for storing the various types of keys, tools for revocation of keys and mechanisms for establishing key reputation in a "web of trust".

Oracle Linux 7 bundles GPG 2.0.22, which uses the 128-bit CAST5 symmetric cipher by default (newer versions have switched to AES128). Here, I will conform to the previous NIST guidelines for a 2868-bit asymmetric keypair of equal strength (note that the GPG documentation does warn that "Moving past RSA-2048 means you lose the ability to migrate your certificate to a smartcard, or to effectively use it on some mobile devices, or to interoperate with other OpenPGP applications that don't handle large keys gracefully."):

$ gpg --gen-key

gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: directory `/home/ol7_user/.gnupg' created
gpg: new configuration file `/home/ol7_user/.gnupg/gpg.conf' created
gpg: WARNING: options in `/home/ol7_user/.gnupg/gpg.conf' are not yet
        active during this run
gpg: keyring `/home/ol7_user/.gnupg/secring.gpg' created
gpg: keyring `/home/ol7_user/.gnupg/pubring.gpg' created
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 2868
Requested keysize is 2868 bits
rounded up to 2880 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 5y
Key expires at Sat 10 Jul 2021 08:40:19 PM CDT
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Oracle Linux
Email address: ol7_user@localhost
Comment: Test Key
You selected this USER-ID:
    "Oracle Linux (Test Key) <ol7_user@localhost>

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: /home/ol7_user/.gnupg/trustdb.gpg: trustdb created
gpg: key 6F862596 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2021-07-11
pub   2880R/6F862596 2016-07-12 [expires: 2021-07-11]
      Key fingerprint = F423 3B2C ACE1 AD0E 95C3  4769 679D 66ED 6F86 2596
uid                  Oracle Linux (Test Key)
sub   2880R/FF79FC31 2016-07-12 [expires: 2021-07-11]

Once the (rounded-up) 2880-bit private key has been created, a command is needed to generate a public key that can be shared with others:

$ gpg --export -a

Version: GnuPG v2.0.22 (GNU/Linux)


This key typically would be included in a signature at the end of email messages, bundled with software or documentation that requires either privacy or authenticity, or pasted in a public forum (such as a website) where similar activities might take place.

Other GPG/PGP users that desired to communicate with the originator of this key might then import it, prior to engaging in these activities:

$ gpg --import /tmp/test.pub
gpg: key F3BD3FF5:
public key "Test User (Test Key) <testuser@localhost>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
$ gpg --import /tmp/ol7.pub
gpg: key 6F862596:
public key "Oracle Linux (Test Key) <ol7_user@localhost>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

There are many email packages that will use various PGP components directly, enabling integrated cryptography. My focus here is flat file encryption, so I will confine my GPG demonstration to this specific action and use it to encrypt the script from the last section, sending from ol7_user@localhost to testuser@localhost:

$ gpg -u ol7_user@localhost -r testuser@localhost --armor
 ↪--sign --encrypt crypter.sh

You need a passphrase to unlock the secret key for
user: "Oracle Linux (Test Key) "
2880-bit RSA key, ID 6F862596, created 2016-07-12

$ mv crypter.sh.asc /tmp

$ head -5 /tmp/crypter.sh.asc
Version: GnuPG v2.0.22 (GNU/Linux)


The recipient (testuser) is then able to log in and decrypt (which will go to the standard output by default):

gpg -d /tmp/crypter.sh.asc

Any activity that causes GPG to request the password to a key will spawn an "agent" that will tie future GPG sessions and supply credentials so the key password need not be entered repeatedly:

testuser 4252 0:00 gpg-agent --daemon --use-standard-socket

The holder of a GPG private key also can sign fies digitally in a manner similar to OpenSSL (but somewhat more flexibly). There are three methods to add signatures: create a compressed binary file that contains a packed copy of the original message, add a clear-text "ASCII-armored" signature that allows the original content to be read, or write a binary signature to a separate file (requiring both a clean file and signature to validate). The first method writes a compressed binary to a new file with a .gpg extension:

gpg -s crypter.sh
gpg --sign crypter.sh

The second method will add a clear-text signature, allowing the original content to remain visible, into a new file with an .asc extension:

gpg --clearsign crypter.sh

The third will write a binary signature to a separate file with a .sig extension:

gpg -b crypter.sh
gpg --detach-sign crypter.sh

All of these methods can be verified by holders of the public key with the gpg -v (file) command, where (file) points at the output of GPG.

"Although GPG has the ability to support many types of digests and ciphers, forcing specific algorithms can cause compatibility problems with users of various distributions and versions of PGP software." It is wise to adhere to the capabilities of general versions, rather than specify algorithms directly (this discussion can be found in the man gpg pages):

man gpg | col -b | awk '/^INTEROPERABILITY/,/reduce/'

  GnuPG tries to be a very flexible implementation of the OpenPGP
  standard. In particular, GnuPG implements many of the optional
  parts of the standard, such as the SHA-512 hash, and the ZLIB and
  BZIP2 compression algorithms. It is important to be aware that
  not all OpenPGP programs implement these optional algorithms and
  that by forcing their use via the --cipher-algo, --digest-algo,
  --cert-digest-algo, or --compress-algo options in GnuPG, it is
  possible to create a perfectly valid OpenPGP message, but one
  that cannot be read by the intended recipient.

  There are dozens of variations of OpenPGP programs available, and
  each supports a slightly different subset of these optional
  algorithms. For example, until recently, no (unhacked) version
  of PGP supported the BLOWFISH cipher algorithm. A message using
  BLOWFISH simply could not be read by a PGP user. By default, GnuPG
  uses the standard OpenPGP preferences system that will always do the
  right thing and create messages that are usable by all recipients,
  regardless of which OpenPGP program they use. Only override this safe
  default if you really know what you are doing.

  If you absolutely must override the safe default, or if the
  preferences on a given key are invalid for some reason, you are far
  better off using the --pgp6, --pgp7, or --pgp8 options. These
  options are safe as they do not force any particular algorithms in
  violation of OpenPGP, but rather reduce the available algorithms
  to a "PGP-safe" list.

GPG also has the ability to be used non-interactively with the --batch and the various --passphrase options. It is likely unwise to use the same keys for both interactive and batch activity—use an email key for online communication and a batch key for automated activities. GPG offers several options for key revocation—be ready to use them for any key that is compromised, especially automated keys.


OpenSSL flat file use might be preferable to network services like TLS (or even SSH) for several reasons:

  • Removing TLS vastly reduces the attack surface of a server.

  • When an encryption process takes place offline and is not visible in action from the network, several classes of exploit are removed or greatly reduced in scope: timing attacks (such as Lucky Thirteen), other side-channel attacks (such as CRIME), and versioning attacks (such as DROWN).

  • Cipher algorithm code within OpenSSL is used in OpenSSH, which attests to quality. OpenSSH reviews are extremely thorough, and the security record is quite good.

  • One of OpenSSL's aes_core.c authors is Vincent Rijmen, who developed AES with fellow cryptographer Joan Daemen (although custom high-speed assembler code is substituted on architectures where it is available). Fragments of the aes_core.c code also are found in the libtomcrypt library that is used directly in the dropbear SSH server, which I discussed in a previous article (see "Infinite BusyBox with systemd" in the March 2015 issue.

  • OpenSSL's support for exotic systems introduces more problem code for networking than for basic math.

  • Far more time is spent in code reviews for OpenSSL's basic cipher algorithms than for the networking features. Merely the legal analysis of source code for the question of patent infringement can dwarf network security reviews (for example, Red Hat's recent decisions on Elliptic Curve within OpenSSL and Sun's careful coding of said routines to avoid existing patents). It was unlikely that the DTLS heartbeat TCP implementation received comparable analysis, and it became the greatest flaw ever found in OpenSSL (which never impacted flat file processing).

  • A scripted solution allows easier interfacing to custom programs (new compression tools, alternate data sources, legacy systems and applications and so on).

There are a few drawbacks to using the crypter script as presented:

  • The script places delimiters between the content of each file. The number of files sent, and their length, will be known by anyone observing the traffic. Use a ZIP utility to send only one file if this is troublesome—some ZIP utilities use AES directly, allowing an RSA exchange of a ZIP archive's password, then the transmission of the ZIP over unencrypted channels (this might allow the ZIP file directory to be read by observers, even if the file content remains opaque).

  • The script will read each file twice—once for the digest and once for the symmetric algorithm. This will cost time, processing power and I/O (GPG does this all in one step).

GPG also has a few concerns:

  • Some PGP implementations can have problems with larger RSA keys.

  • Compatibility issues between PGP implementations greatly influence chosen digests and ciphers.

  • GPG 2.0.22 (the older version found in Oracle Linux 7) uses the SHA-1 digest, which has been deprecated.

None of these tools are perfect, but they are the bedrock of secure communications. To ponder the scale of their influence upon commerce and trusted communication is almost beyond comprehension. These algorithms are as ubiquitous as they are generally unknown.

Hopefully, this tutorial has cast a bit more light upon them.

Load Disqus comments

Community Events

Park City, UT, USA
New Orleans, LA, USA
Copenhagen, Denmark
Austin, TX, USA
Austin, TX, USA