Hack and / - Last-Minute Secondary Mail Server

It's easy to build redundant systems when time and money are limitless. When you have neither, or you are designing a personal system, often backups and redundancy are parts of the project you plan to get to on a rainy day. Of course inevitably, you put those tasks off until the main system fails, and then you scramble to pick up the pieces. Setting up RAID and doing backups, in fact, are probably the most common examples of these do-it-on-a-rainy-day tasks (and if you haven't heard yet, they are not the same thing). We all know we should back up important data, and we should set up a RAID on that important file server, and by now, enough of us have been bitten by that mistake that I'm not going to talk about either today. Instead, I'm going to talk about one of those services that gets less attention: your mail server.

It is important to have a backup mail server, but whether you work for a small company, or you administer your own personal mail server, you might not have gotten around to a secondary mail relay. Then, disaster strikes. It could be that the primary mail server's hardware failed, or maybe it was hacked and, in either case, it is going to be down for a few days. In the meantime, you still would like to be able to send and receive e-mail. In this column, I cover a few easy, and more important, quick steps to create a secondary mail relay to tide you over until the primary can come back on-line.

Now, my preferred method for an emergency mail server uses a Knoppix disc. I always have one around somewhere, and because I can install just about any software I need on the live CD, it is ideal when I need a basic Linux system fast. I can just locate a spare machine, boot Knoppix on it, and set up my server. When the emergency is over, I can shut it off with no commitment.

Then again, you might want to make this secondary mail server a bit more permanent. These same steps will work just fine on any ordinary Linux system that has postfix available.

Before you perform any steps, be sure to choose a server that has enough storage to store your mail. This number varies based on your e-mail traffic and the number of clients on the server, but the machine will need to store all incoming mail locally until the primary server comes back up. So, if you get 50MB of mail each day and plan for the primary to be down for three days, you should have at least 150MB of spare storage for the mail spool in /var/spool/postfix plus extra, just in case. If you don't have enough spare storage or you use Knoppix for this, mount an extra partition, create a postfix directory on it and symlink /var/spool/postfix to it.

Now that you have chosen a server, the first step is to install postfix. Postfix is a common package and should be available for any major Linux distribution you use. On Debian-based systems, the installation process automatically runs a configuration script to set up a reasonable default config. If you do run into this script, choose the “Internet Site” configuration type and accept the rest of the defaults in the script. Alternatively, you can copy a default configuration that ships with your postfix package or run it through a configuration script your distribution includes.

Once postfix is installed, you need to tweak the default configuration so that it can act as your mail relay. Postfix makes this pretty simple, and you need to worry about only a few configuration options. Edit the /etc/postfix/main.cf file, and locate a line called mynetworks. This option tells postfix for which networks to relay mail. Ideally, you should set this only for internal networks or specific external hosts you trust will not relay spam through your system. If you allow all networks, you have just turned your system into an open relay and will likely find yourself on a spam black-hole list in no time. If your local network is 192.168.1.x, for instance, you would add an entry for that and for localhost:

mynetworks = 127.0.0.0/8, 192.168.1.0/24

Next, you need to tell postfix for which incoming domains it will accept mail for relay. This variable will be set to any domains for which you accept incoming mail. So, if you own example.com and example.org, for instance, you would add:

relay_domains = example.com, example.org

You even can act as a secondary mail server for friends. Simply add their domains here as well, and your mail server will accept incoming mail to those domains and then forward it to the appropriate primary mail server. How does it know which server to use? It relies on DNS, which I discuss shortly.

The final postfix options to change tell postfix how long to spool and attempt to deliver mail before it bounces it. By default, postfix queues mail for three days, and during that time, it continuously attempts delivery. After three days, postfix bounces the mail and sends an e-mail notification to the sender that the e-mail could not be delivered. If your primary server is going to be down for a few days, you probably will want to extend this default. Locate the following values (or add them if they aren't defined) in /etc/postfix/main.cf, and edit them so they look like the following:

bounce_queue_lifetime = 14d
maximal_queue_lifetime = 14d

Here I increased the maximum time to 14 days, but you can change it to a value that makes sense for you. Generally, you don't want to hold on to e-mail for too long, as senders likely will want to know eventually if their e-mail could not be delivered. Once these options are changed, type /etc/init.d/postfix start to start the service, or type /etc/init.d/postfix reload, if postfix already is running.

Next, test the server. Either configure your mail client to use this server as its SMTP gateway and then send an e-mail to your domain, or if you feel fancy, connect to port 25 on the server using Telnet, and type the raw SMTP commands. Check /var/log/mail.log or /var/log/maillog to confirm that postfix accepted and spooled your mail.

The last step is to configure your DNS server so that it lists your new machine as a secondary mail server for your domain. Your DNS server should have at least one MX record defined that looks something like this:

example.com.    IN MX    100    mail1.example.com.

If I created a new mail server and added its IP to DNS so that mail2.example.com pointed to it, I then would add the following line to my DNS zone:

example.com.    IN MX    200    mail2.example.com.

Because I assigned mail2 a higher value (200) than mail1 (100), other mail servers know that mail1 is my primary and that mail ultimately will land there. However, if mail1 is unavailable, they know that they can attempt delivery on mail2 (and some mail servers attempt mail delivery on secondary servers first anyway). Once my DNS zone is reloaded, mail that has been queued up on remote servers ever since mail1 went down should start being delivered to mail2. Be sure to add this DNS entry for any domains you added in the relay_domains option. It also may go without saying, but be sure that mail2.example.com points to an external IP address that lands on your mail server.

As this server runs, monitor its storage to make sure you have plenty for new incoming mail. You also can run the mailq command to see all the queued messages. Once your primary server comes back up, postfix will start delivering its queued messages automatically (it actually will have been attempting it the entire time). By default, postfix will throttle this delivery so it doesn't flood the primary mail server, but if you want all of the queued e-mail delivered immediately, type postqueue -f.

With the primary machine back up, you might want to take down this temporary machine or at least work on a more permanent solution. If you do take it down, be sure to remove its MX record from all your DNS servers. You do have redundant DNS servers, right?