The Peculiar Case of Email in the Cloud
Most of the time when I start a project, or spin up a virtual server, it's done in my own basement "server farm". Not too many years ago, if I wanted those services to be public, I'd simply port-forward from my static IP into my personal machines. Or, perhaps I'd set up a name-based virtual host as a reverse proxy if I needed to expose a Web app. The nice thing about hosting projects locally is that when I need to send e-mail messages (usually error notifications), I simply can send them through my ISP's SMTP server. Granted, that's gotten a little more complex through the years, as ISPs are starting to lock down their mail servers and relay mail only for valid domains, but that just means I have to register my static IP properly. It works nicely.
The problem is, the issues I've had with my office Internet connection during the past year really have forced me to reconsider how I host public-facing services. I've been forced, by necessity, to spin up cloud instances and host my numerous projects remotely. I'm actually rather thankful for the need, because although it's not free to host projects remotely, it's fairly inexpensive and much more convenient. I still have my Raspberry Pi colocated in Austria for free (thanks again, Kyle Rankin, for pointing me to that awesome service!). Unfortunately, the Raspberry Pi isn't powerful enough for many of the crazy things I try on-line. It struggles, for instance, to host MySQL. So my main "project" server is a Google Compute instance that I end up paying about $15/month to keep active. That's not cheap, but I actually think I might be able to turn off one of my ESXi machines at home now, and I suspect it uses more than that in electricity.
The problem with Internet-hosted servers is that the lack of a usable SMTP relay makes e-mail very difficult. Yes, it's possible to install Postfix as a full-blown e-mail server, but I have no desire to worry about securing my own e-mail server from attacks attempting to use me as a SPAM relay. And although installing a non-relaying e-mail server certainly is possible, I've found that unless you configure SPF records, MX records and get particularly lucky, e-mail sent from a cloud instance often never arrives at the destination. This is especially true if you spin up servers on the fly.
The truth of the matter is, the only reason I want e-mail in the first place is so I can get notifications from my servers when something goes wrong. I don't ever need to reply to the e-mails. I don't really care where the e-mails come from (address-wise). I just want to have confidence that my notifications will get to me!
If you install Postfix on your server, it's possible to use a Gmail account to send all e-mail on your system. There are a few downsides to this method, but the configuration is simple, and Google's e-mail servers are very reliable. Plus, because you're not acting as an e-mail server yourself, you don't have to worry about having your e-mail rejected by recipients. It's legitimately coming from gmail.com.
The first unfortunate consequence is that for its simplest implementation, you need to enable "less secure apps" to log in to your Gmail account. I actually set up a separate gmail.com account for my server, and then I don't worry about the less secure setting. Thankfully, if this is a concern, it's possible to use two-factor authentication (more on that later).
Second, if you use Gmail as your e-mail relay, every e-mail will be rewritten to come "from" the gmail.com address. For me, this is a non-issue, because I just want my servers to e-mail me reliably when things go wrong. So although this won't be an issue for many servers, it's certainly not a feasible way to provide multi-user e-mail on your server. You can send from multiple users on your Linux system, but every e-mail that is sent will have its headers rewritten so they come from the same address! Thankfully, only the "from" address itself is rewritten, so messages come from an address formed like "User 1 <[email protected]>" and "User 2 <[email protected]>". So even though the underlying addresses are different, you still can tell from which user the e-mail messages are coming. This is useful for me, as it helps determine which app is sending me failure information!
First, you need to install Postfix along with the tools needed for enabling SASL connections. Your procedure will vary based on distribution, but for Ubuntu/Debian folks, it will go something like this:
sudo apt-get install postfix mailutils libsasl2-2 ↪ca-certificates libsasl2-modules
When Postfix installs, it will ask what type of system you're configuring. Multiple options will work, since you're editing the main file afterward, but I recommend you choose "Internet Site" and answer the questions accordingly. (Again, don't worry too much about what answers you put in the setup dialog, most of it will get overridden by your modifications anyway.)
Next, edit the main.cf file:
sudo nano /etc/postfix/main.cf
Then, change or add the following stanza of information somewhere in the file. Pay close attention, because there will be a few lines that look similar, but are subtly different. You'll probably have to add all the lines below:
relayhost = [smtp.gmail.com]:587 smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd smtp_sasl_security_options = noanonymous smtp_tls_CAfile = /etc/postfix/cacert.pem smtp_use_tls = yes
Now you need to create that cacert.pem file. You could just reference the original file directly, but I like to have all the required files in one folder—that makes it easier to replicate when spinning up new servers:
sudo cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem > ↪/etc/postfix/cacert.pem
In order to send mail, you need to have your authentication information on the server. Create the file from scratch:
sudo nano /etc/postfix/sasl/sasl_passwd
Enter your Gmail account information. It feels wrong to type a user name and password into a file, but you're going to lock the file's permissions pretty tight in the next step. Use this format in the file:
[smtp.gmail.com]:587 [email protected]:PASSWORD
PASSWORD obviously need to be substituted with your
account credentials. You also can use a Google Hosted Domain account,
just use the full e-mail address instead of @gmail.com. Then secure the
file and create a hash database so Postfix can read it properly:
sudo chmod 400 /etc/postfix/sasl/sasl_passwd sudo postmap /etc/postfix/sasl/sasl_passwd
Finally, reload Postfix and test outgoing e-mail:
sudo service postfix reload echo "It Worked" | mail -s "Email Test" [email protected]
It's very possible the e-mail will fail. If you get an error like this in your log files:
SASL authentication failed; server smtp.gmail.com[18.104.22.168] ↪said: 534-5.7.14 Please log in via your web browser ↪and then try again.
the most probable reason is that secure login setting I mentioned earlier. Like I mentioned previously, I don't have a problem with doing this on an account I created specifically for relaying e-mail. If you're using your actual e-mail address though, I don't really recommend it.
If you're still interested in softening the security to fix this problem, log in to your Gmail account and head here (Figure 1).
Figure 1. It seems odd to "turn on" insecurity.
You should be able to switch this to "turn on", which turns on the ability for less secure apps to log in. (That sounds a bit counterintuitive, turning "on" a more insecure method, but if you read closely, that's what you want to do.)
Once you make that change, try sending an e-mail again, and it should go through. The original e-mail actually probably will go through too, since Postfix keeps trying to send failed messages.
If You Prefer More Security
I've had mixed results using a Google App Password and two-factor authentication for Postfix e-mail relay setups. I'll leave this as an exercise for those folks who don't want to allow less secure authentication or who already use two-factor authentication on their accounts. (This might be the only option for Google Hosted Domain users whose administrators have not enabled the "less secure app" feature.)
The first step is to turn on two-factor authentication on your account. Otherwise, you won't be able to generate App Passwords. Head here in order to enable and configure two-factor authentication.
Then, create an App Password for your server setup here.
Once you have the App Password, copy it into your Postfix authentication
file in place of the password entered earlier. You'll need to re-create
the password map too using the
postmap command. Then restart Postfix,
and try sending an e-mail. If it doesn't work, check your log files
and start troubleshooting there. Like I said, I've had mixed results.
Not Just Gmail!
If you are struggling with your Gmail account, or just prefer not to rely on Google for relaying your information, the good news is this procedure works for any SMTP server. In fact, the configuration might be far simpler for other SMTP servers. If you have an e-mail account from your ISP, you likely can use that account by tweaking the settings above to match your ISP's account information. I think I have a charter.net e-mail address that I've never used for anything. I suspect many folks have similar addresses.
E-mail might be a dying form of communication, but for things like server notifications, it's hard to beat. The problem is, there are so many security concerns over relaying e-mail, it can be frustratingly difficult to configure one of the oldest messaging protocols!
Usually when I'm setting up a new server, I quickly install Postfix and configure it like this using a Chef or Puppet method for quick and reliable configuration. If you have a simpler or different method for enabling e-mail on cloud servers, drop me an e-mail at [email protected] I love hearing other solutions, and I'll share any really great solutions with the rest of the class in a future issue!