Using Sendmail as a Multi-Platform Mail Router
What follows is a brief introduction to DNS. If you're already familiar with it feel free to skip the next few paragraphs.
DNS stands for Domain Name System. Its job is to keep track of each computer's name on the network. Programs that communicate with other computers require the numerical address of that computer. If all the program has is a name, it gives that name to DNS and asks for the corresponding address. For example, the mail hub has to get the address for gateway1.calibersys.com before it can deliver mail to it. The hub asks DNS for the address and is told something like 220.127.116.11. As soon as the hub has that address, it can contact gateway1 to deliver the mail. The DNS configuration files are filled with lines like these:
mhub.calibersys.com. IN A 18.104.22.168 mhub2.calibersys.com. IN A 22.214.171.124 gateway1.vikingfreight.com. IN A 126.96.36.199
The first column is the name the machine goes by. The second column, IN, isn't important for our discussion. The third column, A, indicates that this is an address record. It just means this line maps a name to an address. The fourth column holds the address of the machine named in column one. In addition to looking up names and giving back addresses, DNS can also indicate that one computer accepts mail for another. When computer A accepts mail for computer B, A is called a Mail Exchanger for B. Whenever sendmail tries to deliver mail to a given machine, it first looks for a mail exchanger. If no mail exchanger is found, it then looks for a regular address. Let's look at an example:
calibersys.com. IN MX 10 mhub.calibersys.com. vikingfreight.com. IN MX 10 mhub.calibersys.com. shiprps.com. IN MX 10 mhub.calibersys.com. roberts.com. IN MX 10 mhub.calibersys.com.
These lines tell sendmail that any mail addressed to calibersys.com, vikingfreight.com, shiprps.com or roberts.com should be sent to mhub.calibersys.com. That's how mail addressed to email@example.com is routed to the hub. The first column can be thought of as a machine name. There doesn't have to be an actual computer using this name; think of it as a pseudo-machine for e-mail purposes. It's what you would see to the right of the @ symbol in an e-mail address. Again, we don't care about the IN for this discussion. The MX in the third column tells DNS that this is a mail exchanger record. Next comes the priority. A machine can have several mail exchangers, each with a different priority. I'll discuss that in a moment. Finally, the last column is the name of the machine acting as the mail exchanger. Any machine acting as a mail exchanger must be a real machine and must have a corresponding address record. Now let's talk about multiple exchangers. Remember that this project involves two hubs, a primary and a secondary, The primary machine is mhub.calibersys.com, the secondary is mhub2.calibersys.com. Both are listed in DNS. Remember that the number 10 above referred to priority. The lower the number, the higher the priority. Let's say that we see the following lines in the DNS configuration file in addition to the ones above:
calibersys.com. IN MX 20 mhub2.calibersys.com. vikingfreight.com IN MX 20 mhub2.calibersys.com. shiprps.com. IN MX 20 mhub2.calibersys.com. roberts.com. IN MX 20 mhub2.calibersys.com.
sendmail would first try to send any mail destined for vikingfreight.com to mhub, since 10 represents a higher priority than 20. If that failed, it would then try to send the mail to mhub2, the backup hub. After either hub receives the message, it looks up firstname.lastname@example.org and converts that address to email@example.com. It would then look up gateway1.vikingfreight.com. It does not have a mail exchanger record listed for it, but it does have an address record. This means that gateway1 accepts its own mail. Using DNS this way to route mail for a domain, e.g., mycompany.com, to an actual computer where the mail is stored, e.g., mail.mycompany.com, is commonplace. What's different here is the address mapping. To see how that's done, we need to look at sendmail.cf.
sendmail.cf is the configuration file for sendmail. So what exactly is sendmail? sendmail is the Swiss Army knife of mail systems. Officially it's known as a message transfer agent, or MTA. There are a few different flavors; Linux Slackware 3.0 comes with the one known as Berkeley V8. Users typically don't interact with it directly. (That task is left to a mail user agent or MUA such as elm or pine.) sendmail runs in the background, silently routing mail from one computer to another. It was written in the 1970's and 1980's by Eric Allman at U.C. Berkeley. Because of Eric's flexible design, sendmail is still the most widely used MTA on the Internet. It's standard issue software with just about any Unix-based operating system. All that flexibility comes at the price of complexity. sendmail is probably the most complex of all the Unix utilities. I'll cover some of sendmail's features and how they can be used to solve our address mapping problems, but a complete discussion of sendmail is beyond the scope of this article. For more information see the resource box. The Caliber mail hub makes heavy use of three sendmail mechanisms: macros, classes, and database lookups. All these are specified in the configuration file, sendmail.cf. sendmail macros are similar to C language macros. A primary difference is that the macro name can only be one character long. For example,
defines the macro G as gateway1.vikingfreight.com. Its C language equivalent would be:
#define G gateway1.vikingfreight.com
The macro is invoked later by specifying a dollar sign followed by the macro name, e.g., $G. Anywhere the symbol $G appears, gateway1.vikingfreight.com would be substituted in its place. Classes are very similar to macros. The difference is that they can expand to one of many different values. For example,
CU xyz123 abc789 def444
defines the class U with three values. Alternatively, sendmail can read the values from an external text file:
This ability is handy if you want to define a class with a large number of values. The class is invoked by specifying a dollar sign, then an equal sign, then the class name, e.g., $=U. I'll explain a little later how classes are useful. The third mechanism exploited by the mail hub is the database lookup. sendmail can consult external databases and swap the lookup key with the value found in the database. A few different database formats are supported; we use GNU dbm databases. The records in these databases have only two fields. The lookup key is the first field. Everything following that is considered a value field. A dbm database with four records could look like this:
xyz123 tlowery ft1100 jdoe bc789 asmith def444 bjones
If sendmail consults the database looking for abc789, it will find the value of asmith and substitute that value in place of abc789 in the e-mail address.
That covers the basic mechanisms. Now let's look at the configuration file itself. sendmail.cf if filled with lines like this one:
R$=U@$G $:$(mapdb $1 $)@$K
These lines are known as rules. The rules are grouped together in subroutines, each of which is called by sendmail to perform a certain task. These subroutines are called sets. It's a terse programming language based on regular expression pattern matching. Each rule examines an e-mail address and may alter it. Rules have two parts, the left-hand-side (LHS) and right-hand-side (RHS), separated by one or more tab characters. The LHS is a pattern; sendmail tries to match the current address with this pattern. If the address matches the pattern, sendmail will rewrite the address based on what the RHS says. If there is no match, the RHS is ignored. Now let's look at the LHS of our sample rule, left to right:
The R simply states that this is a rule. All rules start with the letter R. The next three characters, $=U, are a class reference. Given the class definition above, $=U will match xyz123, abc789 or def444 successfully. If the address begins with any other string, the match will fail. The next character is a literal @. That character must appear in the address for a match to happen. The following two characters, $G, are a macro reference. The macro expands to:
The address firstname.lastname@example.org would match the pattern and the RHS would be invoked. email@example.com would not match since tza555 is not a member of the U class. Likewise, firstname.lastname@example.org would not match since gateway2.calibersys.com is not the value of the G macro. Now let's move on to the RHS to see how an address is rewritten. The RHS of our sample rule is this:
$:$(mapdb $1 $)@$K
The first two characters, $: tell sendmail to only invoke this RHS once. By default, sendmail will invoke the RHS repeatedly as long as the result still matches the LHS. Following that is the string,
$(mapdb $1 $)
which performs the database lookup, tells sendmail to look for a database named mapdb and search for the first item from the LHS. $2 would search for the second item and so on. The first match from our LHS was xyz123. sendmail then searches for that string and finds tlowery, so it replaces xyz123 with tlowery in the address. The next character in the RHS, @, is a literal. It's written to the new address following tlowery. Next comes $K, a macro reference. Let's assume the macro K was defined like so:
sendmail will place calibersys.com after @ in the new address, completing the rewriting process.
From this we see how a private address of email@example.com can be converted to a public address of firstname.lastname@example.org. The recipient of the message will never know the original sender address was not the public address. Switching from public to private can be accomplished in a similar manner. This discussion of address mapping has only scratched the surface; sendmail's flexibility can help the e-mail administrator solve virtually any mail routing task.
- Transitioning to Python 3
- Red Hat OpenStack Platform
- Stepping into Science
- Tech Tip: Really Simple HTTP Server with Python
- Linux Journal December 2016
- CORSAIR's Carbide Air 740
- Radio Free Linux
- The Tiny Internet Project, Part II
- A Better Raspberry Pi Streaming Solution
- FutureVault Inc.'s FutureVault
Pick up any e-commerce web or mobile app today, and you’ll be holding a mashup of interconnected applications and services from a variety of different providers. For instance, when you connect to Amazon’s e-commerce app, cookies, tags and pixels that are monitored by solutions like Exact Target, BazaarVoice, Bing, Shopzilla, Liveramp and Google Tag Manager track every action you take. You’re presented with special offers and coupons based on your viewing and buying patterns. If you find something you want for your birthday, a third party manages your wish list, which you can share through multiple social- media outlets or email to a friend. When you select something to buy, you find yourself presented with similar items as kind suggestions. And when you finally check out, you’re offered the ability to pay with promo codes, gifts cards, PayPal or a variety of credit cards.Get the Guide