Radio E-mail in West Africa: The Complete Version
Here in West Africa, it seems nearly everything just barely hangs together with bent nails and spit. While this is a tribute to the resourcefulness and ingenuity of Guineans in the face of scarcity, darn if this doesn't make a rough neighborhood for hardware and networks. I have found power cords hacked apart and spliced together with scotch tape, motherboards rigged in place with twist ties and wedges of cardboard. It is forever hot, dusty, buggy and humid. Generators go out, plugs get kicked, lightning strikes, nothing is grounded. In short, we face the harshest of environments in trying to implement high-tech reliably.
For these reasons alone, we would probably use Dan Bernstein's outstandingly robust qmail tools to move the mail. qmail is designed with particular attention to reliability, making sure messages are safely written to disk at each stage of processing. If a message is handled by qmail—especially when delivered to a Maildir, the special mailbox directory format developed by Bernstein—you can be sure that message is well and truly delivered. Otherwise, the mail always remains safely queued on disk for delivery at a later time. This feature is essential for the frequent power and network disruptions we face in the African environment.
Going beyond qmail's core reliability, Bernstein also has developed a number of special purpose tools and applications perfectly suited to the Radio E-mail project. Central among these, qmail includes a QMTP server, implementing Bernstein's own Quick Mail Transport Protocol. QMTP is accessible from the serialmail package, a supplemental suite of programs designed specifically by Bernstein for moving mail over slow dial-up connections. And our connections would certainly qualify as slow! By using qmail, QMTP and serialmail—controlling the entire chain of e-mail delivery within our own network—we have an optimal solution for moving mail as efficiently and reliably as possible over HF radio.
As shown on the reference network in Figure 1, qmail runs on no less than five hosts: the central mail server coyah, the radio hub server congo and each of the three field office hosts. That's a lot of qmail! In fact, our production network adds a sixth qmail installation on a separate gateway/firewall system. This is configured with what is known as a mini-qmail system, using Bernstein's Quick Mail Queuing Protocol (QMQP) to queue incoming mail through the firewall directly to the mailhub host coyah. Table 1 shows the qmail components installed on each of these different systems.
With so many qmail servers to install and configure, you intimately become acquainted with all things qmail; it is a wonderfully capable and flexible system. See the Resources for some pointers, and prepare to spend some time with The Big Qmail Picture, Bernstein's FAQ documentation and the PIC files. All installations follow Dave Sill's “Life with qmail” instructions to the letter, the indispensable guide for setting up production qmail systems. The discussion that follows here assumes familiarity with qmail and its ways: its control files, the special user alias the qmail virtual domain mechanism, dot-qmail delivery instructions and Maildir mailbox delivery.
For the e-mail examples given in this article, we'll use the nonexistent refugee.ngo domain to represent IRC-Guinea's actual domain of irc.org.gn. Each user on the IRC-Guinea network, whether in Conakry or a field office, is assigned an e-mail address of the form firstname.lastname@example.org. All mail addressed to the domain refugee.ngo is processed by the mailhub host and routed as necessary from there. No sender needs to know anything about a user's particular location of assignment or otherwise use any special form of addressing.
To show how this can be done with qmail, let's first consider a message arriving (or originating) in Conakry for email@example.com, a user who happens to be posted to the field in Kissidougou. The key processing steps are sketched in Figure 2 and described in more detail below.
On the mailhub server coyah, qmail finds the domain refugee.ngo in its locals control file and attempts to deliver the message locally, that is, to a user account on the host coyah. First it checks for a user named rhonda.rabbit in the system /etc/passwd database. When qmail fails to find the user there, it turns the message over to the special alias user, processing it in accordance with the instructions in /var/qmail/alias/.qmail-default. Here we have installed the fastforward command, directing a lookup in a central database built from the file /etc/aliases. Now an entry is found for rhonda.rabbit, and the e-mail message is returned to qmail with a new envelope address of firstname.lastname@example.org.
Still on coyah, qmail now does not find kissidougou.refugee.ngo in either its locals or virtualdomains control files, so it queues the message for remote delivery. But before qmail tries to find an MX record with a name server lookup for the kissidougou.refugee.ngo domain, which does not exist, it checks in the smtproutes control file and finds a match. In this way all mail destined for kissidougou.refugee.ngo, as well as for all other remote offices, is relayed directly to the radiohub host. (The smtproutes mechanism helps us avoid the complication of setting up and delegating a number of subdomains and MX records in our local nameserver.)
When the message arrives at the radio hub congo, qmail won't find the domain kissidougou.refugee.ngo in the locals control file. In fact the radiohub may be configured without a locals control file. Instead qmail finds a match in its virtualdomains control file. The matching entry in the file kissidougou.refugee.ngo:qturn-kissidougou instructs qmail to forward all messages addressed to the virtual domain kissidougou.refugee.ngo, prepending the string qturn-kissidougou- to the original envelope address. In the case of our example message then, the address is now rewritten as email@example.com and delivered locally to user qturn.
qturn is a special user account we have set up on the radio hub congo, with a home directory of /var/qmail/qturn, to receive all mail destined for delivery to remote offices via HF radio modem. Following qmail's dot-qmail conventions, qmail looks in qturn's home directory for local delivery instructions in /var/qmail/qturn/.qmail-kissidougou. Our instructions here tell qmail to deliver to the Maildir mailbox ./kissidougou/.QMAIL.PPP/. (Similarly for messages destined for Dabola, for example, the dot-qmail file ~/.qmail-dabola will instruct delivery to the Maildir mailbox ./dabola/.QMAIL.PPP/.)
qmail now considers the message delivered. It is out of the qmail queuing mechanism, and no further delivery or forwarding attempts are made autonomously.
Instead, we wait for the next PPP connection with the remote host kenya in Kissidougou. Then we can make use of the serialmail package to blast—relatively speaking—all the mail collected in the /var/qmail/qturn/kissidougou/.QMAIL.PPP/ mailbox across the link using QMTP. The command sequence that sends the collected mail over the link is:
# cd /var/qmail/qturn/kissidougou # maildirqmtp .QMAIL.PPP qturn-kissidougou- 10.0.10.243
The first argument to maildirqmtp is the path to the Maildir directory with the mail we want to transfer. The third argument is the IP address of the destination qmail-qmtpd server, in this case the static IP address we have assigned to the Kissi end of the PPP connection.
The purpose of the second argument may be less obvious. It provides the prepended string which should be removed from the envelope address on each message. Recall that the local part of the example message address has been rewritten as qturn-kissidougou-rhonda.rabbit. When maildirqmtp sends this message, the prepended qturn-kissidougou- is stripped off, and the envelope address is written once again as firstname.lastname@example.org.
The message then wings its way over Guinea, lofting through African skies on gossamer currents of HF ether, coming gently alight upon the Kissi radio aerial after a long and spectacular journey. Here the qmail system on kenya may now complete its delivery.
Again qmail first checks its locals control file and does find kissidougou.refugee.ngo among the entries, since here is where we want local delivery of the message. Although there is no user named rhonda.rabbit in kenya's /etc/passwd file, the fastforward lookup in /etc/aliases does find a match and sets up delivery of the message to the user account of rrabbit. The dot-qmail delivery instructions in /home/rrabbit/.qmail—as for all users set up to receive POP3 e-mail—is configured to deliver to the Maildir mailbox named ./.QMAIL.POP/ in the user's home directory.
By a happy coincidence, we have configured qmail's POP3 daemon to deliver mail from Maildirs named .QMAIL.POP in users' home directories. With her laptop connected to the local network in Kissidougou, eagerly awaiting word from the world beyond, Ms. Rabbit is now at last greeted with the joyous chimes of its arrival, “You have mail.” Radio E-mail works: they laughed, they cried, they danced.
Editorial Advisory Panel
Thank you to our 2014 Editorial Advisors!
- Jeff Parent
- Brad Baillio
- Nick Baronian
- Steve Case
- Chadalavada Kalyana
- Caleb Cullen
- Keir Davis
- Michael Eager
- Nick Faltys
- Dennis Frey
- Philip Jacob
- Jay Kruizenga
- Steve Marquez
- Dave McAllister
- Craig Oda
- Mike Roberts
- Chris Stark
- Patrick Swartz
- David Lynch
- Alicia Gibb
- Thomas Quinlan
- Carson McDonald
- Kristen Shoemaker
- Charnell Luchich
- James Walker
- Victor Gregorio
- Hari Boukis
- Brian Conner
- David Lane