Radio E-mail in West Africa: The Complete Version
Here in West Africa, it seems nearly everything just barely hangs together with bent nails and spit. While this is a tribute to the resourcefulness and ingenuity of Guineans in the face of scarcity, darn if this doesn't make a rough neighborhood for hardware and networks. I have found power cords hacked apart and spliced together with scotch tape, motherboards rigged in place with twist ties and wedges of cardboard. It is forever hot, dusty, buggy and humid. Generators go out, plugs get kicked, lightning strikes, nothing is grounded. In short, we face the harshest of environments in trying to implement high-tech reliably.
For these reasons alone, we would probably use Dan Bernstein's outstandingly robust qmail tools to move the mail. qmail is designed with particular attention to reliability, making sure messages are safely written to disk at each stage of processing. If a message is handled by qmail—especially when delivered to a Maildir, the special mailbox directory format developed by Bernstein—you can be sure that message is well and truly delivered. Otherwise, the mail always remains safely queued on disk for delivery at a later time. This feature is essential for the frequent power and network disruptions we face in the African environment.
Going beyond qmail's core reliability, Bernstein also has developed a number of special purpose tools and applications perfectly suited to the Radio E-mail project. Central among these, qmail includes a QMTP server, implementing Bernstein's own Quick Mail Transport Protocol. QMTP is accessible from the serialmail package, a supplemental suite of programs designed specifically by Bernstein for moving mail over slow dial-up connections. And our connections would certainly qualify as slow! By using qmail, QMTP and serialmail—controlling the entire chain of e-mail delivery within our own network—we have an optimal solution for moving mail as efficiently and reliably as possible over HF radio.
As shown on the reference network in Figure 1, qmail runs on no less than five hosts: the central mail server coyah, the radio hub server congo and each of the three field office hosts. That's a lot of qmail! In fact, our production network adds a sixth qmail installation on a separate gateway/firewall system. This is configured with what is known as a mini-qmail system, using Bernstein's Quick Mail Queuing Protocol (QMQP) to queue incoming mail through the firewall directly to the mailhub host coyah. Table 1 shows the qmail components installed on each of these different systems.
With so many qmail servers to install and configure, you intimately become acquainted with all things qmail; it is a wonderfully capable and flexible system. See the Resources for some pointers, and prepare to spend some time with The Big Qmail Picture, Bernstein's FAQ documentation and the PIC files. All installations follow Dave Sill's “Life with qmail” instructions to the letter, the indispensable guide for setting up production qmail systems. The discussion that follows here assumes familiarity with qmail and its ways: its control files, the special user alias the qmail virtual domain mechanism, dot-qmail delivery instructions and Maildir mailbox delivery.
For the e-mail examples given in this article, we'll use the nonexistent refugee.ngo domain to represent IRC-Guinea's actual domain of irc.org.gn. Each user on the IRC-Guinea network, whether in Conakry or a field office, is assigned an e-mail address of the form email@example.com. All mail addressed to the domain refugee.ngo is processed by the mailhub host and routed as necessary from there. No sender needs to know anything about a user's particular location of assignment or otherwise use any special form of addressing.
To show how this can be done with qmail, let's first consider a message arriving (or originating) in Conakry for firstname.lastname@example.org, a user who happens to be posted to the field in Kissidougou. The key processing steps are sketched in Figure 2 and described in more detail below.
On the mailhub server coyah, qmail finds the domain refugee.ngo in its locals control file and attempts to deliver the message locally, that is, to a user account on the host coyah. First it checks for a user named rhonda.rabbit in the system /etc/passwd database. When qmail fails to find the user there, it turns the message over to the special alias user, processing it in accordance with the instructions in /var/qmail/alias/.qmail-default. Here we have installed the fastforward command, directing a lookup in a central database built from the file /etc/aliases. Now an entry is found for rhonda.rabbit, and the e-mail message is returned to qmail with a new envelope address of email@example.com.
Still on coyah, qmail now does not find kissidougou.refugee.ngo in either its locals or virtualdomains control files, so it queues the message for remote delivery. But before qmail tries to find an MX record with a name server lookup for the kissidougou.refugee.ngo domain, which does not exist, it checks in the smtproutes control file and finds a match. In this way all mail destined for kissidougou.refugee.ngo, as well as for all other remote offices, is relayed directly to the radiohub host. (The smtproutes mechanism helps us avoid the complication of setting up and delegating a number of subdomains and MX records in our local nameserver.)
When the message arrives at the radio hub congo, qmail won't find the domain kissidougou.refugee.ngo in the locals control file. In fact the radiohub may be configured without a locals control file. Instead qmail finds a match in its virtualdomains control file. The matching entry in the file kissidougou.refugee.ngo:qturn-kissidougou instructs qmail to forward all messages addressed to the virtual domain kissidougou.refugee.ngo, prepending the string qturn-kissidougou- to the original envelope address. In the case of our example message then, the address is now rewritten as firstname.lastname@example.org and delivered locally to user qturn.
qturn is a special user account we have set up on the radio hub congo, with a home directory of /var/qmail/qturn, to receive all mail destined for delivery to remote offices via HF radio modem. Following qmail's dot-qmail conventions, qmail looks in qturn's home directory for local delivery instructions in /var/qmail/qturn/.qmail-kissidougou. Our instructions here tell qmail to deliver to the Maildir mailbox ./kissidougou/.QMAIL.PPP/. (Similarly for messages destined for Dabola, for example, the dot-qmail file ~/.qmail-dabola will instruct delivery to the Maildir mailbox ./dabola/.QMAIL.PPP/.)
qmail now considers the message delivered. It is out of the qmail queuing mechanism, and no further delivery or forwarding attempts are made autonomously.
Instead, we wait for the next PPP connection with the remote host kenya in Kissidougou. Then we can make use of the serialmail package to blast—relatively speaking—all the mail collected in the /var/qmail/qturn/kissidougou/.QMAIL.PPP/ mailbox across the link using QMTP. The command sequence that sends the collected mail over the link is:
# cd /var/qmail/qturn/kissidougou # maildirqmtp .QMAIL.PPP qturn-kissidougou- 10.0.10.243
The first argument to maildirqmtp is the path to the Maildir directory with the mail we want to transfer. The third argument is the IP address of the destination qmail-qmtpd server, in this case the static IP address we have assigned to the Kissi end of the PPP connection.
The purpose of the second argument may be less obvious. It provides the prepended string which should be removed from the envelope address on each message. Recall that the local part of the example message address has been rewritten as qturn-kissidougou-rhonda.rabbit. When maildirqmtp sends this message, the prepended qturn-kissidougou- is stripped off, and the envelope address is written once again as email@example.com.
The message then wings its way over Guinea, lofting through African skies on gossamer currents of HF ether, coming gently alight upon the Kissi radio aerial after a long and spectacular journey. Here the qmail system on kenya may now complete its delivery.
Again qmail first checks its locals control file and does find kissidougou.refugee.ngo among the entries, since here is where we want local delivery of the message. Although there is no user named rhonda.rabbit in kenya's /etc/passwd file, the fastforward lookup in /etc/aliases does find a match and sets up delivery of the message to the user account of rrabbit. The dot-qmail delivery instructions in /home/rrabbit/.qmail—as for all users set up to receive POP3 e-mail—is configured to deliver to the Maildir mailbox named ./.QMAIL.POP/ in the user's home directory.
By a happy coincidence, we have configured qmail's POP3 daemon to deliver mail from Maildirs named .QMAIL.POP in users' home directories. With her laptop connected to the local network in Kissidougou, eagerly awaiting word from the world beyond, Ms. Rabbit is now at last greeted with the joyous chimes of its arrival, “You have mail.” Radio E-mail works: they laughed, they cried, they danced.
Practical Task Scheduling Deployment
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.View Now!
|The Firebird Project's Firebird Relational Database||Jul 29, 2016|
|Stunnel Security for Oracle||Jul 28, 2016|
|SUSE LLC's SUSE Manager||Jul 21, 2016|
|My +1 Sword of Productivity||Jul 20, 2016|
|Non-Linux FOSS: Caffeine!||Jul 19, 2016|
|Murat Yener and Onur Dundar's Expert Android Studio (Wrox)||Jul 18, 2016|
- Stunnel Security for Oracle
- The Firebird Project's Firebird Relational Database
- SUSE LLC's SUSE Manager
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Managing Linux Using Puppet
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- Google's SwiftShader Released
- SuperTuxKart 0.9.2 Released
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide