Radio E-mail in West Africa: The Complete Version

Remote networking with high-frequency (HF) radio and Dan Bernstein's qmail.
Resources

Dan Bernstein's qmail and tools

Dave Sill's “Life with qmail”

Andre Opperman's “The Big Qmail Picture”

Bushmail is a commercial provider based in South Africa offering a publicly accessible radio e-mail network to many locations in sub-saharan Africa.

The Mission Aviation Fellowship provides additional information on the use of HF radio modems, particularly oriented to the use of the E-Smith Linux distribution.

Codan

The International Rescue Committee

Wayne Marshall is a UNIX programmer and technical consultant living in Guinea, who never intended to become an e-mail administrator. He and his wife Paula enjoy traveling and African skies.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

CRMF Papua New Guinea

Anonymous's picture

wow great.

did you know CRMF in Papua Neu Guinea is doing the same?
visit www.crmf.com

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

I worked as a Radio operator for NGOs in Guinea and experienced the "huddle" and difficulties involved in it, especially during the time of tight security. Can you imagine rebels (killers) just infront of your office and you are responsible to say inform the HQ office in some where around 600 Kms (NZerekore or Macenta)? I think Wayne can do something by creating a second type "Geekocop". This is amazing, the budget is less than nothing. I worked with IRC and know how strict and small is the operating budget for communication is. Now, as a Network & Systems Engineer myself, I greatly admire Wayne and Paula who took over from Mustafa Elkanzi (Former IRC Country Director & personal friend of mine). Paula you should have the praise for this hard job done by Wayne....

Wayne, I will always be your admirer and fan. Infact, you have inspired me to dig into the Unix /Linux world. I already carry Microsoft Certificates in my pocket and Cisco.

Bravo, bravo and thanks to OpenSource

M. Pathe Jalloh,

Former refugee and IRC staff

Now working with Ashanti Goldfields, Siguiri-Guinea

pathe.diallo@ashantigold.com

www.ashantigold.com

Mustafa Elkanzi

Khizr I. Tajammul's picture

You mentioned you are a good friends with Mr. Elkanzi. I was wondering if you could forward me his contact details. It has come to my knowledge that he has become the country director of IRC in Pakistan.

Bridges is a development consortium and we would like to meet him and possibly assist him in his projects in Pakistan.

Appreciate your time.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

I think your right

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Anyone out there using KA9Q? Any comparisons to using a UNIX-based solution, or comparison to UUCP protocols?

Thanks,

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

When I picked a copy of Linux Journal up from the stands, I had no idea what a great article I was about to read.

The North-South digital divide is something I think a lot about, though I am just learning to appreciate the difficulties of connectivity in less-developed countries.

I'm involved with Indymedia.org (that is, a member of one of over 75 Independent Media Centres world-wide). Some of our American IMCs have been behind a push to send used technology to South American IMCs -- about 250 configured old PIIs are on a ship heading south as I write this.

Yes, those machines will all end up in urban settings. But we now have a real and tangible solution for reaching outside the cities, where the needs are arguably greater.

Those 250 PIIs are just the start of a long-term project. And now it seems there may be no end to it.

Thanks very much for a very inspiring read.

All the best,

David H.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Ham Radio operators have been doing this for YEARS now. Nothing new.

From using simple type of 'modems' like http://www.baycom.org/bayweb/index.htm or using tcp/ip over the air using the baycom modem and software like http://www.fwarig.org/home_jnos.html

Or providing ham radio operators who are sailing around the world access to email as well via HF.

http://www.w2xo.pgh.pa.us/

http://www.hfradio.com/

Of course thats just the tip of it.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Six years ago I set up a similar system for a christian missionary organisation "Operation Mobilisation". At that time you even got in Singapure Harbour a very bad telephone line that was only capable of 300-1200 baud (with a Trailblazer PEP modem). Not to speak for some locations more remote.

In the end we had an central e-mail hub that could do SMTP, UUCP over Modem und UUCP over TCP/IP. Some of the "clients" had DOS based UUCP systems with UUPC and Pegasus Mail for DOS, some had Windows based systems, and some had Linux servers with Pegasus-Mail for Windows. You see, many operation modes were possible and we were able to adjust to the different technology levels of the various countries. Some used SMTP with their local providers.

In not-so-advanced countries we used UUCP-over-TCPIP if we got a local ISP, sometimes we had to use UUCP-over-Modem. In this case we used the uucp-g protocol because of it's efficiency. To my knowledge, we did not use radio-transmitting, althought some people looked into packet-radio like technology.

However, the choice of transpartion was surprisingly often not a function of the available infrastructure. Instead it was quite often a result of the locally available expertise. A missionary organisation does not generate profits, it lives from donation. Therefore money is usually scarce. It was simply not possible to train all operators worldwide up to the same level of expertise, just the flight tickets would have cost us a fortune. And of course not every country had techy-aware people there. So it was good to have different options to choose the right one from.

By the way: the Linux-Server site was interesting, because a simple menu hacked together from "dialog", "bash" and "perl" allowed even novice operators to setup up and create users and an e-mail connection. The system was dubbed "OM Standard Linux Server" because it later also became file-server (samba), print-server (lpr) and so on.

However, we had a different problem domain (getting e-mail in countries without internet access, not getting e-mail to remote areas of countries with full area covering internet access). So we did, up to my best knowledge, not use radio based links.

HolgerSchurig has a mailbox at gee-emm-ex point dee-eeh :-)

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Hello Wayne,

I read your article with great interest, as I am writing a white paper on wireless use and options in our US schools for the Consortium of School Networking (CoSN). I would like permission to use excerpts of your piece and to provide a link to the the entire version. Making wireless work in such extreme conditions is an amazing accomplishment, and would give an interesting perspective.

This white paper is to be presented at the Nov. National School Board Administration conference, and will be featured in several ed-tech print and electronic magazines. If you agree to my request, please provide contact information for those who would like to connect with you directly.

I look forward to hearing from you!

Regards,

Kristen Hammond

Wallingford, CT USA

khammond@edupuppy.com

http://www.EduPuppy.com

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Check out the NATO Standard for HF Data Transmission (STANAG 5066) and the newer standards for HF data modems (MIL-STD-188-110B and STANAG 4539). I'd love to see low-cost (no-cost) implementations of these from the open-source community to complement the existing commercial implementations (which are kinda' pricey). But the data modems support rates up to 9600 bps in a standard 3 kHz bandwidth, and support adaptive data rate when the channel goes to crap. The data transmission standard (STANAG 5066) supports e-mail and IP-over-HF using ARQ or non-ARQ modes. All of the advantages cited by the article were identified by the military HF industry as good reasons to develop the standards and equipment. Think of it as long-haul wireless internet and ISP access from the days when 9600 bps telephone modems were hot stuff. With a minimum of costly infrastructure.

Re: Radio E-mail in West Africa: East Africa since late 90's

Anonymous's picture

In Uganda we have had access to email over HF since the late nineties using Codan HF radios and Windows based systems. I did an interesting version using SuSE Linux (6.4 at the time) and wrote a small paper and sent it to Codan (http://www.codan.com.au). I managed to send and receive files to units 500 Kms from the nearest ISP. My paper and corresponding files are available under http://kym.net/codan/ Read the codan.doc

Kiggs

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Interesting. Having used amatuer digital modes on HF, I find

the application of HF for email, etc, refreshing.

However, readers (at least here in the U.S.) should note...

HF is (afaik) totally "closed" to unlicensed experimentation.

No mention of licensing or legal use was mentioned in

the article (afaict). Here in the U.S., presumably you could

get an commercial HF license for a digital mode, but

experimentation is not allowed. Unlike 802.11 (2.4ghz)

there is no unlicensed band for digital use (note that the

old "unlicensed CB" 27 mhz band only allows for AM/SSB or

remote control applications.)

I'm not totally familiar with the geographic topology of

the africa-based sites mentioned in the article, but

bandwidth (and reliability!) could be greatly improved by

going with 2.4ghz (802.11), locating repeaters on

mountain tops, and using highly directional (hence high gain)

antennas. This depends alot of on the topology and

could very well not be feasible for those sites.

Otoh, possibly 2.4ghz is closed in that part of the world

requiring the fallback to international HF frequencies.

jeff

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Jeff I would love to discuss your concept. How much would it cost? Repeaters are the only solution for short distance 802.11 of course now there is WiMax 802.16 I believe. 802.11 goes how far, a kilometre or 5. If 5 is reliable it will take 20 repeaters for 100 kilometres!

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

I was reading this until I got to the description of the PPP link and remembered the days of UUCP over serial lines. Since the modem took care of the error correction they could send much more data more quickly by using straight serial UUCP instead of trying to get a PPP handshake to get TCP/IP working. A UUCP chat script was always faster than PPP in my experience.

Kris

Store and Forward web searches

Anonymous's picture

The author might want to check out TEK. TEK is a store and forward interface to the google API that is designed to work over slow and inconsistent network interfaces.

To quote from their homepage:

Welcome to the TEK Project. The goal of the TEK project is to build a low-connectivity search engine for use by people at the far side of a bad telephone connection.

In many places in the world, there are no books, there is no television, there is no access to information. And just a modem away there are gigabytes of information streaming by that noone can access -- because telephone charges are too high, because ISP fees are 10% of a local monthly wage, or because there are no computers. Even if there were computers and working phone lines, bandwidth is so narrow that searching the Web and downloading the discovered material will take so long as to be financially impossible.

TEK supports a different model of Web access. Think of inter-library loan. You need a book. Your library doesn't have it, and you can't afford to buy it. But you can get it on loan. You are willing to `pay' to access the infomation in that book with time: you will wait a week to see the book.

With the TEK search engine, the user submits a query which is emailed to Boston. The TEK Server, which is connected to the Internet backbone, searches the Web, locates some pages, selects which pages to send back, compresses them, and returns them back to the user. Because the search results are returned asynchronously, by email, the connectivity charges are lower. Post-processing the search results and selecting which pages to send back reduces the amount of information and addresses the bandwidth question.

http://www.cag.lcs.mit.edu/tek/

It may be greater than the bandwidth available but that may change - the on-off nature of connections in the bush won't for many years :)

Cheers

James

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

> Remote networking with high-frequency (HF) radio and

> Dan Bernstein's qmail.

>

>

They're using PPP to establish a TCP/IP connection over HF. While it's a solution, it's not the best solution - using UUCP and eliminating the overhead of PPP and TCP/IP is a much better, more robust, and higher throughput solution for store-and-forward applications. Been there, done that.

Also, it's not necessary to use such a complicated setup such as qmail if you're using UUCP. qmail isn't Open Source - the licensing is rather restrictive, a fact that leads the major distributions to not distribute qmail.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Linux already supports courtesy of the ax25 kernel modules the ability to use amateur ax.25 protocols on HF. By using these with a NOS driver, tcp/ip connectivity is as easy as typing "insmod ax25".

I suspect that the commercially made modems were using sitor/amtor or pactor or variants thereof.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

ax.25 is not particularly good at HF. The channel is too cruddy to support the quite low error rate that ax.25 requires to be successful. HF protocols such as amtor/sitor/pactor and clover are designed to deal with the vagaries of hf channels. clover being probably the fastest and most robust for large block half-duplex communication. But it works best for large blocks. I suggest that you read up on data-over-HF, you will find a lot of interesting and unexpected things drivin by the fading and noise that is unique to an HF channel. de N6NZ

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

I enjoyed the article and respect the ingenuity of what you have done.

But it's pretty ironic that you talk about how enabling "freely available, best of breed, borderless open-source technologies" are, having spent most of your article describing a network that uses the proprietary, limiting and non-free qmail mail transport agent.

I have no arguments against the use of proprietary software where it's necessary. But please, please don't misrepresent proprietary software as something it is not! Calling qmail "open source" is not only unfair to Dan Bernstein, it is also disrespectful to the authors of MTAs like exim, postfix and sendmail who have spent their lives developing software that's truly free to share, use and modify. qmail is a very powerful piece of software, but "open source" it is not.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Interesting. But I didn't pay anything for qmail, so how can you call it non-free? And if qmail is proprietary, how can it be that I have access to the source? And I can distribute patches? Name another proprietary program for which anybody can download the source and distribute patches for it. I think you are stretching the meaning of "proprietary".

A little ironic

Anonymous's picture

I enjoyed the article and respect the ingenuity of what you have done.

But it's pretty ironic that you talk about how enabling "freely available, best of breed, borderless open-source technologies" are, having spent most of your article describing a network that uses the proprietary, limiting and non-free qmail mail transport agent.

I have no arguments against the use of proprietary software where it's necessary. But please, please don't misrepresent proprietary software as something it is not! Calling qmail "open source" is not only unfair to Dan Bernstein, it is also disrespectful to the authors of MTAs like exim, postfix and sendmail who have spent their lives developing software that's truly free to share, use and modify. qmail is a very powerful piece of software, but "open source" it is not.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Wouldn't your solution create a lot of traffic overhead? (PPP, IP, TCP, SMTP and neither of these are efficient protocols.) My spontaneous idea would be to re-use some of that old FidoNet software and set up something similar: store and forward of compressed packets of e-mail directly over serial links.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

UUCP could be an easier choice, and more Unix friendly.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

The author responded to this with:

Although we might have implemented e-mail to the field using UUCP, our TCP/IP system is easier to configure and integrate with the existing network and e-mail system. This setup is also readily converted to use faster telecommunications linkups, such as through landlines or satellites. If and when these become available in the field (and as the budget may support them), the IRC network is ready to simply plug in.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

HF and VHF packet radio have been going on for years in the Amateur Radio Service. It's good to see others not directly involved in radio finally recoginzing the benefits. Check out www.tapr.org.

73 de N8SQT

(Best regards from Bob)

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Greetings Wayne:

That was a super article! I have done similar with less as you are doing right now, so I think you are doing something quite special indeed.

I wonder if you ever considered Amtor or even Sitor as a way of linking over the HF ether? Amtor is a protocol adapted by amateur radio from Sitor, its commercial predecessor. Further, Pactor further enhances Amtor with better compression and frequency instabilities. I believe Pactor and Sitor are available in the public domain.

My thinking is to marry the best of Pactor with DAMA the add a layer of DHCP/CHAP/PPP so your hub could actually sustain more than just one link at a time. DAMA will ensure all new links are welcomed and added to the round table, the ppp(or whatever you use) would authenticate, and Pactor would ensure the best compression verses time verses link state. I don't think I have seen anything like this in production, but it would sure be fun to have a go with. Unlike tcp/ip packet, a broken packet can be repaired without a complete retransmission of the whole packet.

Pactor II is now available and actually does even more with compression and squeezes more data in even less bandwidth and can cope with signals that are almost inaudible. Unfortunately, it is very proprietary and the makers want a lot of money for each license. Yet it will move even more data over less channels than Pactor can.

One other nagging thought kept creeping into my mind: why don't you use either 2 meter or 6 meter single side band radios? This would effectively release the hf rigs for pure voice/emergency calls while moving the data load to that portion of the spectrum where it would be least likely to interfere with other systems. As you know, HF knows no boarders! The VHF spectrum in SSB mode has amazed and delighted many an operator over the years. The modems you now use on the HF radios would work exactly the same on the VHF rigs, and probably a little faster since transmitter to receiver turn around is likely faster on the VHF rigs. I realize this is probably not doable in tight budgets, but it sure would be fun!

Thank you for taking me on this splendid tour of your works. It seemed like you had a lot of fun making all of this system work.

Take care and happy hacking.

a_hippie@linuxfreemail.com

Re: VHF SSB

Anonymous's picture

Would VHF SSB give 375 mile/600 km coverage?

Excellent article, Wayne. I found every word

useful.

Everett

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

My sister works with Wycliffe in Mali. She's translating the Bible for the 60000 Tadaksahak people living in eastern Mali. She also uses a radio link powered by a car battery to send and get email from her place in Menaka to and from the capital city Bamako. The infrastructure is a ccMail system with an internet mail gateway provided by JAARS.

Beat Bolli

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Solomon Islands People First Network (PFnet), a UNDP-funded ICT in Development project, has been using community radio email since October 2001. We use Wavemail and Pactor 2/3 (level 3 is equivalent in throughput to the CODAN). It's also much cheaper and more user-freindly for less skilled operators. We find that a week's training is suficient even if they have no previous computer experience.

http://www.peoplefirst.net.sb/general/pfnet.htm

David Leeming

Technical Advisor

leeming@pipolfastaem.gov.sb

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

This is an interesting application - I'm just curious if the author was familiar with some of the amateur radio modem systems that have been in operation for around a decade now doing basically the same stuff?

HF is very much a different kind of fish compared to 802.11B, etc. Store and forward works pretty good in this application. There are even radio bbs systems that run under dos that automate this kind of thing too.

Nice article.

Re: Radio E-mail in West Africa: The Complete Version

Anonymous's picture

Ham Radio operators have been doing this for YEARS now. Nothing new.

From using simple type of 'modems' like http://www.baycom.org/bayweb/index.htm or using tcp/ip over the air using the baycom modem and software like http://www.fwarig.org/home_jnos.html

Or providing ham radio operators who are sailing around the world access to email as well via HF.

http://www.w2xo.pgh.pa.us/

http://www.hfradio.com/

Of course thats just the tip of it.

HF Automatic Link Establishment - HFLINK

ALE Automatic Link Establishment's picture

HF ALE Automatic Link Establishment is being used for this purpose now. The system enables the best frequency for ionospheric propagation. This maximises the throughput and baud rate.
See the website
http://hflink.com
for more information about ALE Automatic Link Establishment, Robust and High Speed HF Communications.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix