Setting Up UUCP
Setting up mail didn't take much time, and I was feeling pretty smug when I got my first e-mail from “the outside”. Humility came back with a vengeance when I turned my attention to setting up news.
Like mail, a variety of software packages are available for getting news on and off a local machine. Again like mail, the best known is big and intimidating. Unlike mail, the lesser known is also big and intimidating. The “big boy” in news transport is InterNet News (INN). A close second is the C News package. I took a look at the documentation available for these two packages before deciding which to try. INN runs as a daemon and can be a disk and memory hog. While it can be configured to operate in a situation such as mine, it really is intended to be run on dedicated news servers with high-speed connections and many users. C News is a collection of binaries and scripts that are executed by cron and consumes minimal disk resources. While there's more documentation available for INN, I decided that C News was the way to go. Besides, I just wouldn't feel right as a Linux user if I could just follow directions and have a working system.
C News is sufficiently system-dependent that compiling from a source distribution is the only logical thing to do. The current release of C News can be found at ftp.cs.toronto.edu in /pub/c-news/c-news.tar.Z. After uncompressing and extracting the source tree, take a moment to read through the files in the docs/ subdirectory; there's some interesting material in there.
To configure C News for your system, follow the instructions in the file README.install. Before getting started, edit the file conf/update.ran. It lacks an initial line of #!/bin/sh and will cause the Makefile to exit with an error. After reading through the documentation, begin installing.
Run “quiz”. This brief interactive script sets up the defaults for C News based on your answers. In most circumstances, either the default is fine or you will have an intuitive grasp of a preferred alternative (such as deciding to store articles in /var/spool/news instead of /usr/spool/news). However, there are a few things to look out for:
The default for where to store the news control files is /etc/news. There are a handful of files in this directory that get pretty big, so if you have a small root file system you might want to put these on another file system.
quiz will ask about the presence or absence of 13 system functions. On my Red Hat 3.0.3 system, all but fgetline were present.
quiz will build your makefiles for you, so it needs to know what type of syntax to use for specifying include files. Specify SVR4.
Make sure you specify the same style of UUCP configuration files you used in setting up UUCP, HDB or Taylor.
Specifying the paths to rnews and inews: The paths to these programs should match the command-path line from your UUCP sys file.
quiz will ask in what format the system reports free disk space. I took a wild guess, considering Linux's similarity to SVR4, and answered statfs. Judging from the working setup, I seem to have guessed correctly.
Run make to build the software, then make r to run some regression tests to check out the build.
Create the directories you specified in quiz for the article tree, the overview tree, the binaries, and the control programs. C News calls these NEWSARTS, NEWSOV, NEWSBIN and NEWSCTL, respectively. I put the articles and the overviews in /var/spool/news, the binaries in /usr/lib/news, and the control programs in etc/news. The ownership of /NEWSBIN is largely arbitrary, but the other three should be owned by the /news administrator. I made that user news, group news. If you don't have a news user, create one with adduser or whatever tool you prefer. Some newsreaders look for news in usr/spool/news, so you might need to create a symbolic link in /usr/spool to your NEWSARTS directory.
Use su to log in as the news owner and run make install. You might have a permissions problem trying to write the binaries to NEWSBIN. I got around that by switching virtual consoles, giving everyone write permission to that directory, running make install as news, then restoring the original restrictive permissions. Run make setup next as news.
As the owner of NEWSBIN, run make ui.
C News comes with a news reader, poster and checker which can be best described as functional. You will not want to use these beyond testing your setup, but it's worth installing them for that purpose. Do this by running make readpostcheck.
Make sure NEWSBIN/input/newsspool is owned by news and in the news group. Change its permissions to rwsr-sr-x.
As news, change directories to NEWSCTL and configure the important control files for your particular setup:
batchparms: By default, C News will try to deliver articles as soon as they arrive. For our setup, it's better to collect the articles into a single compressed file and deliver it periodically via UUCP. This is called batching and the file batchparms sets up the size and number of batches. Here's mine:
# NEWSCTL/batchparms # 500KB, after compress, is 4 minutes at # 1000cps 20 batches is somewhat arbitrary, # about 5MB per site # site class size queue command # - -- - -- -- sloth u 500000-750000 20 batcher| compcun | viauux
As with mail, sloth is my ISP's UUCP host. The u class option specifies that batches will be transferred via UUCP. The size of a batch is set to 500KB nominally, with a maximum size of 750KB. If a single value is specified, it is the nominal value, with the maximum being 3 times the nominal. The queue field allows up to 20 batches enqueued. The pipelined command in the final field specifies what to do with a batch. The batcher takes the spool of outgoing news articles and assembles them into batches of nominal size. It passes the batches off to compcun, which compresses the batches. By default, compcun uses the compress function which, while not as efficient as GNU's gzip, is certain to be understood by any system. If you feel passionately about using gzip, contact your ISP to see if his system has gzip installed. (Don't laugh, some ISPs use software from their hardware providers and refuse to install any Free Software Foundation products. Now I'm serious: stop laughing.) compcun now hands off its compressed articles to viauux, a C News front end to UUCP, which delivers the compressed and split articles into the UUCP spool for delivery. Incoming news goes through the reverse process if it is batched: delivery by UUCP, decompression, unbatching, and off to the news spool.
controlperm: The entries in controlperm specify who and what sites are allowed to send control messages for what groups. The example file included with the C News source is pretty self-explanatory, so I'll let this topic go with a word of warning: if you're running C News over a UUCP link from your Linux box, you are not important enough to send out newgroup or rmgroup control messages. Don't do it.
explist: Unless you have a magic hard drive, your news spool will soon fill up if you don't throw away (“expire”) the older articles. Every day or so, you should run the program doexpire, which checks the explist file to see how long to keep articles in the spool. Entries in explist take the form:
grouplist perm times archive
The grouplist is a comma-separated list of newsgroups to which the line is to apply. Subhierarchy expansion is automatic, so to expire the entire rec.* hierarchies, a grouplist of rec is sufficient. The perm entry indicates the type of group: u = unmoderated, m = moderated, x = either.
The times entry is generally a single number, though it can be a dash-separated list of three. If the former, the entry indicates how many days after arrival at your site an article should be kept. Of course, if the article itself contains an Expires: header which evaluates to an earlier date, it will expire then. If the times entry is a trio of numbers, the first represents the number of days that must pass after arrival before the article is considered “expirable”. I'm sure someone could come up with a reason why a number other than zero is desired, but I'm at a loss. The second number is the days-to-expiration as seen in the single-digit format, and the third is an absolute number of days the article is permitted to dodge expiry.
The archive field takes a dash if no archiving is to be done, a full path name to an archive file, or an @ sign. If @ is used, the archive file must be specified with the -a option when doexpire is run.
# NEWSCTL/explist #grouplist perm times archive # hold onto history lines 14 days, nobody gets # >90 days /expired/ x 14 - /bounds/ x 0-1-90 - # High-traffic or trivial groups get dumped # quickly control,junk,swcp,perrin x 3 - # default: 14 days all x 14 -
Note that the earliest match is accepted, so when doexpire sees the 3-day limit on the control group, it takes precedence over the 14-day catch-all at the end. If you want to mix full group names (such as rec.arts.tv) and hierarchies (such as rec.arts.*), remember to put entries into the explist file in increasing generality.
A final comment: the /expired/ grouplist tells doexpire how long to leave article entries in the history file. For multiple feeds, duplicate articles may arrive from time to time. C News keeps a list of articles in the NEWSCTL/history file and junks any duplicates. Lines in the history file are kept the number of days specified by /expired/ so that if a duplicate arrives a few days after the original has expired, it will not be reposted. If you have a single feed which is properly configured and doesn't send multiple copies out to its leaf sites, you can keep this value small.
mailname: Self-evident. My mailname file contains the single line:
# NEWSCTL/mailname perrin.swcp.com
mailpaths: Posts to moderated newsgroups are mailed to the moderator, who decides to post it or not. The mailpaths file has a line entry for each moderated newsgroup consisting of the group name, a tab, and the e-mail address of the moderator. The only moderated group I read is comp.os.linux.announce, so my mailpaths file is simply:
# NEWSCTL/mailpaths comp.os.linux.announce email@example.com
organization: This is just a string containing the organization you are posting from. For a home site, an entry like “Private News Server” is adequate, though you might give into temptation and come up with something dazzlingly funny.
sys: sys is the most important file for talking to the outside world; it specifies what hierarchies you get, from where, and what news you are willing to pass on. For our special situation, only two lines will be needed. Entries take the form:
Any article coming from site not in the (comma-separated) grouplist is junked. The distlist is a (comma-separated) list of distributions we are willing to forward. Any article whose Path: header contains a machine in the list of exclusions is not forwarded.
There are quite a few flags that can be set, but only one is relevant. The f flag tells C News to use batching. If you are using batching, the cmd entry contains not a command but a file name. If the file name does not begin with a slash, it is relative to /var/spool/news/out.going. If the cmd field is empty, the file is remote-system/togo.
With this in mind, let's take a closer look at my sys file:
# NEWSCTL/sys ME:all/all:: sloth/sloth.swcp.com:all,!perrin/all,!local:f:
I gave my ISP a list of the newsgroups I wanted fed to me; he entered them into a configuration file on his end. As a result, I can safely use all/all in the ME entry and not worry about trying to download an entire Usenet feed at 14.4kbps.
The sloth entry again specifies my ISP's machine. By putting sloth.swcp.com into the exclusion list, I don't try to send articles back. I created a perrin.* hierarchy for some simple tests; the grouplist entry in my sys file shows I send all newsgroups except perrin up to my ISP. Similarly, articles intended for all distributions (world, usa, nm, whatever) except local are sent up. I will be using batching, and the articles are going to be stored in /var/spool/news/out.going/sloth/togo.
whoami: The machine's hostname—in my case, perrin.
Return to the C News source directory and make cmp to see that everything is correctly installed and configured. You'll undoubtedly want to pause briefly, get up from your computer, and do a little dance when you see the message “no worrisome differences”, for the end is near.
Time to automate! As news, create/edit a crontab with crontab -e. The entries here will set up a schedule of housekeeping tasks and transfers. Here's mine:
#/var/spool/cron/news 09 * * * * /usr/lib/news/input/newsrun 22 * * * * /usr/lib/news/batch/sendbatches 59 0 * * * /usr/lib/news/expire/doexpire 17 8 * * * /usr/lib/news/maint/newsdaily 05 * * * * /usr/lib/news/maint/newswatch 3000 300 100
At 9 minutes past the hour, newsrun takes the batched articles delivered by UUCP, unpacks them, and stores them into the news spool.
At 22 minutes past every hour, sendbatches takes the outgoing articles, batches and compresses them, and delivers them to the UUCP spool for uploading. These files will be transferred whenever the next UUCP connection takes place, so su to uucp and edit the crontab, as you did with mail.
At 12:59 AM, doexpire takes care of expiring old articles.
Every morning at 8:17, newsdaily tidies up some of the various log files and sends mail to the news admin if something is amiss.
At 5 minutes past the hour, newswatch takes a look at the system to see if there are any problems. Stale locks, full disks, and the like are reported to the newsadmin.
You need to have newsboot run at system bootup to clean up any messes left over after a potential crash. To preserve permissions, it needs to be run as user news. I tacked the line:
su news -c "/usr/lib/news/maint/newsboot"
onto the end of my rc.sysinit file.
Installing the man pages. Due to wild system variations, your best bet is just to change into the C News doc/ directory and copy them manually into your man page subdirectories.
Create a local test group by:
cnewsdo addgroup my-site.test y
and post to it. If you made readpostcheck you can do this with the postnews package. Look in NEWSARTS/in.coming and you should see the article, with header information thoughtfully provided by postnews. Leave the computer room and share some time with your family until after newsrun executes again. Now, your article should have appeared in NEWSARTS/my-site/test and should also have entries in the history and log files in NEWSCTL. If you installed readpostcheck, use readnews -n my-site.test and you should see your message. Pat yourself on the back; you've got yourself a Netnews site.
Stop patting yourself on the back. Unless you plan to spend your time sending messages to my-site.test and reading them a bit later, you'll want newsgroups from the Outer World. The installation directions that come with C News cover this step with “Get a feed from somebody, somehow.” That's best handled by sending your ISP a list of newsgroups you want to receive and asking for e-mail when it's set up.
As news, edit the files NEWSBIN/active and NEWSBIN/newsgroups. They should both have permission modes 644.
active contains the list of newsgroups, their article numbers, and the posting permissions. Each line should be of the form:
news.group.name highest lowest perms
:Highest and lowest are the upper and lower bounds of available articles in news.group.name. For setting up, set highest to 00000000 and lowest to 00001. To save time, C News never updates the lowest value, so if your newsreader (such as trn) looks at lowest, you will need to set up a crontab entry to run updatemin from time to time. The relevant perameterss are y: users can post to the group; n: users can read but not post; m: the newsgroup is moderated. Posts to this newsgroup will actually be sent to the moderator address specified in the $NEWSCTL/mailpaths.
As news comes in, C News will continually update the active file. Here's a snippet of mine:
alt.tv.homicide 0000000002 00001 y alt.tv.nypd-blue 0000000010 00001 y comp.os.linux.announce 0000000009 00001 m comp.os.linux.misc 0000000504 00001 y linux.dev.kernel 0000000048 00001 y rec.arts.sf.written.robert-jordan 00033 00001 y rec.arts.tv 0000000334 00001 y swcp.test 0000000005 00005 y
In addition to “real” newsgroups, you should also have groups for control, junk, to.feed-site and to.my-site.
This file can be maintained by hand or by using addgroup and delgroup. Second Warning: do not send newgroup or rmgroup messages; these commands create groups for all of Usenet and are not for the likes of us mortals. As much as you might want to newgroup alt.fan.back-hair on your own initiative, procedures exist for creating new Usenet groups and violating them is a good way to find yourself unpopular.
Something I noticed after a day or three of running C News: control messages were responsible for about 80% of my news spool. To cut back on disk space, I had my ISP remove control from the list of groups he was feeding me. I can no longer receive newgroups, rmgroups or cancels, but it's worth it to save disk space. Forty non-binary newsgroups, most low-traffic, were eating up nearly 8MB of my disk spool after only three or days. Multiply that by the control-group factor of five and you can see why my poor 50MB spool can't afford control messages. Make sure your ISP still accepts control from you, however, as you might need to cancel a stupid question or hastily-posted flame someday.
newsgroups contains a list of the newsgroups which appear active and a brief description of each. When a user reads a new newsgroup, this description should appear to provide a little newsgroup information. This can be handy for keeping pyrotechnicians from wandering into places like alt.flame. I made my newsgroups file by trimming a copy of my ISP's.
Final Step: do include a local test group in your feed—your ISP probably has one. Post a message to it, wait for batching and UUCP to do their things, then see if it shows up. If so, post a test message to one of the “real” newsgroups. Disguise the fact that it's a test message by making it relevant and on topic. If it shows up, you're in business. Sit back, relax, have a Coke and smile.
Jim Hill is a graduate assistant at Los Alamos National Lab who has been playing with Linux since somewhere in the .99 kernel series. Armed with total ignorance of the topic of this article, he set out to learn a little something and maybe help somebody else. Thanks to Mark Costlow at Southwest Cyberport (firstname.lastname@example.org), without whose help and patience this article would have read “It doesn't work; it can't be done.”
- NSA: Linux Journal is an "extremist forum" and its readers get flagged for extra surveillance
- Tails above the Rest, Part III
- Numerical Python
- Tails above the Rest: the Installation
- Dolphins in the NSA Dragnet
- Are you an extremist?
- Tails above the Rest, Part II
- A Bundle of Tor
- Tech Tip: Really Simple HTTP Server with Python
- RSS Feeds