Getting Rid of Spam
Rather than having all that garbage clog up your in-box and make it unusable for real work, you can now use procmail to filter it out. Earlier I mentioned that spammers try to obscure headers to make it hard to trace. By doing so, they sometimes give inadvertent “signatures” that you can tell procmail to filter on. For instance, a popular bulk e-mailer, the Stealth Mailer, inserts a false Received: line to deter flames. However, both versions generate the wrong time zone. Armed with this knowledge, you can now filter out a great deal of spam. I have yet to see a false positive on this one.
# Filter spam that used the Stealth Mailer Classic :0 * ^Received:.*id GAA.*-0600 \(EST\)$ spam
Another great spam filter looks for a “Comments: Authenticated sender is” header. Unfortunately, filtering on that alone does not do the trick because Pegasus Mail (a popular mail client for the Windows operating system) uses this header legitimately. Fortunately, Pegasus adds an X-Mailer: header in addition to the Comments: header. If both the Comments: and the X-Mailer: exist, then a Pegasus Mail user sent the message (and is probably legitimate); otherwise, it is a bulk mailer. The following recipe will filter this situation. (Note that there is a space and a tab between the square brackets. Unfortunately, procmail does not have a whitespace escape sequence as Perl does.)
# Only Pegasus Mail for the WinOS generates a # valid "Comments: Authenticated sender is ..." # header. If this is present and the X-Mailer is # not; then the message in the question is almost # certainly spam. :0 * ^Comments:[ ]*Authenticated sender * !^X-Mailer: Pegasus Mail spamThese two recipes alone filter out a majority of my spam. You can quickly see that a list of these recipes strung together would be beneficial. This is exactly what several free packages have done. My personal choice came down to Alcor's filters (http://alcor.concordia.ca/topics/email/auto/procmail/spam), which I found to be non-intrusive, easy to understand and quite flexible. Alcor's filters work by applying over 1300 filters to the message. If a filter is matched, the message is tagged with a special header. Then, all you have to do is take whatever action (e.g., delete, write sender, etc.) you deem appropriate for these messages with the special headers. I personally avoid “reply” because I dislike using auto-responders, and “delete” because I believe in checking for false positives (of which you will unavoidably get a few).
I recommend downloading all of the tag recipes (use the “save as source” [not text] feature on your browser). I placed the filters in a new directory, cleverly called ~/.procmail. You will most likely need to edit the file tag-radical in order to comment out (using a # at the beginning of the line) or change the three uncommented INCLUDERC lines. Otherwise, you will see annoying “Couldn't read xxxx” errors in your $LOGFILE each time you process a message. Once that is done, add the following recipe in your ~/.procmailrc at the point you wish to check the incoming message for spam. I check mine at the very top before I do any kind of filtering and have found that this works well.
# This enables Alcor's tagging filters INCDIR=$HOME/.procmail INCLUDERC=$INCDIR/tag INCLUDERC=$INCDIR/tag-agis INCLUDERC=$INCDIR/tag-aol INCLUDERC=$INCDIR/tag-contents INCLUDERC=$INCDIR/tag-jdfalk-cyberpromo INCLUDERC=$INCDIR/tag-jdfalk-llv INCLUDERC=$INCDIR/tag-jdfalk-nancynet INCLUDERC=$INCDIR/tag-panix INCLUDERC=$INCDIR/tag-radical :0: * $ ^$special_header spam
That is all. If everything goes well, you should notice that most (if not all) of your spam now goes into your mailbox named spam. You can test it by sending a message to yourself that contains content that these filters will catch (try sending yourself a message with -- Headers -- somewhere in the body).
Alcor's tagging system might catch legitimate mail, so I do not recommend deleting anything before you look at it. Once you have verified that it is spam, you have two options: complain or delete. If you want to fight spam, I recommend you to read the SPAM-L FAQ (http://www.ot.com/~dmuth/spam-l/) and possibly join the mailing list. Instructions on how to do so are in the FAQ.
This article is only the tip of the iceberg on using procmail and its accompanying programs. If you are interested in the continued use of procmail to filter your e-mail, I recommend the procmail mailing list. The regulars there are knowledgable and willing to help. You may also want to search out other procmail solutions. To put these filters through a stress test and to help further develop them, I have subscribed to a special mailing list that sends nothing but spam that is forwarded through it, which takes special care to try and filter duplicates. At the time of this writing, 83% of mail I have received from this list was properly filtered.
Brandon M. Browning is a Software Engineer for NorthWestNet, Inc., an ISP located in Bellevue, Washington. When he is not hacking Perl or fighting spam, he can often be found pursuing his other interests: The Tick, Babylon 5, Star Wars and on occasion sleeping. He can be reached by e-mail at email@example.com.
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
|Introduction to MapReduce with Hadoop on Linux||Jun 05, 2013|
- Containers—Not Virtual Machines—Are the Future Cloud
- Non-Linux FOSS: libnotify, OS X Style
- Linux Systems Administrator
- Validate an E-Mail Address with PHP, the Right Way
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Introduction to MapReduce with Hadoop on Linux
- RSS Feeds
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?