Hack and / - Mutt and Virtual Folders

If you didn't already know, I'm a mutt-addict^H^H^H^H^Huser. At this point, I can't even remember when I started using mutt. All I do know is that every time I try another mail client, I just get frustrated by how long it takes to go through my e-mail. Well, that, plus try to navigate a GUI e-mail program with vi key bindings—it (usually) doesn't work.

Why try other mail clients if I love mutt so much? Well for one, some of the other mail clients do have a few interesting features, such as virtual folders. With a virtual folder, you can create a keyword search, and then all messages that match the search end up in a special folder you can browse. Well, it turns out, the same functionality is available in mutt if you use maildirs and are willing to do a little scripting.

One major problem with the default search abilities in mutt is that you can search within only one folder at a time. I store just about everything that's important in e-mail, and I have many different folders with even more procmail rules to sort incoming mail between them. The downside to this is occasionally I can't remember exactly in which folder a particular e-mail message is located.

The solution to the mutt search problem takes advantage of the fact that if you use maildirs on your mail server, each folder is a directory on the server, and each e-mail message is a file within that directory. Basically, a script can go through each of your folders and grep for your keyword and then create a new maildir with symlinks to any matching files. I named my solution maildirsearch, and it looks something like this:

#!/bin/sh

MAILDIRS="$HOME/mail"
VFOLDER="search"
VFOLDERPATH="$MAILDIRS/$VFOLDER/cur/"
FOLDERS=`ls $MAILDIRS | egrep -v "search|flagged"`

rm -f $VFOLDERPATH/*

for i in $FOLDERS
do 
   for j in `egrep -lR "$@"  $MAILDIRS/$i`
   do
      ln -s $j $VFOLDERPATH/;
   done;
done;

Next, I created another script called muttsearch that would execute my maildirsearch script, then open a new instance of mutt that reads the new virtual search folder. One nice feature of using this method is that I can see the search results grow within the mutt window and start reviewing results immediately:

#!/bin/sh

VFOLDER="search"
$HOME/bin/maildirsearch "$@" &
sleep 1;
mutt -f "=$VFOLDER" && killall maildirsearch >/dev/null 2>&1

Finally, I set up a key binding in mutt so that I could press Esc-S, type in a keyword (or regex) and start the search:

macro index \eS "<shell-escape>$HOME/bin/muttsearch \""
macro browser \eS "<shell-escape>$HOME/bin/muttsearch \""

Now, there is a downside to this script—it's designed to be run on the server that stores the messages. In my case, I use a tool called offlineimap to sync my remote e-mail server with my laptop, so I always have a copy of my messages locally. If you don't want to go that route, but want to be able to search from your local machine, you potentially could modify my muttsearch script so that it SSHes into your mail server and executes the script.

The script actually works well for me, but I realized after some time that I did a lot of the same searches over and over again. For instance, I am a heavy user of the “flag for follow-up” function in mutt. If you are in the message index and press the F key, it sets a flag on the message and displays an exclamation point in front of it. I use this to remind myself that I need to reply to a message. The downside to this is that I have to go to a particular folder regularly to see the flag, and sometimes I want to see all of my flagged messages at once. It turns out that the same virtual folder concept I used for my search works well for this too.

First, I created a script called flag-folder. When you flag an e-mail in a maildir system, the e-mail gets an F added after the last comma in the filename. Basically, my flag-folder script searches through all my maildirs for files that match the pattern and then symlinks those files into a new maildir named flagged:

#!/bin/sh

# This script finds all the flagged e-mail in a 
# Maildir and symlinks them to a 'flagged' folder 
# in the Maildir. 
# To run it every 5 minutes, for instance, add the 
# following to the user's crontab:
#
#       */5 *    * * *   /home/greenfly/bin/flag-folder

MAILDIR="$HOME/mail/"      # path to your maildir
FLAGGED="${MAILDIR}/flagged/cur" # path to your 
                                 # (precreated) flagged folder

cd $FLAGGED
rm ${FLAGGED}/*

# find all the files in the maildir, then search for 
# flagged files (files with an F after the last comma 
# in the filename) and symlink them
find ${MAILDIR} -type f | perl -ne '$foo = ""; $foo = 
 ↪(split ",", $_)[-1] if(/,/); if($foo =~ /F/){chomp; 
 ↪system "ln -s $_ .\n";}'

As you can see in the comments in the script, you also simply can set this up as a cron job on your mail server so that your flagged folder is updated constantly. Then, I created a script similar to my muttsearch script called muttflag that basically does the same functions, except for the flag-folder script:

#!/bin/sh

VFOLDER="flagged"
$HOME/bin/flag-folder &
sleep 1;
mutt -f "=$VFOLDER";

Finally, I created another key binding so that pressing Esc-F executes the muttflag script:

macro index \eF "<shell-escape>$HOME/bin/muttflag \n"
macro browser \eF "<shell-escape>$HOME/bin/muttflag \n"

There are a lot of other possibilities when you use virtual folders in mutt. So far, the only big downside I have found is that because the script uses symlinks, any flags applied to messages within virtual folders don't apply to the real message. For instance, if you reply to a message within a virtual folder, it updates the filename for the symlink, but not the file it links to. Essentially, this means you lose that reply flag unless you go to the original folder and reply.

Apart from the downsides though, I'm pretty happy with virtual folders in mutt. Once again, it's faster than searches I've seen in other clients, plus I get to keep my vi key bindings, which you can have when you pry them from my cold, dead IBM Model M keyboard.