A Guide to Using procmail at Halcyon^H^H^H^H^H^H^Hany ISP with procmail

procmail is a wonderful anti-entropy device which miraculously sorts an amorphous plate of breakfast fare into sausage, eggs, tofu and spam, hopefully without losing too many beans or tossing them at others in the process

Introduction

NEW: Nothing really new to report. I wrote a web-based document server gateway a few weeks ago (read the docs with a web browser, update the archive via e-mail). I've discovered that Python has some pretty good libraries for dismantling e-mail messages... yes it can be called from Procmail. Is there enough interest that I should bother putting something up?

This guide will tell you some of what procmail is capable of doing for you, and point out some specific issues with using procmail. It was originally written whilst I had an account at halcyon.com, and may be referenced as A Guide to using procmail at Halcyon.

When it was first created (a lifetime ago in internet years!), there wasn't much out there; at this point, there are some pretty good guides, so I'd encourage you not to treat this one as your final destination.

I wrote this locally-run JavaScript-based form that will generate a csh script you can paste into a shell session to configure procmail. You don't have to paste the script into a shell session, that's entirely up to you. Or, edit the generated shell script so that it writes to something other than .procmailrc and .forward, and use it to generate sample code. I added an explanatory note about that, too.

Bugged spam remains an ever-festering abscess on the backside of what is the commercial internet. You just gotta love that easy-to-use, commercial mentality. It's kind of like crack.

What is procmail?

Procmail is a mail processor. You can use it to filter mail or to autoreply to certain messages. You can use it regardless of whether you use a unix shell mail reader such as pine or a mail client such as Eudora. Procmail is only one of a group of programs and utilities which can work together to manage, filter and generally process e-mail.

Is procmail right for me?

Procmail is a serious unix hack. On the other hand, it does a pretty good job. If you don't know a thing about coding, you probably don't want to try to set up procmail.

If you know how to log into the unix shell, use pico or vi, ls and chmod you can probably set up procmail and use a simple .procmailrc (procmail rules) file. You may get that file from someone else; people are starting to share the responsibility of maintaining procmail rules files.

Using procmail

A couple of different things need to be taken care of in order for you to use procmail to filter your incoming mail. Many parts of the following recipes are hyperlinked to relevant portions of the man pages.

The steps involved

In order to use procmail you will need to:

1. Find or create a .procmailrc rules file.
2. Modify or create a .forward file so that your incoming mail is automatically processed.

A simple .procmailrc

A very simple .procmailrc might look very similar to the following:

:0
* ^FROM.*cyberpromo
/dev/null

:0:
* ^Subject:.*surfing
./mail/surfing

What this file does is to send anything coming from the cyberpromo domain to the Big Bit Bucket in the Sky, and collects everything with "surfing" in the subject line into a mail folder in the pine mail subdirectory.

Note the second ":" on the second rule? That throws a lock on the mailfile. You only need (or should, really) do that when the rule writes mail to a file.

Analysis of a .procmailrc file

This is similar to what my .procmailrc file really looks like.

I wanted to accomplish several things. There were certain domains I wanted to suppress. I use an alias for my account when I post to usenet, and in my web client software. I wanted to trap things sent to that address, I wanted to do a little more "handshaking" with the sender to make sure that there really was somebody there and not some 'bot scanning usenet postings for addresses to spam. My overriding goal was to reduce the volume of stuff that needed to be downloaded to Eudora.

The complete uncommented version of this file is available here.

First off, I don't know exactly why but with my account at Halcyon anyway I couldn't pipe outside of the procmail stream itself without inserting the following line at the beginning of my .procmailrc:

SHELL=/bin/csh

It doesn't seem to hurt to do this as a superstitious practice most anywhere.

As I mentioned, I use an alias "fredm3047" in my web browser and when posting to usenet. I call this alias my "cookie". I wanted to catch anything addressed to the cookie and do a number of things to it.

The rule below looks for stuff addressed to "fredm3047" that doesn't have a special keyword, in this case "I-ACK" somewhere on the subject line.

:0
* ^TOfredm3047
* !^Subject:.*I-ACK
{

If they send something to this address without the special keyword, I want to send them back an automatic reply telling them what the special keyword is. The text of the autoreply comes from the file cookie-note.txt.

    :0 h c w
    * !^FROM_DAEMON
    * !^X-Loop: m3047@halcyon.com
    | ( formail -r -A"X-Loop: m3047@halcyon.com"    \
                   -I"From: fredm3047@halcyon.com" ;\
        cat cookie-note.txt ) | $SENDMAIL -oi -t

My experience is that most of the e-mail I get at this alias is junk. Nonetheless on the offchance that it's something I care about, I want Eudora to fetch the message but with all but the first two lines of the body of the message deleted.

If I want the message, then I can reply personally and request it.

    :0 f b w
    | head -2
}

I subscribe to a number of mailing lists. These usually don't specify my address anywhere in the To: information, a trait shared with a lot of junk mail. But I do know the domains where they come from or the name of the list they're addressed to. So I look for that information and insert a gratuitous header line into the message that identifies it as a message from a known mailing list.

:0 f
* ^To:.*MEME
| formail -A"X-Mail-List: MEME"

:0 f
* ^TOwednet|^FROMwednet
| formail -A"X-Mail-List: WEDNET"

:0 f
* ^To.*slime
| formail -A"X-Mail-List: SLIME"

At this point I'm ready to filter out junk mail to my "real" address. Anything not explicitly addressed to me that hasn't been tagged as coming from a mailing list is deemed to be junk.

:0
* !^TO.*m3047
* !^X-Mail-List:
{

I keep a copy in a file on my ISP's server. This file, because it's in the ./mail subdirectory of my account, shows up as a folder in pine.

    :0 c:
    ./mail/junk-mail

I go ahead and let Eudora download the headers, but before I do I add a header line that identifies it as junk mail, and I replace the body of the message with the single word "junk".

    :0 f h w
    | formail -A"X-Junk-Mail: Yes"

    :0 f b w
    | echo "junk"
}

Setting your .forward file

In order for procmail to be called, you have to create a .forward file. So-called because you can use it to forward your e-mail to a different account; so for instance if you have multiple e-mail accounts, you don't have to log into them all to check your mail, you can have all of the mail from various accounts forwarded to a single one.

However you can also specify a program to be called when mail arrives. That's what we're doing here. Substitute your account name for your-account in your-account@halcyon.com.

Use pico or vi (or your favorite editor) to create a file named .forward in your login directory. Typically this file will have one line:

"|exec /usr/local/bin/procmail #your-account@halcyon.com"

Make sure that the file is world-readable, but is only writeable by you:

[coho]> chmod u=rw,go=r .forward
[coho]> ls -l .forward
-rw-r--r--  1 m3047          39 Oct  2 08:02 .forward

Documentation

procmail.org is, well, the definitive starting point for all things procmail! The archives and FAQs from the discussion lists may answer a lot of your questions.

Era is too modest, and doesn't want to be recognized as an "authoritative" resource; but I see the web logs, and I read the e-mails. ;-) This is Era's procmail site in Finland.

There are several sources of documentation available. The Filtering Mail FAQ is a overview of mail filtering that covers other methods as well as procmail.

Panix's E-Spam page might be informative, and has a link to a filter they supply to their own customers.

On your system itself, documentation for procmail may be available through the localman command. So, if man doesn't work, try localman. The specific man pages that you'll find most interesting and edifying are

• procmail, the procmail program itself
• procmailrc, the guide to the commands and syntax of the .procmailrc file
• procmailex, examples and recipes
• formail, the mail formatter often used in recipes

[coho]> localman procmail
[coho]> localman procmailrc
[coho]> localman procmailex
[coho]> localman formail

Troubleshooting

You may want to review the procmailex manual especially, as it includes several recipes for saving off your mail while you're debugging your .procmailrc file.

You may also find inserting the following two directives at the beginning of your .procmailrc to be useful:

LOGFILE=procmail.log
VERBOSE=yes

Both of them are documented in the procmailrc manual.

I proffer this note from http://www.lns.cornell.edu/public/COMP/mail-upgrade.html:

There's an additional subtlety to filtering mail to different dropboxes--if your INBOX is in mbx format (which it is by default), any new folders created via IMAP will be in mbx format, and you cannot append messages to an mbx format mailbox. Therefore, if you wish to use a filter to append to a folder, do not create it via IMAP--let it be created by the filter program, and it should stay in Unix From format.

rev. date: 04-Mar-2005

rev. by: Fred Morris