Anatomy of Postfix

Developed with security and speed in mind, Postfix has become a popular alternative to Sendmail. The Book of Postfix published by No Starch Press is a complete guide to Postfix whether used by the home user, as a mailrelay or virus scanning gateway, or as a company mailserver. Practical examples show how to deal with daily challenges like protecting mail users from SPAM and viruses, managing multi

Please note that this book is based on is based on Postfix 2.1, later versions of Postfix have new daemons not discussed in the book.

This chapter describes how Postfix works, what each piece of the system does, and how these components relate to each other. After going through this material, you should have an understanding of Postfix as a whole, so that you can focus on individual goals.

Postfix consists of a small number of programs that interact with user processes (sendmail, postqueue, postsuper, and so on) and a larger number of programs that run in the background. Only the programs that run in the background are controlled by the master daemon. The master daemon's job is to determine what work there is to do and dispatch the appropriate program to do the work. This modular design allows for a higher level of security because each program runs with the lowest privilege set needed to fulfill its task.

You can think of the whole Postfix system as a router. This may sound strange at first, but remember that a router's job is to look at an IP packet, determine the destination IP address (and possibly the source), and then choose the right interface to route the packet toward its destination. Postfix does the same thing with mail (see Figure 5-1), looking at the destination of a message (the envelope recipient) and the source (the envelope sender) to determine the application that will move the message closer to its final destination.

Postfix works like a router

Now let's look more closely at the system. A real router usually accepts IP packets from multiple interfaces, routing them back out through the interfaces. The same is true for Postfix; it accepts messages from multiple sources and then passes the mail on to multiple destinations. A message's origin may be the local sendmail binary or an SMTP or QMQP connection. The destination can be a local mailbox, outgoing SMTP or LMTP, a pipe into a program, and more. Figure 5-2 shows this view of Postfix.

A Postfix "router" accepts and establishes all kinds of connections

The origin and destination of a message seem clear enough, but how does Postfix pick a delivery method given a destination? A router uses routing tables that match IP addresses to networks to determine a path. Postfix does the same thing with email addresses.

In Postfix, lookup tables are called maps. Postfix uses maps not only to find out where to send mail, but also to impose restrictions on clients, senders, and recipients, and to check certain patterns in email content. Figure 5-3 shows where the maps--to name but a few, aliases, virtual, and transport are shown--fit in.

Maps are the lookup tables of the Postfix "router"

Postfix Daemons

Figure 5-4 shows an overview of the Postfix daemons and how they fit together.

Figure 5-4: The relationships between the Postfix daemons

Note:

Postfix is constantly under development. The following list of daemons is based on Postfix 2.1.

master

The master daemon is the supervisor of Postfix, and it oversees all other Postfix daemons. The master waits for incoming jobs to be delegated to subordinate daemons. If there is a lot of work to do, the master can invoke multiple instances of a daemon. You can configure the number of simultaneous daemon instances, how often Postfix can reuse them, and a period of inactivity that should elapse before stopping an instance.

If you have ever worked with the inetd server on a Unix machine, you will find many similarities between it and the master daemon.

bounce and defer

A mail transfer agent must notify the sender about undeliverable mail. In Postfix, the bounce and defer daemons handle this task, which is triggered by the queue manager (qmgr). Specifically, the two event types that cause sender notices are unrecoverable errors and a destination that is unreachable for an extended period of time. The latter case results in a delay warning.

error

The error daemon is a mail delivery agent like local or smtp. It is a delivery agent that always causes mail to be bounced. Usually you don't use it unless you configure a domain as undeliverable by directing mail to the error delivery agent. If a mail is sent to the error daemon it will inform the bounce daemon to record that a recipient was undeliverable.

trivial-rewrite

The trivial-rewrite daemon acts upon request by the cleanup daemon in order to rewrite nonstandard addresses into the standard user@fqdn form.

This daemon also resolves destinations upon request from the queue manager (qmgr). By default, trivial-rewrite distinguishes only between local and remote destinations.

showq

The showq daemon lists the Postfix mail queue, and it is the program behind the mailq (sendmail -bp) command. This daemon is necessary because the Postfix queue is not world-readable; a non-setuid user program cannot list the queue (and Postfix binaries are not setuid).

flush

The flush daemon attempts to clear the mail queue of pending and deferred messages. By using a per-destination list of queued mail, it improves the performance of the SMTP Extended Turn (ETRN) request and its command-line equivalent, sendmail -qR destination. You can maintain the list of destinations with the fast_flush_domains parameter in the main.cf file.

qmgr

The qmgr daemon manages the Postfix queues; it is the heart of the Postfix mail system. It distributes delivery tasks to the local, smtp, lmtp, and pipe daemons. After delegating a job, it submits queue file path-name information, the message sender address, the target host (if the destination is remote), and one or more message-recipient addresses to the daemon it delegated the delivery task to.

The qmgr design is a good example of how Postfix handles jobs to avoid resource starving and to maintain stability. Two things stand out:

  • qmgr maintains a small active queue, with just a few messages pending for delivery. This queue effectively acts as a limited window for the potentially larger incoming and deferred queues, and it prevents qmgr from running out of memory under a heavy load.

  • If Postfix cannot immediately deliver a message, qmgr moves the message to the deferred queue. Keeping temporarily undeliverable messages in a separate queue ensures that a large mail backlog does not slow down normal queue access.

qmgr uses the bounce and error daemons to bounce mail for recipients listed in the relocated table that contains contact information for users or domains that no longer exist on the system.

proxymap

Postfix client processes can get read-only access to maps through the proxymap daemon. By sharing a single open map among many Postfix daemons, proxymap circumvents chroot restrictions and reduces the number of open lookup tables.

spawn

The spawn process creates non-Postfix processes on request. It listens on a TCP port, Unix domain socket, or FIFO connected to the standard input, output, and error streams. The only use for spawn discussed in this book is for the Postfix external content filtering system in Chapter 13.

local

As the name suggests, the local daemon is responsible for local mailbox delivery. The Postfix local daemon can write to mailboxes in the mbox and Maildir formats. In addition, local can access data in Sendmail-style alias databases and user .forward files.

Note: These capabilities make local the counterpart to the Sendmail mail posting agent, and they both maintain the same user interface.

As an alternative, local can delegate mailbox delivery to a local delivery agent (LDA) that provides more advanced features, such as filtering. Two very popular LDAs are procmail (www.procmail.org) and maildrop (www.flounder.net/~mrsam/maildrop).

Postfix can run multiple instances of local.

virtual

The virtual daemon, sometimes called the virtual delivery agent, is a stripped-down version of local that delivers exclusively to mailboxes. It is the most secure Postfix delivery agent; it does not perform alias and .forward file expansions.

This delivery agent can deliver mail for multiple domains, making it especially suitable for hosting several small domains on a single machine (a so-called POP toaster) without the need for real system or shell accounts.

smtp

The smtp client is the Postfix client program that transports outbound messages to remote destinations. It looks up destination mail exchangers, sorts the list by preference, and tries each address until it finds a server that responds. A busy Postfix system typically has several smtp daemons running at once.

lmtp

The lmtp client communicates with local and remote mailbox servers with the Local Mail Delivery Protocol (LMTP) defined in RFC 2033 (ftp.rfc-editor.org/in-notes/rfc2033.txt). It is often used with the Cyrus IMAP server (asg.web.cmu.edu/cyrus/imapd).

The advantages of a setup using Postfix's lmtp client are that Postfix handles all of the queue management and one Postfix machine can feed multiple mailbox servers (which need to have an LMTP daemon) over LMTP. The opposite is also true: several Postfix machines can feed one mailbox server through lmtp. These mailbox server(s) could, for example, be running Cyrus IMAP.

pipe

The pipe mailer client is the outbound interface to other mail transport mechanisms. It invokes programs with parameters and pipes the message body into their standard input.

pickup

The pickup daemon picks up messages put into the maildrop queue by the local sendmail user client program. After performing a few sanity checks, pickup passes messages to the cleanup daemon.

smtpd

The smtpd daemon handles communication with networked mail clients that deliver messages to Postfix through SMTP. smtpd performs a number of checks that protect the rest of the Postfix system, and it can be configured to implement unsolicited commercial email (UCE) controls (local or network-based blacklists, DNS lookups, other client requests, and so on).

After accepting a message, smtpd hands the message down to cleanup

cleanup

The cleanup daemon is the final processing stage for new messages. It adds any required missing headers, arranges for address rewriting, and (optionally) extracts recipient addresses from message headers. The cleanup daemon inserts the result into the incoming queue and then notifies the queue manager that new mail has arrived.

sendmail

sendmail is a Postfix command that replaces and emulates Eric Allman's MTA Sendmail. Its purpose is to provide a Sendmail-compatible interface to applications that will only invoke /usr/sbin/sendmail. It interacts with the postdrop binary to put mail into the maildrop queue for pickup.

Note: sendmail is the slowest way to inject mail into the Postfix queue system. If you need to send a large amount of mail at once, use SMTP instead.

qmqpd

The Postfix QMQP server implements the Quick Mail Queuing Protocol (QMQP; see http://cr.yp.to/proto/qmqp.html) to make Postfix compatible with qmail and the ezmlm list manager.

anvil

The Postfix anvil is a preliminary defense against SMTP clients and denial-of-service attacks that swamp the SMTP server with too many simultaneous or successive connection attempts. It comes with a whitelist capability for disabling restrictions for authorized clients. anvil is not included with Postfix 2.1, but it is available in the Postfix 2.2 experimental release. anvil will stay experimental until there is enough experience with Postfix rate limiting.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Chill

Anonymous's picture

the f**k out people. you're not learning anything by having a go at each other.

Postfix is easier to install

V. Prague's picture

You say that Postfix is a useful alternative to Sendmail, but it is actually better.
Just a week ago, I was unable to set up Sendmail properly on the 64 bit Mandrake - it just refused to send mail. I asked a very experienced friend to do this, only to hear that Sentmail apparently does not respect some DNS related settings. He installed Postfix and it works perfectly. I do not wish to hear about Sendmail again.

This is THE Postfix book to have

Scott Kitterman's picture

I found the Book of Postfix invaluable to me when I was getting started with Postfix. If you want to learn Postfix, this book is the place to start.

This is THE Postfix book to have - hardly

Anonymous's picture

Postfix specialists might disagree with you. I certainly do.

2nd edition?

Chris's picture

The book has an extraordinary amount of errata -- so much that I have to keep the errata page open whenever I read the book. Are there plans for a second (proof-read) edition?

A second edition is in the

Anonymous's picture

A second edition is in the works. Unfortunately we need to backport the text from RTF to our native XML format first :(

Why bother?

Anonymous's picture

Basically, you're promoting your book, much of which I have read. The article itself is simply a brain dump for you to jack up your ego and say, "look what I know". Only a postfix administrator or developer would understand this article, so it teaches little - like your book.

Readers might need a warning: No Starch Press did a poor job of editing the book (if they did any editing at all): No copy editing, no technical editing, etc. and most of it reads like a German with little understanding of native English - "now, I will tell you how you must do this....First you must ...then you must, etc."

Additionally, the main Chapter about building a company server is broken. You won't build a functional server following their instructions.

So, consider this with caution.

Considering this is the only

Anonymous's picture

Considering this is the only article online I've found that explains clearly the process that postfix uses to process mail, and how all the pieces fit together, I hardly see it as just an ego braindump.

The entire postfix documentation is written so that only a postfix administrator or developer would understand it.

Considering that every organisation is going to do mail a little differently, standard how-to guides very rarely can be followed completely. This is why it's so important to actually understand the system. If I can understand the system, each configuration value is just a manpage away.

Why bother....

Keith Daniels's picture

Uh.... I assume you have written something better? If so I would like to hear about it...

As far as the promotion goes, the authors had nothing to do with it, if that is who you are accusing of promoting it.

RE: "Only a postfix administrator or developer would understand this book"...... duh, who else but someone who was, or wanted to be, an administrator or developer WOULD read it?

If formal American style English is your prime criteria for the excellence of a book and not having that makes it not worthy of reading -- then your basic orientation must be academic liberal arts instead of technical. Either that or you have a personal issue with the authors or their nationality and are just trying to put them, the book and their nationality down because of prejudice or personal dislike.

RE: the Chapter about building a company server -- did you check their web site for corrections they might have posted for that page? Did you actually try to build that server setup or is that just your opinion from reading it?

"I have always wished that my computer would be as easy to use as my telephone.
My wish has come true. I no longer know how to use my telephone."
-- Bjarne Stroustrup

Yea, yea, yea.

Anonymous's picture

Yea, yea, yea.

If only an experienced postfix admin could read it, why bother?

In other words, if you're interested in learning postfix, how about a book that guides someone with less knowledge to become more able. That seems to be the purpose of a book like this.

As far as American English - how about just English instead of the horrible language used. Try this from page 313: "Once we got this going, we will make the system more complex." then, "You should have profound understand of LDAP schema and OpenLDAP before you start to implement the company mail server we describe in this chapter".

This book is crammed full of this kind of grand work.

Take your sophomoric diatribe to Slashdot, freak.

yea, yea, yea

Keith Daniels's picture

What's the matter? Your little company having trouble using Postfix with their "Controled Email" and you are blaming it on some poor non native English speakers who wrote a book?

"I have always wished that my computer would be as easy to use as my telephone.
My wish has come true. I no longer know how to use my telephone."
-- Bjarne Stroustrup

yea, yea, yea

Anonymous's picture

DANG! I read two other books before this, both of those helped very little and go me nowhere. I had no previous experience other than HOWTO's I found on line, which weren't many. After I read this book, I can honestly say that I had a successful Postfix email server running and operating SPAM FREE.

These cats don't know what they are talking about! It's a good book and Postfix is definitely easier and faster to learn than bloated sendmail.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix