Anatomy of Postfix
Please note that this book is based on is based on Postfix 2.1, later versions of Postfix have new daemons not discussed in the book.
This chapter describes how Postfix works, what each piece of the system does, and how these components relate to each other. After going through this material, you should have an understanding of Postfix as a whole, so that you can focus on individual goals.
Postfix consists of a small number of programs that interact with user processes (sendmail, postqueue, postsuper, and so on) and a larger number of programs that run in the background. Only the programs that run in the background are controlled by the master daemon. The master daemon's job is to determine what work there is to do and dispatch the appropriate program to do the work. This modular design allows for a higher level of security because each program runs with the lowest privilege set needed to fulfill its task.
You can think of the whole Postfix system as a router. This may sound strange at first, but remember that a router's job is to look at an IP packet, determine the destination IP address (and possibly the source), and then choose the right interface to route the packet toward its destination. Postfix does the same thing with mail (see Figure 5-1), looking at the destination of a message (the envelope recipient) and the source (the envelope sender) to determine the application that will move the message closer to its final destination.
Now let's look more closely at the system. A real router usually accepts IP packets from multiple interfaces, routing them back out through the interfaces. The same is true for Postfix; it accepts messages from multiple sources and then passes the mail on to multiple destinations. A message's origin may be the local sendmail binary or an SMTP or QMQP connection. The destination can be a local mailbox, outgoing SMTP or LMTP, a pipe into a program, and more. Figure 5-2 shows this view of Postfix.
The origin and destination of a message seem clear enough, but how does Postfix pick a delivery method given a destination? A router uses routing tables that match IP addresses to networks to determine a path. Postfix does the same thing with email addresses.
In Postfix, lookup tables are called maps. Postfix uses maps not only to find out where to send mail, but also to impose restrictions on clients, senders, and recipients, and to check certain patterns in email content. Figure 5-3 shows where the maps--to name but a few, aliases, virtual, and transport are shown--fit in.
Figure 5-4 shows an overview of the Postfix daemons and how they fit together.
Postfix is constantly under development. The following list of daemons is based on Postfix 2.1.
The master daemon is the supervisor of Postfix, and it oversees all other Postfix daemons. The master waits for incoming jobs to be delegated to subordinate daemons. If there is a lot of work to do, the master can invoke multiple instances of a daemon. You can configure the number of simultaneous daemon instances, how often Postfix can reuse them, and a period of inactivity that should elapse before stopping an instance.
If you have ever worked with the inetd server on a Unix machine, you will find many similarities between it and the master daemon.
- bounce and defer
A mail transfer agent must notify the sender about undeliverable mail. In Postfix, the bounce and defer daemons handle this task, which is triggered by the queue manager (qmgr). Specifically, the two event types that cause sender notices are unrecoverable errors and a destination that is unreachable for an extended period of time. The latter case results in a delay warning.
The error daemon is a mail delivery agent like local or smtp. It is a delivery agent that always causes mail to be bounced. Usually you don't use it unless you configure a domain as undeliverable by directing mail to the error delivery agent. If a mail is sent to the error daemon it will inform the bounce daemon to record that a recipient was undeliverable.
The trivial-rewrite daemon acts upon request by the cleanup daemon in order to rewrite nonstandard addresses into the standard user@fqdn form.
This daemon also resolves destinations upon request from the queue manager (qmgr). By default, trivial-rewrite distinguishes only between local and remote destinations.
The showq daemon lists the Postfix mail queue, and it is the program behind the mailq (sendmail -bp) command. This daemon is necessary because the Postfix queue is not world-readable; a non-setuid user program cannot list the queue (and Postfix binaries are not setuid).
The flush daemon attempts to clear the mail queue of pending and deferred messages. By using a per-destination list of queued mail, it improves the performance of the SMTP Extended Turn (ETRN) request and its command-line equivalent, sendmail -qR destination. You can maintain the list of destinations with the fast_flush_domains parameter in the main.cf file.
The qmgr daemon manages the Postfix queues; it is the heart of the Postfix mail system. It distributes delivery tasks to the local, smtp, lmtp, and pipe daemons. After delegating a job, it submits queue file path-name information, the message sender address, the target host (if the destination is remote), and one or more message-recipient addresses to the daemon it delegated the delivery task to.
The qmgr design is a good example of how Postfix handles jobs to avoid resource starving and to maintain stability. Two things stand out:
qmgr maintains a small active queue, with just a few messages pending for delivery. This queue effectively acts as a limited window for the potentially larger incoming and deferred queues, and it prevents qmgr from running out of memory under a heavy load.
If Postfix cannot immediately deliver a message, qmgr moves the message to the deferred queue. Keeping temporarily undeliverable messages in a separate queue ensures that a large mail backlog does not slow down normal queue access.
qmgr uses the bounce and error daemons to bounce mail for recipients listed in the relocated table that contains contact information for users or domains that no longer exist on the system.
Postfix client processes can get read-only access to maps through the proxymap daemon. By sharing a single open map among many Postfix daemons, proxymap circumvents chroot restrictions and reduces the number of open lookup tables.
The spawn process creates non-Postfix processes on request. It listens on a TCP port, Unix domain socket, or FIFO connected to the standard input, output, and error streams. The only use for spawn discussed in this book is for the Postfix external content filtering system in Chapter 13.
As the name suggests, the local daemon is responsible for local mailbox delivery. The Postfix local daemon can write to mailboxes in the mbox and Maildir formats. In addition, local can access data in Sendmail-style alias databases and user .forward files.
Note: These capabilities make local the counterpart to the Sendmail mail posting agent, and they both maintain the same user interface.
As an alternative, local can delegate mailbox delivery to a local delivery agent (LDA) that provides more advanced features, such as filtering. Two very popular LDAs are procmail (www.procmail.org) and maildrop (www.flounder.net/~mrsam/maildrop).
Postfix can run multiple instances of local.
The virtual daemon, sometimes called the virtual delivery agent, is a stripped-down version of local that delivers exclusively to mailboxes. It is the most secure Postfix delivery agent; it does not perform alias and .forward file expansions.
This delivery agent can deliver mail for multiple domains, making it especially suitable for hosting several small domains on a single machine (a so-called POP toaster) without the need for real system or shell accounts.
The smtp client is the Postfix client program that transports outbound messages to remote destinations. It looks up destination mail exchangers, sorts the list by preference, and tries each address until it finds a server that responds. A busy Postfix system typically has several smtp daemons running at once.
The lmtp client communicates with local and remote mailbox servers with the Local Mail Delivery Protocol (LMTP) defined in RFC 2033 (ftp.rfc-editor.org/in-notes/rfc2033.txt). It is often used with the Cyrus IMAP server (asg.web.cmu.edu/cyrus/imapd).
The advantages of a setup using Postfix's lmtp client are that Postfix handles all of the queue management and one Postfix machine can feed multiple mailbox servers (which need to have an LMTP daemon) over LMTP. The opposite is also true: several Postfix machines can feed one mailbox server through lmtp. These mailbox server(s) could, for example, be running Cyrus IMAP.
The pipe mailer client is the outbound interface to other mail transport mechanisms. It invokes programs with parameters and pipes the message body into their standard input.
The pickup daemon picks up messages put into the maildrop queue by the local sendmail user client program. After performing a few sanity checks, pickup passes messages to the cleanup daemon.
The smtpd daemon handles communication with networked mail clients that deliver messages to Postfix through SMTP. smtpd performs a number of checks that protect the rest of the Postfix system, and it can be configured to implement unsolicited commercial email (UCE) controls (local or network-based blacklists, DNS lookups, other client requests, and so on).
After accepting a message, smtpd hands the message down to cleanup
The cleanup daemon is the final processing stage for new messages. It adds any required missing headers, arranges for address rewriting, and (optionally) extracts recipient addresses from message headers. The cleanup daemon inserts the result into the incoming queue and then notifies the queue manager that new mail has arrived.
sendmail is a Postfix command that replaces and emulates Eric Allman's MTA Sendmail. Its purpose is to provide a Sendmail-compatible interface to applications that will only invoke /usr/sbin/sendmail. It interacts with the postdrop binary to put mail into the maildrop queue for pickup.
Note: sendmail is the slowest way to inject mail into the Postfix queue system. If you need to send a large amount of mail at once, use SMTP instead.
The Postfix QMQP server implements the Quick Mail Queuing Protocol (QMQP; see http://cr.yp.to/proto/qmqp.html) to make Postfix compatible with qmail and the ezmlm list manager.
The Postfix anvil is a preliminary defense against SMTP clients and denial-of-service attacks that swamp the SMTP server with too many simultaneous or successive connection attempts. It comes with a whitelist capability for disabling restrictions for authorized clients. anvil is not included with Postfix 2.1, but it is available in the Postfix 2.2 experimental release. anvil will stay experimental until there is enough experience with Postfix rate limiting.
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
|Introduction to MapReduce with Hadoop on Linux||Jun 05, 2013|
- Containers—Not Virtual Machines—Are the Future Cloud
- Non-Linux FOSS: libnotify, OS X Style
- Linux Systems Administrator
- Validate an E-Mail Address with PHP, the Right Way
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Introduction to MapReduce with Hadoop on Linux
- RSS Feeds
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?