Introduction

Email remains a cornerstone of modern communication. From business notifications to personal messages, having a robust and reliable mail server is essential. While cloud-based solutions dominate the mainstream, self-hosting a mail server offers control, customization, and learning opportunities that managed services can't match.

In this guide, we will explore how to set up a secure and efficient mail server using Dovecot on an Ubuntu Server. Dovecot is a lightweight and high-performance IMAP and POP3 server that provides secure access to mailboxes. When paired with Postfix, it forms a powerful mail server stack capable of sending and receiving messages seamlessly.

Whether you're a system administrator, a DevOps enthusiast, or simply curious about running your own mail infrastructure, this article provides a deep dive into configuring Dovecot on Ubuntu.

Prerequisites

Before we dive into configuration and deployment, ensure the following requirements are met:

• Ubuntu Server (20.04 or later recommended)

• Root or sudo access

• Static IP address assigned to your server

• Fully Qualified Domain Name (FQDN) pointing to your server

• Proper DNS records:

  • A record pointing your domain to your server IP

  • MX record pointing to your mail server’s FQDN

  • Optional: SPF, DKIM, and DMARC for email authentication

You should also ensure that your system is up-to-date:

sudo apt update && sudo apt upgrade -y

Understanding the Mail Server Stack

A modern mail server is composed of several components:

• Postfix: SMTP server responsible for sending and routing outgoing mail.

• Dovecot: Handles retrieval of mail via IMAP/POP3 and secure authentication.

• SpamAssassin / ClamAV: For filtering spam and malware.

• TLS/SSL: Provides encrypted communication channels.

Here's how they work together:

1. Postfix receives email from external sources.

2. It stores messages into local mailboxes.

3. Dovecot lets users access their mail securely using IMAP or POP3.

4. TLS/SSL encrypts the entire process, ensuring privacy.

Step 1: Installing Postfix and Dovecot

Install Postfix

sudo apt install postfix -y

During installation, you will be prompted to choose a configuration. Select:

• General type of mail configuration: Internet Site

• System mail name: yourdomain.com

You can always reconfigure it later:

sudo dpkg-reconfigure postfix

Install Dovecot

sudo apt install dovecot-core dovecot-imapd dovecot-pop3d -y

Step 2: Configure Mail Directories

We'll use the Maildir format as it stores each message in a separate file, making it easier to manage.

Update Postfix to deliver to Maildir:

Edit /etc/postfix/main.cf and add:

home_mailbox = Maildir/

Then reload Postfix:

sudo systemctl restart postfix

For each user, create a Maildir:

sudo mkdir /home/username/Maildir sudo maildirmake.dovecot /home/username/Maildir sudo chown -R username:username /home/username/Maildir

Step 3: Configuring Dovecot

Dovecot's configuration files are located in /etc/dovecot/. The primary file is dovecot.conf.

Mail Location

Edit /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

Ensure mail user privileges:

first_valid_uid = 1000

Authentication Configuration

Edit /etc/dovecot/conf.d/10-auth.conf:

disable_plaintext_auth = yes auth_mechanisms = plain login

Use system users for authentication:

!include auth-system.conf.ext

Configure Services

Edit /etc/dovecot/conf.d/10-master.conf and enable the following section under service auth:

unix_listener /var/spool/postfix/private/auth { mode = 0660 user = postfix group = postfix }

Restart Dovecot:

sudo systemctl restart dovecot

Step 4: Enabling SSL/TLS Encryption

For production, use Let’s Encrypt. For testing, create a self-signed certificate:

sudo openssl req -new -x509 -days 365 -nodes -out /etc/ssl/certs/mailcert.pem -keyout /etc/ssl/private/mailkey.pem

Edit /etc/dovecot/conf.d/10-ssl.conf:

ssl = required ssl_cert = </etc/ssl/certs/mailcert.pem ssl_key = </etc/ssl/private/mailkey.pem

Restart Dovecot again:

sudo systemctl restart dovecot

Step 5: Configure the Firewall

Open necessary ports:

sudo ufw allow 25,587,110,995,143,993/tcp sudo ufw enable

Common ports:

• 25: SMTP

• 587: Submission (SMTP with auth)

• 110: POP3

• 995: POP3S

• 143: IMAP

• 993: IMAPS

Step 6: Adding Mail Users

To add local users:

sudo adduser mailuser

Create mail directories:

sudo mkdir /home/mailuser/Maildir sudo maildirmake.dovecot /home/mailuser/Maildir sudo chown -R mailuser:mailuser /home/mailuser/Maildir

These users can now connect using an email client via IMAP or POP3.

Step 7: Testing the Mail Server

Use openssl to test IMAPS:

openssl s_client -connect yourdomain.com:993

You can also use telnet to check connections or configure a mail client like Thunderbird:

• Incoming: IMAP, port 993, SSL/TLS, normal password

• Outgoing: SMTP, port 587, STARTTLS, normal password

Check logs for errors:

sudo tail -f /var/log/mail.log

Step 8: Hardening and Maintenance

Enable Fail2Ban

sudo apt install fail2ban -y

Fail2Ban monitors logs and bans IPs that show signs of malicious activity.

Regular Updates

Set up unattended upgrades:

sudo apt install unattended-upgrades

Mail Backup

Backup /etc/postfix, /etc/dovecot, and mailboxes (usually under /home/*/Maildir or /var/mail).

You may use rsnapshot or rsync for daily incremental backups.

Conclusion

By following this guide, you've built a fully functional and secure mail server using Postfix and Dovecot on Ubuntu. You now have:

• A working SMTP server for sending mail.

• A Dovecot-based IMAP/POP3 server for accessing messages.

• SSL/TLS secured communication.

• Local users who can send and receive emails from mail clients.

With fine-tuned configurations and proper security measures, your mail server is ready for real-world use. You can further expand your setup by integrating webmail clients like Roundcube, enabling spam filtering, and setting up email authentication (SPF, DKIM, DMARC).