What's the first thing you do when setting up as an indie gamedev? Why, set up a mailserver, of course! This is Part 4 of the process I went through, where we set up Dovecot to allow mail user agents to connect...

If you're here from Part 3, welcome back. If not, you've joined in the middle of me documenting the rather lengthy journey I took from an empty server to one handling email with TLS, spam and virus checking, mail accounts handled in a database, and with SPF, DKIM, and DMARC checking and compliance.

Part 4 covers getting Dovecot up and running and plugged into the Postgres database we created in Part 3 so that our email users can send and receive emails using a mail user agent such as Thunderbird.

Mail User Agents

Let's get away from having to send and check mail from the Linux command line. We installed Dovecot, the software we'll use to make this possible, in Part 3, so we can dive into configuring it.

The package we installed earlier, dovecot-imap, will allow us to get email using IMAP (Internet Message Access Protocol). The alternative is POP3 (Post Office Protocol 3), but IMAP is generally preferable. POP3 downloads all your emails onto your device (which might take ages on a new device if you have lots of old emails), but doesn't do any syncing with the server. That means that if you access your email in two places (e.g. your PC and your phone), then anything you do to your email on one device (deleting it, marking it as read, etc) won't be reflected on the other device. Having to mark emails as read more than once isn't much fun. Depending on how you set it up, POP3 may also delete emails on the server as you get them, so you might end up only being able to get a given email on a single device. IMAP syncs between the server and client, so if you read an email or move it into a different folder, this will automatically sync onto other devices. Emails won't be deleted unless you do it yourself, and when you do this'll be synced as well. IMAP doesn't have to download your entire email history, either, you can just get the most recent, say, 100 emails if you want to.

Various bits of Dovecot's config live in /etc/dovecot/conf.d/, but there's also a file at /etc/dovecot/dovecot.conf, and we can put config changes in there. dovecot.conf is the "main" config file in that it contains a line to pull in the files in conf.d

!include conf.d/*.conf

So, open up /etc/dovecot/dovecot.conf, and let's tell Dovecot we want to use IMAP:

protocols = imap

There are also some settings already present in the file, but commented out. These are the defaults, and we can just leave them as they are.

Now it's time to change some of the pre-existing settings in conf.d/ - first of all, open up 10-ssl.conf

First of all, where we have a line that says ssl = yes, this only means that Dovecot will allow a secure connection, but it won't force one. Change yes to required so that mail user agents connecting must use a TLS connection:

ssl = required

Below that you'll see that Dovecot has created a couple of self-signed SSL certificates for you at /etc/dovecot/private/. We could use these, but it would be better to use the proper Let's Encrypt ones:

ssl_cert = </etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/example.com/privkey.pem

Note the < at the start of the filename - this tells Dovecot to set the parameter to the contents of the file rather than just the filename as a string.

We can also beef up the minimum TLS version that Dovecot is willing to use. Depending on who you're serving with this mailserver, you might be able to get away with TLSv1.2 only. Unfortunately Dovecot doesn't yet support TLSv1.3. Hopefully you're in a position to tell anyone using something so old it doesn't support 1.2 to go away and upgrade their software, rather than having to downgrade your security. Find the ssl_protocols parameter, uncomment it, and set it as tight as you can.

ssl_protocols = TLSv1.2

If you're using a newer version of Dovecot (>=2.3), you can use the ssl_min_protocol option instead. We can also tighten up the ciphers that Dovecot will allow. If you're using Dovecot 2.2.x (which is what Ubuntu 18 has), you can override the default cipher list with the version from 2.3, or whatever the latest list is when you read this. At time of writing, the latest default list was available at https://wiki.dovecot.org/SSL/DovecotConfiguration. You can also turn on the ssl_prefer_server_ciphers option, which will mean the server will use the first cipher in its list that a client supports, even if a client lists a weaker one first - preference in these lists is defined by the order. Save 10-ssl.conf, exit, and reload Dovecot's config.

/etc/init.d/dovecot reload

We should now be able to attempt a connection to Dovecot to see if it's responding. Try this - if you get a whole splurge of output including an SSL certificate, it's working:

openssl s_client -connect 127.0.0.1:imaps

The next step is to get Dovecot to talk to Postgres to do authentication in the same way we did Postfix. Open up /etc/dovecot/conf.d/auth-sql.conf.ext

There are two sections here, one for passdb and one for userdb. Because we've put all our user info into the same table as the password, we actually only need to use the passdb section. To tell Dovecot that it can get the info it expects from userdb (e.g. real name) from passdb, comment out the lines within the braces and add another line like this:

userdb {
#  driver = sql
#  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = prefetch
}

passdb you can leave alone. It's expecting to access an SQL database, and it's expecting the settings in /etc/dovecot/dovecot-sql.conf.ext, which is fine. Save and exit. Next we need to tell Dovecot to use the auth-sql.conf.ext file instead of the auth-system.conf.ext file so that it switches over to the database-backed authentication. Open up 10-auth.conf, and way down at the bottom comment out the auth-system.conf.ext and uncomment the sql one:

#!include auth-system.conf.ext
!include auth-sql.conf.ext

While we're in this file, you may see a troubling-looking, uncommented line that reads auth_mechanisms = plain. As you might guess, this means that the only authenication offered by Dovecot is a plaintext password. However, this isn't as scary as it looks. Right at the top of the file, you can see the default behaviour of a parameter called disable_plaintext_auth, which defaults to yes (so that's the value, even though this line is commented out). The comment above that line should explain that this only actually disables plaintext auth if SSL/TLS isn't being used. Because we set ssl = required in 10-ssl.conf, we can send the raw password over the TLS connection without worrying. We need to do this because Dovecot needs the plain password so that it can do PBKDF2 on it and check it matches the hash stored in the database.

Now open /etc/dovecot/dovecot-sql.conf.ext. This file is full of examples but all commented out. Add the stuff we need at the bottom:

driver = pgsql
connect = host=localhost dbname=mail user=mailreader password=your_password_here
default_pass_scheme = PBKDF2
password_query = SELECT email as user, password, 'maildir:/home/mail/'||maildir as userdb_mail FROM users WHERE email = '%u'

• driver = pgsql tells Dovecot that when we ask it to connect to an SQL database, we specifically mean a Postgres database.
• The connect parameter tells it exactly how to log in - you'll need your plaintext password for the mailreader user again.
• default_pass_scheme should be the same as whatever you used for your user's passwords before (in my case, PBKDF2).
• password_query tells Dovecot how to get the username (email as user means to treat the email field as the username), hashed password, and mail directory from the users database. We have to hardcode the prepended /home/mail/ bit here, Dovecot doesn't have a setting for that. %u is a Dovecot placeholder that stands in for the full email address (i.e. dave@example.com). This is what we used in the email field in the users table, so we can use that safely in the WHERE clause.

We also need to tell Dovecot about the GID and UID of the mail user. We can do that in /etc/dovecot/conf.d/10-mail.conf. There are two entries for this already, uncomment them and set the relevant values for your setup:

mail_uid = 1002
mail_gid = 1002

That's it for this file, save and exit.

Before we start connecting with an MUA like Thunderbird, there's one final change we should make to Postfix. Once you're connecting from your own PC or whatever, Postfix will log a Received header which will contain your IP address (the address of your PC, home connection, whatever it is, not your mailserver). You probably don't want to tell the whole world what your IP address is, so we should strip that out. Thankfully that's easy, we can do it using the cleanup service, which does a pass over all mail just before Postfix delivers it somewhere. Find the service in master.cf and add the following underneath:

cleanup   unix  n       -       y       -       0       cleanup
  -o header_checks=regexp:/etc/postfix/header_checks

Then create the file /etc/postfix/header_checks with the content:

/^Received:.*with ESMTPSA/ IGNORE

and reload Postfix. What we've done is told Postfix to strip out any Received header that contains the with ESMTPSA string, which will be what Postfix spits out for your SASL-authenticated MUA connection, which will have your IP address in it. By putting it in the cleanup service, it'll also run over any email you receive, so anyone who's accidentally sending you their IP address will have that cleaned up for them as well.

Reload Dovecot's config again, and you should now be able to connect via something like Thunderbird. If you didn't open up ports 587 and 993 on your firewall earlier, do that now, and also configure your mail client. Use the email address as the username, and set to connect with TLS for IMAP and STARTTLS for submission over SMTP (port 587). Again, if there are any problems, check the logs. Dovecot logs by default to syslog, so check out /var/log/syslog and search for dovecot. Once you're in, you should hopefully receive any test emails you sent yourself. You may notice, however, that there's only an Inbox folder - no Sent, Junk, Draft, etc. (NB - I found that Thunderbird created a "Deleted" folder for me which maps onto a "Trash" folder on the mailserver.) Dovecot has some handy support to get those up and running quickly. Open up conf.d/15-mailboxes.conf, look for a section starting namespace inbox {. In here are lots of folder definitions, and you can edit these and mark them to be automatically created and subscribed to. The ones already specified have a special_use parameter for some standard behaviours, such as "this is the Sent Mail folder". Here's my setup after editing:

namespace inbox {
  # These mailboxes are widely used and could perhaps be created automatically:
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }

  # For \Sent mailboxes there are two widely used names. We'll mark both of
  # them as \Sent. User typically deletes one of them if duplicates are created.
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  #mailbox "Sent Messages" {
  #  special_use = \Sent
  #}

  # If you have a virtual "All messages" mailbox:
  #mailbox virtual/All {
  #  special_use = \All
  #  comment = All my messages
  #}

  # If you have a virtual "Flagged" mailbox:
  #mailbox virtual/Flagged {
  #  special_use = \Flagged
  #  comment = All my flagged messages
  #}
}

Reload Dovecot's config and tell your MUA to recheck which folders you subscribe to and you should see the new ones appear. You can receive email in your MUA at this point because Postfix puts it in /home/mail, and that's where Dovecot is looking for it - they don't really need to talk to each other directly. That isn't true for sending mail because, as you may remember, we set Postfix up only to accept mail on the submission port from SASL-authenticated users if they're not on the local machine (which they won't be once you're using your MUA). So, the final step is to wire Dovecot and Postfix together for that. First, the Postfix end - head over to master.cf, and add two new overrides to the submission daemon:

  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth

The first of these tells Postfix to use Dovecot for SASL, and the private/auth bit is a relative path to a Unix socket that Dovecot will use to talk to Postfix. We need to tell Dovecot about that, but first let's take a last look at some extra commented-out lines you probably have in your submission daemon, which we're going to leave as they are:

#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=

• The smtpd_tls_auth_only option we don't need to worry about, we're requiring TLS authentication in Dovecot, so this is redundant.
• smtpd_reject_unlisted_recipient defaults to yes, and is probably what you want - if Postfix can't work out who to deliver the mail to, it just rejects it. So there's no need to change or uncomment this line.
• The next three are a bit odd. Future versions of Postfix may fix this issue, but if you search all the config files for $mua_[client|helo|sender]_restrictions, you won't find any hits other than these lines. By default, these are set to point to variables that don't exist. The overrides themselves (smtpd_[client|helo|sender]_restrictions) take similar options to smtpd_relay_restrictions (check the docs for the full lists), so you can stop clients connecting, making HELO requests, or making MAIL FROM commands. More advanced setups might make use of these, but here we've gone with a simple policy of "if you're on the local machine or you've got a valid username and password, fire away", so you can leave those out.
• smtpd_recipient_restrictions is the place to put your spam blocking measures. We should set those up, but we don't want any restrictions on the submission port, which is for outgoing mail - we've already locked it down by limiting who can send mail anyway.

The principle for smtpd_recipient_restrictons, however, is this: once the sender has got as far as saying who its email is for, the restrictions list can check for such things as the sender being in a Domain Block List (DBL) for spam. SpamAssassin does some of this for you - it incorporates DBL hits into its heuristics, for example - but there is one option you might want to turn on here, which is reject_unknown_client_hostname (this was reject_unknown_client in Postfix versions before 2.3). This will reject out of hand any email that doesn't resolve both forward and reverse DNS lookups for the sending domain (I'll explain more about that in a moment in Part 5.) However, rather than doing it in the -o section (which would only turn it on for the submission port, where we don't want it), instead uncomment the empty smtpd_recipient_restrictions here so that we override with an empty list, and add the line to main.cf and reload Postfix's config.

smtpd_recipient_restrictions=reject_unknown_client_hostname

Anyway, on to the Dovecot half of the mail sending link-up. Find the service auth {} section of /etc/dovecot/conf.d/10-master.conf, and get it looking like this:

service auth {
  unix_listener auth_userdb {
    path = /var/spool/postfix/private/auth
    mode = 0600
    user = postfix
    group = postfix
  }
}

This gets Dovecot to create a socket at /var/spool/postfix/private/auth, owned by the postfix user and group, and accessible only to the postfix user. This will allow Postfix to use Dovecot to do authentication checks. Reload configs for Postfix and Dovecot, and you should be able to send an email to the outside world.

At this point, your email system should be pretty usable. You're not limited to the command line any more, and you've got spam and virus checking. However, what we're not doing yet is checking that incoming emails really are from who they say they are, and we're also not taking steps to allow others to check that our emails are genuine either. We'll cover that in Part 5 - the final part, we're nearly there!