In this tutorial, I will introduce step by step how to setup a full-fledge email server.

At the end of the tutorial, the email server will support

  • SMTP server to receive email
  • SMTP to send email
  • Receiving SMTP server supports TLS
  • Sending SMTP server support DKIM
  • Configure DMARC, SPF records
  • IMAP/POP3 server with dovecot
  • (TODO) Automatically renew TLS certificate with letsencrypt
  • (TODO) Periodically backup email data to S3
  • (TODO) Setup a simple web-base client

Refer to this post to know how to test sending SMTP/receiving SMTP/POP3/IMAP with(out) TLS servers.

My server information:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04 LTS
Release:	20.04
Codename:	focal
$ uname -a
Linux ip-10-0-20-69 5.4.0-1018-aws #18-Ubuntu SMP Wed Jun 24 01:15:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Part 1: Install exim4

  • sudo apt install -y exim4 certbot
  • Allow port 80 (HTTP for letsencrypt), 25 (SMTP) in iptables/security group/network ACL, etc.
  • Generate a dkim key

To check the username who is running the exim4 process: ps aux | grep exim. In ubuntu, the username is Debian-exim

sudo openssl genrsa -out /etc/exim4/dkim.pem 2048
openssl rsa -in /etc/exim4/dkim.pem -pubout -out dkim.pub -outform PEM
sudo chown Debian-exim:Debian-exim /etc/exim4/dkim.pem
cat dkim.pub

copy the result to setup the DNS txt dkim record

  • Setup DNS records

Add following records.

+ Receiving server:
mx / @ / mail / 10
A / mail / 54.199.209.24

+ Sending server: A / smtp / 54.199.209.24

+ SPF record: TXT / @ / "v=spf1 ip4:54.199.209.24 -all"'

+ DMARC record: TXT / _dmarc / v=DMARC1\;p=reject\;rua=mailto:tr@nsang.me\;ruf=mailto:tr@nsang.me\;rf=afrf\;pct=100. Note that the rua/ruf email target is to a nsang.me domain. Because this target domain (nsang.me) is different from the being setup domain (phanthuha.me), it requires an additional record in the nsang.me domain.

+ DKIM record.
Note 1: mail is a key selector and can be an arbitrary value.
Note 2: DNS records are limited by 255 characters. Some DNS providers support long DNS record (with namecheap, up to 2500 characters) in the panel. In fact, they (the provider) separate the long value to multiple less-than-256 characters entries under the hood.

TXT / mail._domainkey / k=rsa; t=s; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAo+/XQCEHaV/3Yp9ZWQ0r1i9Eyb8l8ZPOdFvyZIFFRI2ZLQo86fyDu8HSs2MZXC0wtr1Ec5STsyG1kAmai+OOgN9JS1g050NCJFXAjd0fhxymWSsG+bHvP70BirwU7oaiUCsJ8H9cKUdZ4BkiHQfJ1k6+D4jMTra62hEMgkg8cillG9reW69PfwF4hhiYg+qNcUm1lxYFPAqLbxD5n2wG9f6bhSuntVomwGPAnUMivfi0lpb+rLM9XW1RUZI/aDjI5imfMhhTefRPW4pPnnhtOwyMMs78jccmEZroCcWVD3wMTgZbxQyB2nZRm/80sLrP+HDZMMZUlpK8pZCC6pTPywIDAQAB

The long value MIIBI... is the value copied in the DKIM generating step.

+ In nsang.me, add DMARC report record: TXT / phanthuha.me._report._dmarc / v=DMARC1

  • Add user group. This group will also used for the dovecot user and exim user to read cert keys generated by letsencrypt.
sudo groupadd mailers
sudo usermod -aG mailers Debian-exim
  • Get TLS certificates with certbot
sudo certbot certonly -d smtp.phanthuha.me --standalone --agree-tos -m tr@nsang.me -n

The cert key files are at:
+ Public cert and chain: /etc/letsencrypt/live/smtp.phanthuha.me/fullchain.pem
+ Key file (private): /etc/letsencrypt/live/smtp.phanthuha.me/privkey.pem

  • Allow exim user to access the private key
sudo chgrp mailers /etc/letsencrypt/{live,archive}{,/smtp.phanthuha.me} /etc/letsencrypt/archive/smtp.phanthuha.me/privkey1.pem
sudo chmod g+x /etc/letsencrypt/{live,archive}
sudo chmod g+r /etc/letsencrypt/archive/smtp.phanthuha.me/privkey1.pem
  • (optional) Configure exim4 from interactively: sudo dpkg-reconfigure exim4

This script merely updates the configuration file at /etc/exim4/update-exim4.conf.conf.

The default content of this file is as following

# /etc/exim4/update-exim4.conf.conf
#
# Edit this file and /etc/mailname by hand and execute update-exim4.conf
# yourself or use 'dpkg-reconfigure exim4-config'
#
# Please note that this is _not_ a dpkg-conffile and that automatic changes
# to this file might happen. The code handling this will honor your local
# changes, so this is usually fine, but will break local schemes that mess
# around with multiple versions of the file.
#
# update-exim4.conf uses this file to determine variable values to generate
# exim configuration macros for the configuration file.
#
# Most settings found in here do have corresponding questions in the
# Debconf configuration, but not all of them.
#
# This is a Debian specific file

dc_eximconfig_configtype='local'
dc_other_hostnames='ip-10-0-20-69.ap-northeast-1.compute.internal'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='false'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'

My config (comment stripped)

dc_eximconfig_configtype='internet'
dc_other_hostnames='phanthuha.me'
dc_local_interfaces=''
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='false'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='maildir_home'

If you change the content of this file or the configuration in /etc/exim4/exim4.conf.template or /etc/exim4/conf.d, you need to execute sudo update-exim4.conf to generate and update the updated configuration.

From exim4 manpage

The script update-exim4.conf generates the main configuration files /var/lib/exim4/config.autogenerated for Exim v4 by merging the data in the template file /etc/exim4/exim4.conf.template or the ones in the /etc/exim4/conf.d directory tree respectively and /etc/exim4/update-exim4.conf.conf to the output file /var/lib/exim4/config.autogenerated.

If dc_use_split_config in /etc/exim4/update-exim4.conf.conf specifies a split configuration, update-exim4.conf processes the /etc/exim4/conf.d subdirectories in the order main, acl, router, transport, retry, rewrite and auth. Within each directory it takes files in lexical sort order by file name. It concatenates all these files and makes the debconf replacement described below.

If you are not using split configuration update-exim4.conf concatenates /etc/exim4/exim4.conf.localmacros (if this file exists) and /etc/exim4/exim4.conf.template (in this order) and makes the debconf replacement described below.

For some configuration, it requires restarting the exim4 service with sudo systemctl restart exim4.

  • Config the exim4 server: sudo vim /etc/exim4/exim4.conf.template

+ TLS

MAIN_TLS_ENABLE = true
MAIN_TLS_CERTIFICATE = /etc/letsencrypt/live/smtp.phanthuha.me/fullchain.pem
MAIN_TLS_PRIVATEKEY = /etc/letsencrypt/live/smtp.phanthuha.me/privkey.pem

+ Enable authentication by uncommenting

login_server:
  driver = plaintext
  public_name = LOGIN
  server_prompts = "Username:: : Password::"
  server_condition = "${if crypteq{$auth2}{${extract{1}{:}{${lookup{$auth1}lsearch{CONFDIR/passwd}{$value}{*:*}}}}}{1}{0}}"
  server_set_id = $auth1
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_in_cipher}{}{}{*}}
  .endif

+ Enable DKIM by adding

DKIM_DOMAIN=phanthuha.me
DKIM_SELECTOR=mail
DKIM_PRIVATE_KEY=/etc/exim4/dkim.pem

Update the configuration with sudo update-exim4.conf && sudo systemctl restart exim4
Note: changing MAIN_TLS_CERTKEY and MAIN_TLS_PRIVATEKEY requires exim restarted: sudo systemctl restart exim4

  • Check configuration:
    + exim4 -bV
    + ps aux | grep exim
Debian-+   32196  0.0  0.1  16376  4516 ?        Ss   21:21   0:00 /usr/sbin/exim4 -bd -q30m
  • Alias for email receiving

Suppose that the OS username is ubuntu, we have an associated email ubuntu@phanthuha.me at /home/ubuntu/Maildir. To accept all emails sent to me@phanthuha.me and store in the mailbox for the ubuntu user, we need to config alias.
sudo vim /etc/aliases, add me: ubuntu.

  • Email addresses for sending

By default, when the ubuntu user calls exim to send an email from me@phanthuha.me, exim will use ubuntu@phanthuha.me in the envelope from and return path fields. To overwrite this config,
sudo vim /etc/email-addresses, add ubuntu: me@phanthuha.me.
Now: me@phanthu.me is in used instead.

  • Configure the remote SMTP connection (username/password).

+ Install package to get password hash sudo apt install -yq whois
+ mkpasswd to get password. For example: real password mypassword is hashed as tQ1xW9c3f5OP.
+ Add credential username = me, password = mypassword by sudo vim /etc/exim4/passwd, add me:tQ1xW9c3f5OP.

Note: adding/removing/modifying user database does not require restarting server or running update-exim4.conf

From: me@phanthuha.me
To: ping@tools.mxtoolbox.com
Subject: Test email from exim
hello. this is the test email's body

Sang

Ctrl-D to finish.

  • Setup gmail to sending using the setup SMTP server
    + name: any
    + email: me@phanthuha.me
    + smtp server: smtp.phanthuha.me
    + username: me
    + password: mypassword
    + secured connection using TLS

Setup dovecot

  • Install dovecot
sudo apt install -yq dovecot-imapd dovecot-pop3d
  • Open ports: 110 (POP3), 995 (POP3S), 143 (IMAP), 993 (IMAPS) in iptables/security group/network ACL/etc.
  • In DNS panel, add A / imap / 54.199.209.24. Note: we will share the imap and pop servers at imap.phanthuha.me.
  • Generate TLS cert
sudo certbot certonly -d imap.phanthuha.me --standalone --agree-tos -m tr@nsang.me -n
  • Add dovecot user to the mailers group: sudo usermod -aG mailers dovecot
  • Allow the mailers group to read the TLS private key
sudo chgrp mailers /etc/letsencrypt/{live,archive}{,/imap.phanthuha.me} /etc/letsencrypt/archive/imap.phanthuha.me/privkey1.pem
sudo chmod g+r /etc/letsencrypt/archive/imap.phanthuha.me/privkey1.pem
  • Config dovecot
    + Setup keychain. In /etc/dovecot/conf.d/10-ssl.conf,
    change lines
ssl_cert = </etc/dovecot/private/dovecot.pem
ssl_key = </etc/dovecot/private/dovecot.key

to

ssl_cert = </etc/letsencrypt/live/imap.phanthuha.me/fullchain.pem
ssl_key = </etc/letsencrypt/live/imap.phanthuha.me/privkey.pem

+ In the same dir, in file 10-auth.conf,
change line auth_mechanism = plain
to auth_mechanism = dovecot_plain dovecot_login
And, uncomment !include auth-passwdfile.conf.ext.
And, comment out #!includeauth-system.conf.ext.
+ In the same dir, in file 10-master.conf, in the block service auth { unix_listener ..., uncomment and change to the following

service auth {
  unix_listener auth-userdb {
    mode = 0660
    user = mail
  }
}

+ In the same dir, in file 10-mail.conf
change line mail_location = mbox:~/mail:INBOX=/var/mail/%u
to mail_location = maildir:~/Maildir
+ In the same dir, in file 10-logging.conf, uncomment log_path = syslog

  • Add dovecot credentials. Create file sudo touch /etc/dovecot/users
  • Add user. For example with username = me, password = mypassword.
    + Use sudo doveadm pw to generate the password. For e.g.: {CRYPT}$2y$05$pFZ8zDO.o.FtcTIWNOTqdeTgRj0OmoxzK2HineVAKEv91DEP4DXY6
    + Add following entry in /etc/dovecot/users
me:{CRYPT}$2y$05$pFZ8zDO.o.FtcTIWNOTqdeTgRj0OmoxzK2HineVAKEv91DEP4DXY6:1000:1000::/home/ubuntu:/bin/bash:userdb_mail=maildir:/home/ubuntu/Maildir

Check this dovecot document for the meanings of these fields.

  • Restart dovecot: sudo systemctl restart dovecot
  • Test the created dovecot credential: sudo doveadm auth login me
  • Setup POP3 in gmail
    + Email address: me@phanthuha.me
    + Gmailify: no support
    + Username: me
    + Password: mypassword
    + POP server: imap.phanthuha.me
    + Port: 995
    + Check "Always use a secure connection (SSL) when receiving mail.
    + All other 3 checkbox are optionals and up to your preferences.

Helpful commands and information

  • To test an SMTP connection: swaks -a -tls -q HELO -s smtp.phanthuha.me -au test -ap '<>'
  • Exim log dir: tail -f /var/log/exim4/mainlog. Also check rejectlog, and paniclog.
  • View exim stat: eximstats /var/log/exim4/mainlog
  • List all listening ports/running pid/running user: sudo lsof -i -P
  • Get dovecot user information: sudo doveadm user me
  • To debug userdb/passwd in dovecot. Add
auth_debug_passwords = yes
auth_debug = yes
  • Watch dovecot log with tail -f /var/log/syslog

TODO

  • Test if exim/dovecot require a restart if TLS cert keys change. Hint: use openssl s_client to reveal the current certificates.
  • Support multiple domains
  • Periodically backup to AWS s3.