Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revisionLast revisionBoth sides next revision | ||
setup_basic_mailserver_with_postfix_dovecot_sieve [31.12.2019 20:17] – [install roundcube] Pascal Suter | setup_basic_mailserver_with_postfix_dovecot_sieve [20.03.2021 07:38] – [Create new users] Pascal Suter | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Setup basic mailserver with Postfix + Dovecot + Sieve for Virtualmail ====== | ||
+ | .. on ubunut 18.04 server | ||
+ | pastebin... | ||
+ | |||
+ | [[https:// | ||
+ | |||
+ | [[https:// | ||
+ | |||
+ | < | ||
+ | apt install pwgen | ||
+ | # usefull to create safe passwords | ||
+ | apt install mariadb-server | ||
+ | # we need this to store our accounts and domains database | ||
+ | apt install postfix postfix-mysql | ||
+ | </ | ||
+ | choose to configure postfix as **Internet Site** and enter fully qualified domain name of the server, the name entered here must not match any email domains you want to handle later (since we are configuring our server to process virtual mailboxes and not canonical domains). | ||
+ | |||
+ | < | ||
+ | apt install apache2 php7.2 swaks mutt certbot dovecot-mysql dovecot-pop3d dovecot-imapd dovecot-managesieved dovecot-lmtpd adminer ca-certificates | ||
+ | |||
+ | rm / | ||
+ | |||
+ | cat > / | ||
+ | < | ||
+ | ServerName mail.yourdomain.ch | ||
+ | DocumentRoot / | ||
+ | </ | ||
+ | EOF | ||
+ | a2ensite webmail-http | ||
+ | systemctl reload apache2 | ||
+ | mkdir -p / | ||
+ | chown www-data.www-data / | ||
+ | echo "it works" > / | ||
+ | </ | ||
+ | now try http:// | ||
+ | |||
+ | < | ||
+ | certbot certonly --webroot --webroot-path / | ||
+ | cat > / | ||
+ | < | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | Alias /adminer / | ||
+ | </ | ||
+ | EOF | ||
+ | a2ensite webmail-https | ||
+ | a2enmod ssl | ||
+ | systemctl restart apache2 | ||
+ | </ | ||
+ | to auto-forward all non-https traffic except for the certbot renewal stuff to https, add this to your ''/ | ||
+ | < | ||
+ | RewriteEngine On | ||
+ | RewriteCond %{REQUEST_URI} !.well-known/ | ||
+ | RewriteRule ^(.*)$ https:// | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | a2enmod rewrite | ||
+ | systemctl restart apache2 | ||
+ | </ | ||
+ | to make sure letsencrypt will restart all our servers once the ssh keys change, we need to add this: | ||
+ | < | ||
+ | cat > / | ||
+ | #!/bin/bash | ||
+ | service postfix restart | ||
+ | service dovecot reload | ||
+ | service apache2 reload | ||
+ | EOF | ||
+ | chmod +x / | ||
+ | |||
+ | </ | ||
+ | |||
+ | setup mysql database: | ||
+ | |||
+ | create pw: | ||
+ | < | ||
+ | pwgen -s1 30 2 | ||
+ | </ | ||
+ | note down the two passwords and start the mysql client | ||
+ | mysql | ||
+ | in the mysql client console enter the following mysql commands: | ||
+ | < | ||
+ | CREATE DATABASE mailserver; | ||
+ | grant all on mailserver.* to ' | ||
+ | grant select on mailserver.* to ' | ||
+ | USE mailserver; | ||
+ | CREATE TABLE IF NOT EXISTS `domains` ( | ||
+ | `id` int(11) NOT NULL auto_increment, | ||
+ | | ||
+ | | ||
+ | ) ENGINE=InnoDB DEFAULT CHARSET=utf8; | ||
+ | |||
+ | | ||
+ | `id` int(11) NOT NULL auto_increment, | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | ) ENGINE=InnoDB DEFAULT CHARSET=utf8; | ||
+ | |||
+ | | ||
+ | `id` int(11) NOT NULL auto_increment, | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | ) ENGINE=InnoDB DEFAULT CHARSET=utf8; | ||
+ | |||
+ | exit | ||
+ | </ | ||
+ | create a password hash and add a test user (using adminer). to create the hash i used | ||
+ | dovecot pw -s SHA256-CRYPT | ||
+ | |||
+ | now let's configure postfix: | ||
+ | < | ||
+ | cat > / | ||
+ | user = mailserver | ||
+ | password = <second password goes here> | ||
+ | hosts = 127.0.0.1 | ||
+ | dbname = mailserver | ||
+ | query = SELECT 1 FROM domains WHERE name=' | ||
+ | EOF | ||
+ | |||
+ | postconf virtual_mailbox_domains=mysql:/ | ||
+ | |||
+ | #postconf virtual_mailbox_base=/ | ||
+ | #postconf virtual_uid_maps=static: | ||
+ | #postconf virtual_gid_maps=static: | ||
+ | |||
+ | </ | ||
+ | postconf adds stuff to your main.cf file and also reloads postfix, so we can now test with our test domain if it works: | ||
+ | postmap -q yourdomain.ch mysql:/ | ||
+ | this shouwld now show '' | ||
+ | |||
+ | now add virtual mailboxes: | ||
+ | < | ||
+ | cat > / | ||
+ | user = mailserver | ||
+ | password = <second password goes here> | ||
+ | hosts = 127.0.0.1 | ||
+ | dbname = mailserver | ||
+ | query = SELECT 1 FROM users WHERE email=' | ||
+ | EOF | ||
+ | |||
+ | postconf virtual_mailbox_maps=mysql:/ | ||
+ | |||
+ | postmap -q john@yourdomain.ch mysql:/ | ||
+ | |||
+ | |||
+ | cat > / | ||
+ | user = mailserver | ||
+ | password = <second password goes here> | ||
+ | hosts = 127.0.0.1 | ||
+ | dbname = mailserver | ||
+ | query = SELECT destination FROM aliases WHERE source=' | ||
+ | EOF | ||
+ | |||
+ | postconf virtual_alias_maps=mysql:/ | ||
+ | |||
+ | postmap -q forwarded@mydomain.ch mysql:/ | ||
+ | #this should return the target email address | ||
+ | |||
+ | </ | ||
+ | |||
+ | next up is dovecot | ||
+ | |||
+ | create a user who will own all the mail stored on your server: | ||
+ | < | ||
+ | groupadd -g 5000 vmail | ||
+ | useradd -g vmail -u 5000 vmail -d /var/vmail -m | ||
+ | </ | ||
+ | |||
+ | all the configs are stored in ''/ | ||
+ | |||
+ | edit '' | ||
+ | auth_mechanisms = plain login | ||
+ | |||
+ | and that the sql backend at the end of this file is the only enabled backend: | ||
+ | !include auth-sql.conf.ext | ||
+ | |||
+ | next is '' | ||
+ | |||
+ | make sure the userdb section looks like this (it already did on ubuntu) | ||
+ | < | ||
+ | userdb { | ||
+ | | ||
+ | args = / | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | next is '' | ||
+ | set the mail location to | ||
+ | mail_location = maildir: | ||
+ | and enable the quota plugin | ||
+ | mail_plugins = quota | ||
+ | |||
+ | in the file '' | ||
+ | edit the '' | ||
+ | < | ||
+ | # Postfix smtp-auth | ||
+ | unix_listener / | ||
+ | mode = 0666 | ||
+ | user = postfix | ||
+ | group = postfix | ||
+ | } | ||
+ | </ | ||
+ | further more, find the '' | ||
+ | < | ||
+ | service lmtp { | ||
+ | unix_listener / | ||
+ | group = postfix | ||
+ | mode = 0600 | ||
+ | user = postfix | ||
+ | } | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | restart dovecot | ||
+ | systemctl restart dovecot | ||
+ | |||
+ | next up is '' | ||
+ | ssl = required | ||
+ | and | ||
+ | ssl_cert = </ | ||
+ | ssl_key = </ | ||
+ | |||
+ | < | ||
+ | cat >> / | ||
+ | driver = mysql | ||
+ | connect = host=127.0.0.1 dbname=mailserver user=mailserver password=< | ||
+ | user_query = SELECT email as user, \\ | ||
+ | concat(' | ||
+ | '/ | ||
+ | 5000 AS uid, 5000 AS gid \\ | ||
+ | FROM users WHERE email=' | ||
+ | password_query = SELECT password FROM users WHERE email=' | ||
+ | EOF | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | cat >> / | ||
+ | |||
+ | plugin { | ||
+ | quota = maildir: | ||
+ | |||
+ | quota_status_success = DUNNO | ||
+ | quota_status_nouser = DUNNO | ||
+ | quota_status_overquota = "552 5.2.2 Mailbox is full and cannot receive any more emails" | ||
+ | } | ||
+ | |||
+ | service quota-status { | ||
+ | executable = / | ||
+ | unix_listener / | ||
+ | user = postfix | ||
+ | } | ||
+ | } | ||
+ | |||
+ | plugin { | ||
+ | | ||
+ | | ||
+ | | ||
+ | } | ||
+ | service quota-warning { | ||
+ | | ||
+ | | ||
+ | group = dovecot | ||
+ | mode = 0660 | ||
+ | } | ||
+ | } | ||
+ | EOF | ||
+ | |||
+ | </ | ||
+ | <code bash / | ||
+ | #!/bin/sh | ||
+ | PERCENT=$1 | ||
+ | USER=$2 | ||
+ | cat << EOF | / | ||
+ | From: postmaster@webmail.example.org | ||
+ | Subject: Quota warning - $PERCENT% reached | ||
+ | |||
+ | Your mailbox can only store a limited amount of emails. | ||
+ | Currently it is $PERCENT% full. If you reach 100% then | ||
+ | new emails cannot be stored. Thanks for your understanding. | ||
+ | EOF | ||
+ | </ | ||
+ | < | ||
+ | chmod 755 / | ||
+ | |||
+ | systemctl restart dovecot.service | ||
+ | |||
+ | postconf " | ||
+ | |||
+ | chown root:root / | ||
+ | chmod go= / | ||
+ | </ | ||
+ | |||
+ | tell postfix to send email via lmtp to dovecot: | ||
+ | postconf virtual_transport=lmtp: | ||
+ | |||
+ | enable sieve plugin on lmtp protocol (this is where we get mails from postfix, so it's where we want to route them through sieve rules) | ||
+ | |||
+ | edit ''/ | ||
+ | edit the line like this: | ||
+ | mail_plugins = $mail_plugins sieve | ||
+ | |||
+ | **pending issue** i had to allow the world to read and write to ''/ | ||
+ | chmod a+w / | ||
+ | chmod a+r / | ||
+ | |||
+ | now we can do some testing with '' | ||
+ | swaks --to myuser@yourdomain.ch --server mail.yourdomain.ch | ||
+ | mutt -f imaps:// | ||
+ | | ||
+ | |||
+ | ===== postfix smtp(d) config ===== | ||
+ | < | ||
+ | postconf smtpd_sasl_type=dovecot | ||
+ | postconf smtpd_sasl_path=private/ | ||
+ | postconf smtpd_sasl_auth_enable=yes | ||
+ | postconf smtpd_tls_security_level=may | ||
+ | postconf smtpd_tls_auth_only=yes | ||
+ | postconf smtpd_tls_cert_file=/ | ||
+ | postconf smtpd_tls_key_file=/ | ||
+ | postconf smtp_tls_security_level=may | ||
+ | </ | ||
+ | to enable submission service (port 587 for sending emails from clients) edit ''/ | ||
+ | |||
+ | < | ||
+ | submission inet n | ||
+ | -o syslog_name=postfix/ | ||
+ | -o smtpd_tls_security_level=encrypt | ||
+ | -o smtpd_sasl_auth_enable=yes | ||
+ | -o smtpd_tls_auth_only=yes | ||
+ | -o smtpd_reject_unlisted_recipient=no | ||
+ | # -o smtpd_client_restrictions=$mua_client_restrictions | ||
+ | # -o smtpd_helo_restrictions=$mua_helo_restrictions | ||
+ | # -o smtpd_sender_restrictions=$mua_sender_restrictions | ||
+ | # -o smtpd_recipient_restrictions= | ||
+ | -o smtpd_relay_restrictions=permit_sasl_authenticated, | ||
+ | -o milter_macro_daemon_name=ORIGINATING | ||
+ | </ | ||
+ | |||
+ | systemctl restart postfix | ||
+ | |||
+ | ===== additional postfix settings ===== | ||
+ | ==== mail size limit ==== | ||
+ | i think 10MB is just too small, so i allowed 30mb instead | ||
+ | postconf message_size_limit=31457280 | ||
+ | |||
+ | ==== regex based virtual aliases ==== | ||
+ | i've added [[postfix_virtual_mail_addresses_with_regular_expressions|this]] as well to my config. however, I called the config file / | ||
+ | |||
+ | ===== install roundcube ===== | ||
+ | i went with the latest roundcube because the versioni that came with ubuntu at the time of writing did not have the new elastic gui which i really wanted. | ||
+ | |||
+ | download the lastest verison of [[https:// | ||
+ | cd /tmp | ||
+ | wget https:// | ||
+ | |||
+ | cd / | ||
+ | tar xvf / | ||
+ | mv roundcubemail-1.4.1/ | ||
+ | rmdir roundcubemail-1.4.1/ | ||
+ | cd pub | ||
+ | |||
+ | install dependencies according to INSTALL instructions | ||
+ | apt install php7.2-json php7.2-xml php7.2-mbstring php7.2-zip php7.2-intl php7.2-gd | ||
+ | further more i had to set the '' | ||
+ | |||
+ | follow the INSTALL instructions.. for version 1.4.1 i did this: | ||
+ | < | ||
+ | chown -R www-data.www-data logs temp | ||
+ | pwgen -s1 30 | ||
+ | </ | ||
+ | take note of the password created with pwgen and start the mysql client | ||
+ | mysql | ||
+ | inside the mysql console run these commands to create the database: | ||
+ | < | ||
+ | CREATE DATABASE roundcubemail /*!40101 CHARACTER SET utf8 COLLATE utf8_general_ci */; | ||
+ | GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost | ||
+ | IDENTIFIED BY '< | ||
+ | quit | ||
+ | </ | ||
+ | < | ||
+ | mysql roundcubemail < SQL/ | ||
+ | </ | ||
+ | continue in the web-based installer found at https:// | ||
+ | |||
+ | here are the things i've changed: | ||
+ | * imap port changed to 993 | ||
+ | * imap server set to < | ||
+ | * smtp server set to < | ||
+ | * set database password to the one you noted down before | ||
+ | * add at least '' | ||
+ | |||
+ | |||
+ | move install directory out of www root.. we can always symlink to it later if we need the installer again | ||
+ | cd / | ||
+ | mv installer ../ | ||
+ | |||
+ | configure the plugins: | ||
+ | |||
+ | create a new password for a db admin user which we need for the password plugin: | ||
+ | pwgen -s1 30 | ||
+ | create the db admin user for roundcube: | ||
+ | mysql | ||
+ | in the mysql console add the user: | ||
+ | grant all on mailserver.* to ' | ||
+ | quit | ||
+ | now write our config for the password plugin: | ||
+ | < | ||
+ | cat > / | ||
+ | <?php | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | \$config[' | ||
+ | ?> | ||
+ | EOF | ||
+ | </ | ||
+ | the managesieve plugin is simpler, just provide the hostname of the sieve server and we're golden: | ||
+ | < | ||
+ | cat > / | ||
+ | <?php | ||
+ | \$config[' | ||
+ | ?> | ||
+ | EOF | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== client auto configuration ===== | ||
+ | postponed.. [[https:// | ||
+ | |||
+ | ===== proxmox mail gateway ===== | ||
+ | i decided on using proxmox mail gateway (PMG) for spam and virus filtering rather than setting up rspamd or any other mail filter solution i have to maintain myself. i've tested PMG in the past and it yielded a pretty good detection rate. my ultimate goal is that i don't need to spend too much time dealing with spam filters, they should be there and just do their job.. PMG did just that during my tests using some catchall domains to gather as much spam as i could :) | ||
+ | |||
+ | i've installed PMG onto another Virtual Machine as i host a virtual host myself. if you have to pay alot of money for a vps and you already have one for your mailserver, you can also run PMG inside a LXC container, more details on the installation can be found in the admin guide. | ||
+ | |||
+ | so i've downloaded the latest ISO from [[https:// | ||
+ | |||
+ | i set a public ip with a hostname filter.yourdomain.ch. | ||
+ | |||
+ | after the installation is complete, you can access the web-interface on https:// | ||
+ | |||
+ | your root password is also your login for the web-interface. i did disable ssh password login and i've blocked all ports except 22 and 25 from the outside world in my firewall, so nobody can access the web-interface and brute-force my password. | ||
+ | |||
+ | once you're logged in to the web-interface, | ||
+ | |||
+ | once this is all set, go ahead and click on the "Mail Proxy" settings. | ||
+ | * under Relaying enter your mailserver' | ||
+ | * leave port 25 | ||
+ | * i've disabled MX lookups, not sure why they should be needed here. | ||
+ | * under "Relay Domains" | ||
+ | * in the Options tab i've enabled " | ||
+ | * i have disabled Greylisting as this delays mail delivery significantly and that's a bit of a pain if you wait for account confirmation emails or booking confirmations etc. i'll re-enable it if the spam detection rate is too low. | ||
+ | * also in Options, i have enabled DNSBL and i've entered the following two blacklists to query: '' | ||
+ | * there is no need to configure any transports. this is only needed if you want to route incoming mails for different domains or addresses to different servers. | ||
+ | * in the networks tab, you can add the network or ip of your mailserver, in case it is not in the same subnet as your filter.. if it's in the same subnet there is no need to add anything here, as the same subnet is allowed to relay through PMG by default. | ||
+ | * since we will be relaying our outgoing emails from our mailserver via PMG as well, we will use PMG's DKIM signing function. to enable this, in the DKIM tab you need to **first add a new selector** before you can enable DKIM .. that's a bit confusing. as selector i've entered the current date like '' | ||
+ | * for DKIM to work you need to add a TXT entry to your domian' | ||
+ | * once all these settings where done, i had to login to the filter via ssh and **manually restart postfix**. otherwise postfix would bind port 25 to 127.0.0.1 only. i guess rebooting the entire filter would fix this issue as well. | ||
+ | |||
+ | ==== tag and deliver spam instead of quarantine ==== | ||
+ | I'm not sure I or my users would be happy with waiting for reports to find out why a recently sent mail did not reach them. after all it's always a good feeling if you can tell someone on the phone that you didn't find their email in your spam folder either, to convince them that they might have had a typo in your email address :) .. | ||
+ | |||
+ | First you need to make sure that spam is no longer quarantined but instead marked and forwarded. | ||
+ | |||
+ | you can either mark an email by modifying its subject or by adding a header element.. i don't like changing the visible part of the email message, so i opted to go for an additional header field that marks spam. | ||
+ | |||
+ | to create it, go to the '' | ||
+ | |||
+ | next we need to make sure that all spam mail is tagged with this header field instead of quarantined. | ||
+ | |||
+ | in the PMG web interface click on '' | ||
+ | |||
+ | now on to your postfix mail server.. we need to add a global sieve rule to dovecot that will move spam into a spam folder.. edit ''/ | ||
+ | sieve_after = / | ||
+ | now create the sieve-after directory: | ||
+ | mkdir / | ||
+ | all filters found in this directory will be executed AFTER each user's own filters. so a user can create his own filters to whitelist spam in our case. | ||
+ | < | ||
+ | cat > / | ||
+ | require [" | ||
+ | |||
+ | if header :contains " | ||
+ | | ||
+ | stop; | ||
+ | } | ||
+ | EOF | ||
+ | </ | ||
+ | now compile the sieve filter: | ||
+ | sievec / | ||
+ | lastly restart dovecot to re-read the config we altered bove | ||
+ | systemctl restart dovecot | ||
+ | to test, send an email from outside to your mail account with the following line in the body: | ||
+ | < | ||
+ | XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X | ||
+ | </ | ||
+ | |||
+ | ==== future improvements ==== | ||
+ | === Bayesian learning by moving mail to junk folder === | ||
+ | in order for proxmox' | ||
+ | |||
+ | some information i will need to do this: | ||
+ | * the command on PMG to submit a ham or spam to the learning system is < | ||
+ | * a solution on how to use imap sieve filters to trigger a bash script when a mail is moved into a specific folder can be found in the [[https:// | ||
+ | |||
+ | |||
+ | ==== adjustments to postfix settings on our mailserver ==== | ||
+ | we can now limit access for incoming mails so that postfix only accepts connections from our mailfilter. to do this, edit the '' | ||
+ | < | ||
+ | smtp inet n | ||
+ | -o smtpd_client_restrictions=permit_mynetworks, | ||
+ | </ | ||
+ | don't forget to restart postfix | ||
+ | |||
+ | further more we can configure our mailserver to send all its mails through our proxmox gateway to allow proxmox to track outgoing mails and scan them for viruses as well. to do that we can set the '' | ||
+ | postconf relayhost=filter.yourdomain.ch: | ||
+ | note port 26, that's because proxmox mail gatway distinguishes between incoming and outgoing mail by accepting them on different smtp ports. by default port 25 is for incoming and port 26 for outgoing mail. | ||
+ | |||
+ | ==== greylisting ==== | ||
+ | by default PMG uses greylisting. this means, that every email coming from a new sender address will first be rejected for a duration of a couple of minutes. i think 3 minutes is the actual greylist timeout on PMG. however, the delay that occurrs in reality will be dependent also on the sending mail server' | ||
+ | |||
+ | you can see all attempts that where blocked by geylisting if you go to the tracking center and check the " | ||
+ | |||
+ | ==== enterprise vs. free ==== | ||
+ | PMG is free open source software with an optional enterprise subscription. For a private person, the enterprise license is too expensive, but if you use this setup for a production server in a copmany, you might want to consider getting the enterprise subscription, | ||
+ | |||
+ | by default, PMG comes with the enterprise repo pre-configured which means you won't be able to update if you don't have a subscription. if you want to use the free repo, you need to change your apt configuration: | ||
+ | rm / | ||
+ | echo "deb http:// | ||
+ | |||
+ | as a non-enterprise user you will have to run updates from the command line, while enterprise users can do it from the web-ui IIRC (i am a poor private user ;)) | ||
+ | |||
+ | ===== Create new users ===== | ||
+ | to manage user accounts, login to adminer to edit your '' | ||
+ | |||
+ | first make sure the domain is added by checking the '' | ||
+ | |||
+ | make note of the domain id of the domain you want to add a new user for. | ||
+ | |||
+ | to add a new forwarding, make a new entry in the '' | ||
+ | |||
+ | to add a new user account, make a new entry in the users table. make sure you don't forget to fill in the '' | ||
+ | dovecot pw -s SHA256-CRYPT | ||
+ | and then enter the new password. It will return a SHA hash which you can then enter in the password field. | ||
+ | |||
+ | lastly, to activate the new mail user account, send an email to that address. | ||
+ | |||
+ | ===== add a new domain ===== | ||
+ | to add a new domain to the mail system, complete the following steps: | ||
+ | - add the domain on the Proxmox Mail Gateway to the "Relay Domains" | ||
+ | - add the domain to your mailserver database, as mentioned above | ||
+ | - update the DNS records of your domain to point to the mailfitler and also add these entries: < | ||
+ | autoconfig | ||
+ | autodiscover | ||
+ | _dmarc | ||
+ | @ | ||
+ | 20200101._domainkey | ||
+ | </ |