setup_basic_mailserver_with_postfix_dovecot_sieve

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
setup_basic_mailserver_with_postfix_dovecot_sieve [01.01.2020 13:13] – [postfix smtp(d) config] Pascal Sutersetup_basic_mailserver_with_postfix_dovecot_sieve [20.03.2021 07:47] (current) – [add a new domain] Pascal Suter
Line 65: Line 65:
 to make sure letsencrypt will restart all our servers once the ssh keys change, we need to add this:  to make sure letsencrypt will restart all our servers once the ssh keys change, we need to add this: 
 <code> <code>
-echo -e "\npost-hook = service postfix restart service dovecot restart ; service apache2 restart" >> /etc/letsencrypt/cli.ini+cat > /etc/letsencrypt/renewal-hooks/post/restart_services.sh <<EOF 
 +#!/bin/bash 
 +service postfix restart 
 +service dovecot reload 
 +service apache2 reload 
 +EOF 
 +chmod +x /etc/letsencrypt/renewal-hooks/post/restart_services.sh 
 </code> </code>
  
Line 312: Line 319:
   swaks --to myuser@yourdomain.ch --server mail.yourdomain.ch   swaks --to myuser@yourdomain.ch --server mail.yourdomain.ch
   mutt -f imaps://myuser@yourdomain.ch@mail.yourdomain.ch   mutt -f imaps://myuser@yourdomain.ch@mail.yourdomain.ch
 +  
 +
 +===== postfix smtp(d) config =====
 +<code>
 +postconf smtpd_sasl_type=dovecot
 +postconf smtpd_sasl_path=private/auth
 +postconf smtpd_sasl_auth_enable=yes
 +postconf smtpd_tls_security_level=may
 +postconf smtpd_tls_auth_only=yes
 +postconf smtpd_tls_cert_file=/etc/letsencrypt/live/mail.yourdomain.ch/fullchain.pem
 +postconf smtpd_tls_key_file=/etc/letsencrypt/live/mail.yourdomain.ch/privkey.pem
 +postconf smtp_tls_security_level=may
 +</code>
 +to enable submission service (port 587 for sending emails from clients) edit ''/etc/postfix/master.cf'' and uncomment the lines for the submission service. I left the restrictions commented out, because i don't want any furhter restrictions for my clients besides the need to authenticate. 
 +
 +<code>
 +submission inet n                               smtpd
 +  -o syslog_name=postfix/submission
 +  -o smtpd_tls_security_level=encrypt
 +  -o smtpd_sasl_auth_enable=yes
 +  -o smtpd_tls_auth_only=yes
 +  -o smtpd_reject_unlisted_recipient=no
 +#  -o smtpd_client_restrictions=$mua_client_restrictions
 +#  -o smtpd_helo_restrictions=$mua_helo_restrictions
 +#  -o smtpd_sender_restrictions=$mua_sender_restrictions
 +#  -o smtpd_recipient_restrictions=
 +  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
 +  -o milter_macro_daemon_name=ORIGINATING
 +</code>
 +
 +  systemctl restart postfix
 +
 +===== additional postfix settings =====
 +==== mail size limit ====
 +i think 10MB is just too small, so i allowed 30mb instead 
 +  postconf message_size_limit=31457280
 +
 +==== regex based virtual aliases ====
 +i've added [[postfix_virtual_mail_addresses_with_regular_expressions|this]] as well to my config. however, I called the config file /etc/postfix/regex_aliases.map so i can create mappings for different domains and different purposes. 
  
 ===== install roundcube ===== ===== install roundcube =====
Line 352: Line 398:
   * imap port changed to 993   * imap port changed to 993
   * imap server set to <code>ssl://mail.yourdomain.ch</code>   * imap server set to <code>ssl://mail.yourdomain.ch</code>
-  * smtp server left at default "localhost" +  * smtp server set to <code>tls://mail.yourdomain.ch</code>
-  * smtp port set to 25 instead of 587 +
-  * smtp user and password to empty string (no need to authenticate localhost smtp traffic) +
   * set database password to the one you noted down before   * set database password to the one you noted down before
   * add at least ''managesieve'' and ''password'' plugin   * add at least ''managesieve'' and ''password'' plugin
Line 398: Line 442:
 </code> </code>
  
-===== postfix smtp(dconfig =====+ 
 +===== client auto configuration ===== 
 +postponed.. [[https://workaround.org/ispmail/buster/mail-client-auto-configuration/|tutorial]] 
 + 
 +===== proxmox mail gateway ===== 
 +i decided on using proxmox mail gateway (PMGfor spam and virus filtering rather than setting up rspamd or any other mail filter solution i have to maintain myself. i've tested PMG in the past and it yielded a pretty good detection rate. my ultimate goal is that i don't need to spend too much time dealing with spam filters, they should be there and just do their job.. PMG did just that during my tests using some catchall domains to gather as much spam as i could :)  
 + 
 +i've installed PMG onto another Virtual Machine as i host a virtual host myself. if you have to pay alot of money for a vps and you already have one for your mailserver, you can also run PMG inside a LXC container, more details on the installation can be found in the admin guide.  
 + 
 +so i've downloaded the latest ISO from [[https://www.proxmox.com/en/proxmox-mail-gateway|the Proxmox webpage]] and installed my VM using this image. the installation is very easy and very fast.  
 + 
 +i set a public ip with a hostname filter.yourdomain.ch.  
 + 
 +after the installation is complete, you can access the web-interface on https://filter.yourdomain.ch:8006  
 + 
 +your root password is also your login for the web-interface. i did disable ssh password login and i've blocked all ports except 22 and 25 from the outside world in my firewall, so nobody can access the web-interface and brute-force my password.  
 + 
 +once you're logged in to the web-interface, do the basic setup. first make sure the dns and time settings are correct. you can change those by clicking on "Configuration" in the left column.  
 + 
 +once this is all set, go ahead and click on the "Mail Proxy" settings. 
 +  * under Relaying enter your mailserver's ip as "Default Relay". this is the ip to which we want to relay incoming mail from the internet after it passes all the filters.  
 +  * leave port 25 
 +  * i've disabled MX lookups, not sure why they should be needed here.  
 +  * under "Relay Domains" enter all your domains you want to accept emails for on your mailserver 
 +  * in the Options tab i've enabled "Verify Receivers" which will verify that the receiver address is actually valid before accepting the email. i've set it to "Yes(550)" to work with my above postfix setup.  
 +  * i have disabled Greylisting as this delays mail delivery significantly and that's a bit of a pain if you wait for account confirmation emails or booking confirmations etc. i'll re-enable it if the spam detection rate is too low.  
 +  * also in Options, i have enabled DNSBL and i've entered the following two blacklists to query: ''b.barracudacentral.org,zen.spamhaus.org''. please note that you need to register your dns servers at barracudacentral prior to using them and spamhaus asks you to rsync their database to your own dns if you have a high volume server.. i don't :)  
 +  * there is no need to configure any transports. this is only needed if you want to route incoming mails for different domains or addresses to different servers.  
 +  * in the networks tab, you can add the network or ip of your mailserver, in case it is not in the same subnet as your filter.. if it's in the same subnet there is no need to add anything here, as the same subnet is allowed to relay through PMG by default.  
 +  * since we will be relaying our outgoing emails from our mailserver via PMG as well, we will use PMG's DKIM signing function. to enable this, in the DKIM tab you need to **first add a new selector** before you can enable DKIM .. that's a bit confusing. as selector i've entered the current date like ''20200101'' you can be more creative if you want to. i then enabled DKIM and checked the box to sign all outgoing traffic. like this there is no need to add each of your domains separately to the DKIM domains.  
 +  * for DKIM to work you need to add a TXT entry to your domian's DNS record. you can click on "View DNS Record" to get a copy-paste snippet to paste right into your bind zone file if you're using bind as your name-server.  
 +  * once all these settings where done, i had to login to the filter via ssh and **manually restart postfix**. otherwise postfix would bind port 25 to 127.0.0.1 only. i guess rebooting the entire filter would fix this issue as well. 
 + 
 +==== tag and deliver spam instead of quarantine ==== 
 +I'm not sure I or my users would be happy with waiting for reports to find out why a recently sent mail did not reach them. after all it's always a good feeling if you can tell someone on the phone that you didn't find their email in your spam folder either, to convince them that they might have had a typo in your email address :) ..  
 + 
 +First you need to make sure that spam is no longer quarantined but instead marked and forwarded.  
 + 
 +you can either mark an email by modifying its subject or by adding a header element.. i don't like changing the visible part of the email message, so i opted to go for an additional header field that marks spam.  
 + 
 +to create it, go to the ''Mail Filter-->Action Objects'' page in the PMG web-interface and add a new action object of the ''Header Attribute'' type. start the header atribute with ''x-'' and choose something meaningful.. i went for ''x-spam-mail:yes''  
 + 
 +next we need to make sure that all spam mail is tagged with this header field instead of quarantined.   
 + 
 +in the PMG web interface click on ''Mail Filter'' in the left column. you will now see a list of all active or inactive mail filters. by selecting a filter, you can then see on the right hand side column which actions will be executed. you can simply remove the Quarantine action from the active ones and then drag and drop the newly created action object to the active actions or click the + sign to add it.  
 + 
 +now on to your postfix mail server.. we need to add a global sieve rule to dovecot that will move spam into a spam folder.. edit ''/etc/dovecot/conf.d/90-sieve.conf'' and look for "sieve_after" templates.. add a new line after those that looks like this:  
 +  sieve_after /etc/dovecot/sieve-after 
 +now create the sieve-after directory:  
 +  mkdir /etc/dovecot/sieve-after 
 +all filters found in this directory will be executed AFTER each user's own filters. so a user can create his own filters to whitelist spam in our case. 
 <code> <code>
-postconf smtpd_sasl_type=dovecot +cat > /etc/dovecot/sieve-after/spam-to-folder.sieve <<EOF 
-postconf smtpd_sasl_path=private/auth +require ["fileinto","mailbox"]; 
-postconf smtpd_sasl_auth_enable=yes + 
-postconf smtpd_tls_security_level=may +if header :contains "x-spam-mail" "yes" { 
-postconf smtpd_tls_auth_only=yes + fileinto :create "INBOX.Junk"; 
-postconf smtpd_tls_cert_file=/etc/letsencrypt/live/mail.yourdomain.ch/fullchain.pem + stop; 
-postconf smtpd_tls_key_file=/etc/letsencrypt/live/mail.yourdomain.ch/privkey.pem +
-postconf smtp_tls_security_level=may+EOF 
 +</code> 
 +now compile the sieve filter:  
 +  sievec /etc/dovecot/sieve-after/spam-to-folder.sieve 
 +lastly restart dovecot to re-read the config we altered bove  
 +  systemctl restart dovecot  
 +to test, send an email from outside to your mail account with the following line in the body:  
 +<code> 
 +XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X
 </code> </code>
-to enable submission service (port 587 for sending emails from clients) edit ''/etc/postfix/master.cf'' and uncomment the lines for the submission service. I left the restrictions commented out, because i don't want any furhter restrictions for my clients besides the need to authenticate.  
  
 +==== future improvements ====
 +=== Bayesian learning by moving mail to junk folder ===
 +in order for proxmox's Bayesian filter to learn and start working you need to provide spam and ham mails. to do this, one needs to copy spam that passed the filter to a temporary location on the proxmox mail gateway and then tell the Bayesian filter that this is spam. it needs at least 200 spam and ham mails to start working. ideally, we should be able to do this by simply moving an email to the spam folder on our imap account and the email should then be submitted to the bayesian filter of PMG automatically.. 
 +
 +some information i will need to do this: 
 +  * the command on PMG to submit a ham or spam to the learning system is <code>sa-learn --spam filename</code>
 +  * a solution on how to use imap sieve filters to trigger a bash script when a mail is moved into a specific folder can be found in the [[https://workaround.org/ispmail/buster/filtering-out-spam-with-rspamd-2/|all mighty ISPmail tutorial]]
 +
 +
 +==== adjustments to postfix settings on our mailserver ====
 +we can now limit access for incoming mails so that postfix only accepts connections from our mailfilter. to do this, edit the ''smtpd'' line in ''/etc/postfix/master.cf'' and add the following option: 
 <code> <code>
-submission inet n                               smtpd +smtp      inet  n                               smtpd 
-  -o syslog_name=postfix/submission +  -o smtpd_client_restrictions=permit_mynetworks,reject
-  -o smtpd_tls_security_level=encrypt +
-  -o smtpd_sasl_auth_enable=yes +
-  -o smtpd_tls_auth_only=yes +
-  -o smtpd_reject_unlisted_recipient=no +
- -o smtpd_client_restrictions=$mua_client_restrictions +
-#  -o smtpd_helo_restrictions=$mua_helo_restrictions +
-#  -o smtpd_sender_restrictions=$mua_sender_restrictions +
-#  -o smtpd_recipient_restrictions= +
-  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject +
-  -o milter_macro_daemon_name=ORIGINATING+
 </code> </code>
 +don't forget to restart postfix 
  
-  systemctl restart postfix+further more we can configure our mailserver to send all its mails through our proxmox gateway to allow proxmox to track outgoing mails and scan them for viruses as well. to do that we can set the ''relayhost'' accordingly:  
 +  postconf relayhost=filter.yourdomain.ch:26 
 +note port 26, that's because proxmox mail gatway distinguishes between incoming and outgoing mail by accepting them on different smtp ports. by default port 25 is for incoming and port 26 for outgoing mail. 
  
-===== client auto configuration ===== +==== greylisting ==== 
-postponed.. [[https://workaround.org/ispmail/buster/mail-client-auto-configuration/|tutorial]]+by default PMG uses greylisting. this means, that every email coming from a new sender address will first be rejected for a duration of a couple of minutes. i think 3 minutes is the actual greylist timeout on PMG. however, the delay that occurrs in reality will be dependent also on the sending mail server's retry interval.  
 + 
 +you can see all attempts that where blocked by geylisting if you go to the tracking center and check the "Include Greylist" search option, then click search.  
 + 
 +==== enterprise vs. free ==== 
 +PMG is free open source software with an optional enterprise subscriptionFor a private person, the enterprise license is too expensive, but if you use this setup for a production server in a copmany, you might want to consider getting the enterprise subscription, because besides support, it also offers access to an enterprise repository, which contains heavily tested packages whereas the non-subscription repo is kind of the "beta tester" repo, meaning, a new package gets to the no-sub repo first and only after a certain amount of time of successful distribution through the no-sub repos will it enter the enterprise repoSo you either contribute by testing new packages or by paying money :)  
 + 
 +by default, PMG comes with the enterprise repo pre-configured which means you won't be able to update if you don't have a subscription. if you want to use the free repo, you need to change your apt configuration:  
 +  rm /etc/apt/sources.list.d/pmg-enterprise.list 
 +  echo "deb http://download.proxmox.com/debian/pmg buster pmg-no-subscription" >> /etc/apt/sources.list.d/pmg-no-sub.list 
 + 
 +as a non-enterprise user you will have to run updates from the command line, while enterprise users can do it from the web-ui IIRC (i am a poor private user ;))  
 + 
 +===== Create new users ===== 
 +to manage user accounts, login to adminer to edit your ''mailserver'' database.  
 + 
 +first make sure the domain is added by checking the ''domains'' table. if the domain does not exist, add it.  
 + 
 +make note of the domain id of the domain you want to add a new user for.  
 + 
 +to add a new forwarding, make a new entry in the ''aliases'' table, make sure you don't forget to fill in the ''domain_id'' field 
 + 
 +to add a new user account, make a new entry in the users table. make sure you don't forget to fill in the ''domain_id'' field. leave ''quota'' empty or 0 in order to not enforce a mail box quota. if you want to set a quota, enter the amount of space the mailbox is allowed to use **in bytes**. for the ''password'' field, you need to generate a password hash on the command line of your mailserver. use the command  
 +  dovecot pw -s SHA256-CRYPT 
 +and then enter the new password. It will return a SHA hash which you can then enter in the password field. 
  
 +lastly, to activate the new mail user account, send an email to that address. 
  
 +===== add a new domain =====
 +to add a new domain to the mail system, complete the following steps: 
 +  - add the domain on the Proxmox Mail Gateway to the "Relay Domains" list
 +  - add the domain to your mailserver database, as mentioned above
 +  - update the DNS records of your domain to point to the mailfitler and also add these entries: <code>
 +autoconfig              IN      CNAME   mail
 +autodiscover            IN      CNAME   mail
 +_dmarc                  IN      TXT     "v=DMARC1; p=quarantine; pct=100"
 +@                       IN      TXT     "v=spf1 mx ~all"
 +00000000._domainkey     IN      TXT     ( "v=DKIM1; h=sha256; k=rsa; ".... copy/paste from PMG
 +</code>you can get the correct entry for he DKIM1 key from your proxmox mail gateway web gui, just go to Configuration --> Mail Proxy --> DKIM and click on the "view DNS record" button, then copy/paste the entry to your bind config. 
  • setup_basic_mailserver_with_postfix_dovecot_sieve.1577880789.txt.gz
  • Last modified: 01.01.2020 13:13
  • by Pascal Suter