HOWTO setup ubuntu 18.04 sendmail + DKIM + SPF + DMARC or how I managed to get into the Gmail bastion

Foreword

There are 2 separate domains on my server. Prior to the Roskomnadzor blocking history, we collected traffic of about 2,000 visits per day, and the mail server sent about 200 emails per day to all popular mail services, incl. Google and Yandex. Everything was great. But as they say in the famous video: “Everything was so good until Navalny Roskomnadzor came!

Now, when we managed to find a hoster whose IP addresses do not fall under the state censorship blocking, another problem has appeared. Google Good Corporation has ceased to let our postman on the threshold.

The google MX server throws out my mail with something like the following message: “Your message is like an unwanted message, bye bye ...”


I sent a test message with this command:

 echo "Subject: Hello baby!" | sendmail -v legale.legale@gmail.com 

I will not write about my dances with a tambourine in an attempt to break through the spam filters. I can only say that I was taken all night, but I could only succeed the next day. That is why I decided to write this manual.

INSTALLATION INSTRUCTIONS SPF + DKIM ON UBUNTU 18.04

The steepness of this manual is that it works, unlike many others. I guarantee that you can configure at least 2 domains on 1 server without much difficulty.

1. Install the necessary packages

 apt-get install sendmail opendkim -y 

2. Configure opendkim

The config is here: /etc/opendkim.conf

3. Set up keys and signing rules

I will describe the procedure for creating your own keys for those who want complete autonomy. Personally, I use the Yandex Mail domain service, pdd.yandex.ru, so I have the keys generated by Yandex.

 #   mkdir -p /etc/opendkim/keys/*****.ru #  cd /etc/opendkim/keys/******.ru #  #-s   -d      opendkim-genkey -s mail -d ******.ru #  2  mail.txt     mail.private   #    opendkim:opendkim chown opendkim:opendkim mail.* #     chmod 600 mail.private 

Next, we connect our DKIM secret key created or uploaded from Yandex.

We are interested in these three lines from the opendkim config:

 InternalHosts refile:/etc/opendkim/TrustedHosts KeyTable refile:/etc/opendkim/KeyTable SigningTable refile:/etc/opendkim/SigningTable 

The first /etc/opendkim/TrustedHosts stores hosts that will have access to the opendkim server for signing emails.


The second /etc/opendkim/KeyTable stores a table of secret keys and associated DKIM entries, which are the following: [selector]._domainkey.[domain_name]


In the third /etc/opendkim/SigningTable is a table of rules for signing letters. Here it is indicated whose letters and with what key to sign.


Create these files by specifying your domain and your path to the key file. This completes the opendkim setting.

4. Configure sendmail

Setting up sendmail is as simple as possible. We just need to add the following lines to the end of the /etc/mail/sendmail.mc file prototype:

 #    starttls include(`/etc/mail/tls/starttls.m4')dnl # sendmail    opendkim INPUT_MAIL_FILTER(`opendkim', `S=inet:8891@localhost')dnl 

For the default mail to be sent from the correct domain, you need to make another change in the /etc/mail/sendmail.mc file

We are looking for a line similar to this: MASQUERADE_AS(`st-goods.ru')dnl Replace with the desired domain. This domain will be used as the default return address. Note that the quotes there are different, strictly speaking, the first ` sign is called gravis. So for some reason it’s got in the sendmail config.

Now we will add entries to the / etc / hosts file. This is necessary so that sendmail passes messages through the filter. It has been empirically established that if no entries are added, all logs remain pristine clean, not showing a single error, but the letters are not signed.

 #ipv4 echo -e "127.0.0.1 st-goods.ru" >> /etc/hosts #ipv6 echo -e "::1 st-goods.ru" >> /etc/hosts 

Now you need to rebuild sendmail with new settings.

 #  #    "" sendmailconfig #  opendkim  sendmail service opendkim restart && service sendmail restart 

Now our sendmail server is able to pass outgoing emails through the opendkim server, which adds a digital signature and encrypts the headers of the sent emails. It remains to publish the public key for the receiving party so that you can decrypt the headers and make sure that the headers are not changed.

5. Settings DKIM TXT records on DNS server

The key must be hung on the DNS server in the TXT type string as follows:
host: mail._domainkey
value: v=DKIM1; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2wtGTw/5KPjtlIEh282JY7ovxZ/8eqveFn9ivhzpYJldl3fBEOKw v=DKIM1; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC2wtGTw/5KPjtlIEh282JY7ovxZ/8eqveFn9ivhzpYJldl3fBEOKw

Explanations:
p= this is our public key
v= determines the version number of DKIM
t= sets the flags, by default the flags are not set. There are 2 flags 'y' and 's'.
y says that DKIM is running in test mode.
s means the mode when the right side of the email address after @ must completely coincide with the domain specified in the d = DKIM parameter of the letter signature.

The host must be specified as follows: [selector]._domainkey
We have a mail selector, so the host will be like this: mail._domainkey

The public key must be /etc/opendkim/keys/st-goods.ru/mail.txt from the file /etc/opendkim/keys/st-goods.ru/mail.txt


The key goes from p = to the last quote. The quotes with spaces must be removed.

6. Configure SPF

SPF (Sender Policy Frameword) is another 1 anti-spam technology, I will not talk about it. Only 2 words to understand the essence of this technology. The DNS records of the domain are similar to the DKIM public key. The addresses of servers that have the right to send mail from this domain are recorded. We need to register in this entry ip address of your server. In my case, a Yandex server has also been added to the list.

Here is an example of my record:
host: @
value: v=spf1 a mx include:_spf.yandex.net ip4:173.249.51.13 ip6:2a02:c207:2018:3546::1 ~all

The syntax is simple. First, the spf version is specified, and then the spaces of the servers that have the right to send mail from this domain are indicated with a space, with or without a plus. I have a and mx records, which means that servers from DNS A and MX records are also on the allowed list. Via include: the address of the Yandex server is specified, and then the ipv4 and ipv6 addresses of my server. ~ all means softfail "soft failure". Those. all addresses that are not listed in the resolved can still pass at the discretion of the receiving server. -all would mean a hard failure.

7. Configure DMARC

The last nail to penetrate the gmail bastion is the DMARC. DMARC sets what to do with emails that do not pass SPF or DKIM checks.

We do everything just as well through a DNS TXT record.
host: _dmarc
value: v=DMARC1; p=none v=DMARC1; p=none

Here we establish that if the letters did not pass SPF or DKIM, then nothing needs to be done. You can put p = reject. Then such letters will be rejected.

Check

We send a letter. echo "Subject: Hello baby!" | sendmail -v legale.legale@gmail.com

Enjoying recent mail log entries. tail -f /var/log/mail.log | grep dkim
A line similar to this should appear:

 Jun 11 22:07:55 sevenlight opendkim[6473]: w5BK7sl9008069: DKIM-Signature field added (s=mail, d=st-goods.ru) 

If there is a string, then the sendmail server and opendkim worked together and signed your letter. If there is no such line, see the “Possible Problems” section.

Now we look what happened in the mailbox. Open the letter and press the button with the down arrow in the upper right corner of the letter. In the drop-down menu, select the item “Show original”.

That's what happened with me:

 Message ID <201806112007.w5BK7sUS008068@******.ru> Created on: 11 June 2018 at 22:07 (Delivered after 2 seconds) From: info@*****.ru To: legale.legale@gmail.com Subject:  №2221 -   ******.ru SPF: PASS with IP 2a02:c207:2018:3546:0:0:0:1 Learn more DKIM: 'PASS' with domain *****.ru Learn more DMARC: 'PASS' Learn more 

PROFIT $$$

Possible problems

I sincerely hope that there will be no need to read this section, but the technique is a difficult thing ... Sometimes it is not clear who works for whom.

First, we check everything you did in the exact sequence specified in the instructions.
First, we check the / etc / hosts file for correctness of changes in it. I had the most problems there. Next, check the system log for errors dkim. While I was picking, I happened to face the situation of the impossibility of reading the opendkim secret key, although I appointed the user opendkim as the owner of the file. Then you should carefully study the sendmailconfig command execution log. I had a case in which I used a regular single quote instead of a fixed gravis, and sendmailconfig cursed it. If nothing helped write me and keep trying.