Setting up email properly with SPF/DKIM/DMARC

Webmaster Discussion
#1

In the past month I’ve migrated my server from TSS to MojoHost and part of that was setting up email services on my new server (as opposed to using TSS’ mail server). Having mail services on my own server made me delve into how to do email properly since I need a good reputation as a sending server to get my emails though to their recipients. I thought I’d share a bit of what I learned. There were definitely things I learned that can help…

The first thing I learned was that you’ll need reverse DNS entries for your IP address (in simple terms your IP address needs to have a name) or receiving mail servers will reject the email you send them. MojoHost had set up reverse DNS for my IPv4 address, but not my IPv6 address and it caused bounces.

The next thing I discovered was that one of the big mail services in Germany (GMX – they own mail.com) will refuse your email if your reverse DNS looks auto-generated. MojoHost names their servers things like cs1234.mojohost.com – and that looks auto-generated to GMX, so I had to purchase a new generic domain (since both both adult and mainstream stuff is served/sent from my server). So my server is now named dionysus.ampelus.net. In Greek mythology Ampelus was a satyr that Dionysus was in love with. Ampelus mocked the moon god while riding a bull. She caused the bull to stumble, Ampelus was gored by the horn of the bull and died. Dionysus was so sad that he made a grape vine out of Ampelus’ body, and turned his blood into the first wine. Point being – have some fun with the name of your server!

But, then I found that another mail service won’t receive emails from servers on newly registered domains. So I still have a problem, but that one will go away at some point.

Next – there are three types of things that need to be set up in DNS…

SPF – tells mail recipients what servers are allowed to send email for your domain and how strict to be if they see email from a server not on the list.

DKIM – a cryptographic key that’s embedded in your email that recipients can validate so they know the sending server hasn’t been forged.

DMARC – a set of rules that tells recipients how to handle failures and who to report the failures to.

There are lots of tutorials and other tools out there to help you configure SPF/DKIM/DMARC – that’s not the point of this point of this post so I won’t go into detail about them. If you want details I recommend MXToolbox.com. But absolutely do go through the effort. Doing so will get more of your emails delivered (and not put in the spam folder).

But I did learn that it’s important to set up SPF/DKIM/DMARC on ALL your domains – even the ones that don’t send or receive email. Spammers will randomly pick domains and try to pretend to be that domain when sending their spam. You can make that spam fail if you set up the following DNS TXT records on the domains that have no outbound email…

SPF
yourdomain.com “v=spf1 -all”

That says reject email from all senders. (But you don’t stop there…)

DKIM
*._domainkey.yourdomain.com “v=DKIM; p=”

That says use DKIM, but there is no encryption key, which will cause DKIM to always fail.

DMARC
_dmarc.yourdomain.com “v=DMARC1; p=reject; rua=mailto:[email protected]; fo=1; aspf=s; adkim=s; ruf=mailto:[email protected]; rf=afrf;”

That says don’t just put the emails in the spam folder – reject them completely if they fail the following tests. Be strict about SPF. Be strict about DKIM. Send samples of emails that fail either the SPF or DKIM test to [email protected]. And send summary reports in XML format to [email protected].

Then, to receive the reports you’ll need this TXT record set up on youremaildomain.com…

DMARC
yourdomain.com._report._dmarc.youremaildomain “v=DMARC1”

That says youremaildomain.com wants to receive DMARC reports for yourdomain.com.

When all of that is set up you’ll start getting reports. While the XML reports are more-or-less human readable, you really need some sort of tool to use them properly. But the example emails are somewhat amusing. This morning I got a report of spam sent pretending to be from one of my domains. The subject was encoded, but when I finally figured out how to decode it it was “Unis pour demain avec -30 %”. But it’s nice to know that spam never got to its intended target.

Anyway, I hope all that helps someone. Before all this started I had set up SPF, and knew about DKIM, but didn’t know how to get DKIM set up, or anything about DMARC, and didn’t know all those things should be set up on domains with no email.

#2

Jesus… you’re way above my level…

Why do you use your own mail servers? I moved all of my emails to Google Apps, works a charm.

#3

Well, I was using the NatNet/TSS mail server for inbound mail and for outbound mail (for my forum site, etc.) I was a bit sloppy and was just letting PHP send on its own (not through an SMTP server). But receiving mail servers can detect that, my emails didn’t look at legit as they should have.

Honestly I didn’t plan on having all the mail services on my server. When I migrated to MojoHost I briefly mentioned email (thinking they’d set me up on their mail server like NatNet/TSS) but before I knew it they were all set up on my server. So I figured I’d just figure it out and run with it. I asked for DKIM keys and they were no problem, so I just continued to run with it until I got it right. It’s really not all that complicated.

But what you see above isn’t the 1/2 of it. My forum site has >60K members and is 10 years old now and a lot of the emails have gone bad. Because people had signed up for email notifications when there were updates to posts they were following I was sending probably hundreds of emails every day that were bouncing. So having my own mail server made me care more about those bounces and I’ve figured out how to login with PHP, read the email inbox, and parse the messages to figure out which are bouncing, whether they’re a soft bounce, hard bounce or there’s a reputation problem with my mail server, and then I disable those emails. Now I’m working on more stuff that involves reading stuff in the email inboxes.

Yeah, I could use Google, but that doesn’t really make my life all that much simpler. I’d still need to set up all the DNS records properly with SPF/DKIM/DMARC. I’d still need to clean up bouncing emails.

#4

Oh of course, yes it does get a lot more complicated when you have things like you do. I never got into sending out notification by emails or newsletters.

#5

Heh. Maybe we are separated at birth, Jay. :wink: I had go through all the SPF, DKIM, DMARC stuff at dudesnude, for all the reasons you outline. It was soo much work, and not made much easier by the endless confusing info out there. Your post summarized it nicely!

#6

Since you’ve revived the thread. I’ll add some of what I’ve learned since I wrote the post…

  • Some mail providers (e.g. Deutsche Telekom) won't accept your email if you've got private WHOIS on the domain in your reverse DNS. I thought of going back to a something.mojohost.com server name, but mojohost.com has all the details redacted as well. I'm hoping these providers will soften their stance as the domain ages. But given that I have a little mainstream content on my server, the domain I'm using for the server name is the one domain where I need private registration.
  • Deutsche Telekom also didn't like the name of my server. They wanted it to be mail.something.com. But they were less insistent on that. IMHO, it just shows the lack of creative thinking in big corporations since my server name is quite descriptive if you know Greek mythology.
  • Then there's BellSouth that rejects my email saying my reverse DNS isn't set up when it is set up and matches the sender in the header perfectly. They also tell their customers "we're not blocking email from that server" when they are. But shortly after my member complained, he was able to receive email from me.
#7

Really? Maybe I’m misunderstanding - my mail comes from dudesnude.com and it’s private reg… and my emails seem to get through to @t-online.de (which I think is them)… ?

#8

Right, I suspect it’s just a rule for new domains. dudesnude.com is ~18 years old, I’m sure they have different rules for new and old domains.

To a point I felt like I was playing whack-a-mole. Deutsche Telekom probably would have been fine with the original server name of cs2172.mojohost.com, but I changed it to please GMX (a different German email provider). I wrote Deutsche Telekom and gave them my contact details, but they insisted it had to be in WHOIS or on the home page of ampelus.net. Since no users were complaining, I ignored them. I suspected once my domain ages a bit they’ll be fine with it. If a user does complain then I’ll have them contact DT. I think ultimately their paying customers have more influence than server admins.

#9

Ah, gotcha. Yeah, what a hassle. For my forums I ended up just signing up for a 3rd-party mail service, Mailjet. They handle all the configuration blah blah and it has been working so well, I wish I had just started with that for handling all mail on my site (not just forums), instead of spending so much time getting my mail server working right. Would this be an option for you? (And yeah, I checked: they do allow adult content to be sent through their services.)

#10

How is MailJet able to mark emails in you system as bouncing and get the member to update their email address?

My strategy the last few years has been that if it’s important it’s gotta be custom code that I write myself. Th exception is the two frameworks I use (Fat Free Framework & Bootstrap), and I also use certain other 3rd party code for small things, where if I couldn’t use them, it wouldn’t be a major problem. But my experience with WordPress over the years makes me really hate dependence on on other people’s code.

#11

@jay

Mailjet tells my forum software (Discourse) about bounces, and stops there. But I only use Mailjet for forum emails. For all other site emails (transactional), I use my own SMTP server with SPF DKIM DMARC all set up. Then I capture the bounces and go from there.