The Death of Email Forwarding

While it’s Spam™ rather than spam, the proportion is about right.
(Image from Eduardo Unda-Sanzana from Antofagasta, Chile, CC BY 2.0 via Wikimedia Commons).

A History Lesson

Back in the mists of time, not that long ago, you could happily forward email to another account sending all your email to a single mailbox and replying through your internet provider’s SMTP server with whatever address you wanted as the sender.

Unfortunately, back in the 1970s, someone realised you could send that new fancy ‘electronic mail’ to people you didn’t know. The few hundred users affected were rather negative about it, and it didn’t really catch on. But, within a couple of decades, the ARPANET became the Internet, and unsolicited email started to become annoying.

With the ‘Information Superhighway’ appearing in the mid 90s and the growth of personal computers, the Internet started to grow rapidly, and ‘spamming’ (a term adopted from Usenet) became more common.

Blacklists and whitelists were developed, with lists of known spammer email addresses and IPs which were known to send spam, but the fact is that email addresses could be trivially faked, and there were always other servers to send mail through.

Over the years there have been various different approaches to authenticating email senders, but it’s a continual arms race. Spam remains a problem, as does the plethora of scams and fraud that are enabled by the ease with which email can be forged.

SPF & SRS

In the early 2000s,  ‘Sender Policy Framework’ (SPF) started to gain traction. SPF works by publishing a DNS record that declares which servers are allowed to send mail from your domain.

Unfortunately, SPF had two fatal flaws:

1. Pretty much by design, it breaks email forwarding. You can’t, for example, use our servers to simply forward mail to your Gmail account, because our servers aren’t allowed to send mail from the original sender’s domain.
2. SPF restricts the envelope sender, which is the hidden address that bounce messages get sent to. The one that users actually see in their mail client – the “From” address – is still unrestricted!

(1) could be fixed by another Three Letter Acronym: SRS or ‘Sender Rewriting Scheme’. As the name suggests, this replaces the original sender address with a different address in your own domain, and because this sender is usually hidden, the received mail looks no different. We’ve supported SRS forwarding on our servers for a long time, and until recently, it worked well.

(2) was a trickier problem, which we’ll come back to…

DKIM

In the 2010s and with processing becoming cheaper, ‘DomainKeys Identified Mail’ (DKIM) also appeared, which electronically signs the mail so that the remote server can verify the signature. The public keys are published in the sender’s domain’s DNS, so recipients can verify that mail came from the claimed sender.

DMARC

The issue with DKIM is that it’s not universally adopted, so whilst you can check if a signed mail is legitimate, you can’t assume that an unsigned email isn’t. Enter yet another acronym: DMARC (‘Domain-based Message Authentication, Reporting and Conformance’).  DMARC allows a domain-owner to publish yet another DNS record which says “all my mail will either be DKIM signed, or pass SPF, or both, and if it isn’t, bin it!”

Adoption of DMARC has been slow, not helped by the fact that publishing a strict DMARC policy in 2014 broke just about every mailing list of the time. Fast-forward ten years, mailing lists have learned to adapt, and the big mail providers are pushing the world towards DMARC. Gmail, probably the largest free email provider, now requires either SPF or DKIM to be enabled on a domain in order for them to accept any mail from it, and if you’re sending bulk email, you need a DMARC record too.

The Effect On Forwarding Mail

While these measures have all helped greatly to combat spam, with a large proportion of it now coming from compromised servers or mailboxes, it’s had some down-sides, the most obvious of which is difficulty in forwarding email.

Lots of our customers like to register a domain with us, set up some email addresses in that domain, and forward them to existing email accounts at other providers, such as Gmail or Yahoo. We also host lots of clubs and other organisations that have role-based addresses (like “treasurer@myclub.org”) that forward to the person currently doing that role, and mail expanders like “committee@myclub.org” that forward to a handful of people.

DMARC alignment

SPF is now widely adopted and, as noted above, you used to be able to work around the issue that it creates for forwarding using SRS.

But, one of the subtle features of DMARC is that it quietly fixes problem (2) with SPF. In order to pass SPF in a way that makes DMARC happy, the hidden sender address that SPF restricts has to be in the same domain as the visible “From” address (this is called “alignment”).

• If you forward without SRS, the messages that you forward will fail an SPF check.
• If you forward with SRS, the messages will technically pass SPF, but DMARC will call it a fail because the sender and from addresses don’t match.

In other words, there is no way to pass a DMARC check using SPF when forwarding mail. You have to rely on DKIM, and hope that forwarding mail (with or without SRS) doesn’t break DKIM.

You would hope that senders wouldn’t publish a strict DMARC policy without first ensuring that all of their outbound mail is DKIM-signed, but you may be disappointed, and with mail providers increasingly enforcing DMARC policies, mail forwarding has become less and less reliable.

Yet another acronym – ARC – promises to fix mail forwarding, but it’s not a magic bullet. We’ll discuss that another time.

Of course, forwarding mail works fine if you’re in control of the server you’re forwarding to, and can whitelist the server doing the forwarding, but that’s not common. Most people want to forward to providers like Gmail where they have no such control.

For some platforms there is a workaround; rather that forwarding mail, have the platform collect it from a mailbox using POP3 or IMAP.  For example, Gmail have their “check messages from other accounts” functionality in their mailboxes, which picks up you mail from a mailbox as if it’s been received by the Gmail mailbox directly, and other platforms are adding similar options.

This works well enough for personal email addresses, but it’s a lot more hassle for things like role-based addresses at emails at clubs and societies, and doesn’t work at all for mail expanders where multiple recipients need to collect the mail.

This is why we can’t have nice things.