As if it’s not bad enough that we have to waste a huge amount of time, not to mention a non-trivial amount of hardware, bandwidth and electricity, trying to deal with spam, we also waste a fair amount of time dealing with the “ingenious” ways that the anti-spam brigade come up with to stop legitimate mail from getting through.

This week’s contribution was so special that it took three of us to confirm that they really were doing something as silly as it first appeared.

First some background: the IP address that you get given by your ISP for your broadband connection is usually dynamically allocated. This means that it may change every time you reconnect, and may be reallocated to other users when you’re not using it. This obviously makes it impossible to selectively blacklist the users of such addresses in response to spam complaints, so it is common practice for mail servers to block connections from all IPs that are known to be allocated on this basis, using something like the PBL. Users of such IP addresses are expected to use their ISP or hosting provider’s mail servers to send outgoing mail, and the administrators of those servers take responsibility for policing their customers (on pain of having their mail servers blacklisted).

Today, a customer complained that a legitimate email being sent via our server in this manner (using authenticated SMTP) was being blocked. On closer inspection, it turned out that the IP addresses that the receiving server was objecting to was not our server’s IP address, but the sender’s IP address on the basis of it having a “poor reputation”. Well duh – it’s a dynamically allocated IP: there’s a decent chance that at some point in its life it’s been allocated to an infected computer and used to send spam. They’s why you don’t accept mail from them directly, right?

The stupid thing is that the only way that the receiver knows the originating IP address is through a “Received” header that we add. That’s right, we could trivially defeat this anti-spam measure by configuring our mail server to not add the header.

Of course we’re not going to do that, as it would break the incredibly useful trace that is provided by the received headers, and is one of the few things that helps keep sane mail admins sane.

If you want to hear what we’ve got to say, but can only cope with 140 characters at a time then you can now follow us on Twitter.

We may use Twitter to give updates on service status when we’ve got problems, but the official source for such information remains our status page.

A few days ago we unveiled our IPv6 Health Check tool, and it very quickly proved its worth.

There are plenty of other IPv6 website checkers already out there that do a cursory check to make sure that you have some IPv6 addresses for your website, nameservers, and mail servers. Our checker attempts to dig a little deeper. Do your webservers actually respond over IPv6? On all addresses? Do all your MXs have working IPv6 reverse DNS? Are your DNS entries dependent on other zones that don’t have IPv6 nameservers?

One user pointed us towards the results for one of the Regional Internet Registries, initially because it broke the checker. A few bug-fixes (both in our code, and in CPAN modules) later, and we’d determined that there was a real problem behind it: www had two AAAA records, both servers were up, but only one was accepting connections on its IPv6 address. Connections to the other server eventually timed out.

Although this issue would cause real problems to an IPv6-only user, this is exactly the kind of problem that the Happy Eyeballs (aka Fast Fallback) algorithm does a perfect job of masking. If you pick the duff IPv6 server out of the DNS, it’ll almost immediately and silently fall back on an IPv4 server. Even if you use a tool like SixOrNot to show you what connection got used, it may not be obvious that something is amiss, as falling back to IPv4 becomes part of normal operation.

Even in a dual-stack world, such a problem isn’t without side effects, as it would likely lead to an imbalance in the load spread between the two web servers.

We’re continuing to broaden the range of tests performed by the tool in order to help catch the less obvious problems that can occur when IPv6-enabling your site.

Ever wondered whether users would get to your website in an IPv6-only world? Well, now you can find out. Our IPv6 Health Check tool checks not only that your web and mail servers are accessible via IPv6 addresses, but also that you can obtain those addresses from DNS using IPv6.

You can try it out by entering a domain name below:

Should you care? Well, typing domains for a few well-known websites into the checker reveals that the IPv6 Internet isn’t currently a very exciting place, so it’s going to be a little while before not having an IPv6 presence becomes a problem. On the other hand, there’s a growing number of users with both IPv4 and IPv6 connectivity, with the latter being preferred. Google recently announced that over 2% of their traffic was now over IPv6¹, so if you’re going to list IPv6 addresses for your servers then it’s important that they work.

1. Although you still need an IPv4 connection to find Google in the first place.

I’ve been keeping an eye on the effectiveness of the blacklist that we recently installed to block spam from Communicado Ltd.

The number of messages directed at our servers seems to vary significantly, but we’ve seen close to 1,000 in a single day rejected by the filter. Whilst not a huge number in the grand scheme of things (our servers reject several connections per second using IP blacklists), it’s a pretty significant number to be spread amongst a relatively small number of customers.

What we have noticed is that the domains that we’re now seeing have been registered increasingly recently, suggesting that the older domains are becoming unusable due to people blocking them. So, we have an arms race between our ability to keep our blacklist up to date, and their ability to keep buying and deploying domains. This is actually a good thing because the domains cost the spammers at Communicado real money, so if enough people use the blacklists and keep them up-to-date then sending spam in this way will become uneconomical. In the meantime, the only consolation of the spammers chucking all this money at Nominet, is that we occassionally get to drink it.

I did some mining of our mail logs, and identified another half dozen Communicado domains. Martin fed these into his Nominet search tool yielding another 1,000 or so domains, so we’ve now got over 5,000 on the list.

If you run a mail server and aren’t already using it, please add the blacklist to your configuration and keep it up-to-date.

We’ve just put in place a new spam blacklist. What’s unusual about this list is that it’s a list of sender domains. Filtering on sender domains is not normally something we’d consider since standard operating procedure for spammers is to use a fake sender address containing a legitimate domain.

In this case, it seems that a particular company, Communicado Ltd, has gone to the trouble of registering a very large number of UK domains specifically for the purposes of spamming. We first noticed this in response to a customer complaint last week, and the thing that got my interest was that the various different domains had valid SPF records. For example:

$ host -t txt hurvabne.co.uk
hurvabne.co.uk descriptive text "v=spf1 a mx ip4:76.73.88.0/24 ip4:76.73.91.0/24 ~all"

SPF isn’t the world’s greatest anti-spam measure, but a pass on a record with specific IPs is generally a pretty positive indicator, as you have to either compromise the mail server or the DNS server – or own the domain.

What I didn’t twig was just how many domains were involved, or that they were all owned by the same company, until one of my colleagues came across Martin A. Brooks’ blog post, which identifies over 4,500 domains owned by this single outfit.

Martin has kindly shared his list of domains and we’re now filtering using it.

For what it’s worth, those domains will have cost them the best part of £17k for a year’s registration (somehow, I don’t think they’ll be renewed).

We’ve just extended our range of Virtual Servers to include a 512MB “VS Lite” option for just £7 + VAT per month – or the equivalent of £5.83/month if you pay annually.

We’ve been doing Virtual Servers for 10 years now, and we’ve always hosted them on machines with hot-swappable hardware RAID, meaning that we can replace failed disks without customers even being aware that there was a problem. One of the great things about virtualisation is that reliability shares really well – but it still adds a bit to the cost

A large proporption of our Dedicated Server customers use machines without hot-swappable drives, and we thought we’d extend the option of using cheaper hardware to our Virtual Server range in the form of the VS Lite, our most affordable Virtual Server yet.

The VS Lite host hardware still has monitored RAID, so we can cope with disk failures without data loss^*, but it requires downtime in order to replace the drive.

Our standard Virtual Servers start at £12.50 + VAT per month for a 1GB server, and all run RAID-10 on hot-swappable drives, and offer a choice of data centres, allowing you to spread multiple servers across geographically diverse sites.

In the unlikely event that even £7/month sounds expensive, then we’ve got an even more affordable alternative with a bit of a twist up our sleeves which we’ll be unveiling shortly. The first person to guess what we’re planning can have one free for a year, and don’t worry, as with all our Virtual Servers you can upgrade and downgrade easily, so don’t be put off by the prospect of an even better offer in the pipeline!

On the other hand, there are no prizes for guessing what hardware we’re using for the VS Lite host servers…

* RAID is not backup.

Last year we announced some improvements to the Mythic Beasts DNS API, and I asserted that this made it good for implementing a Dynamic DNS service. Dynamic DNS is simply a mechanism for programmatically updating a DNS record, typically used to provide a consistent name for a computer that is at the end of an internet connection with a dynamically assigned IP address.

Well, last weekend I had the opportunity to try implementing a Dynamic DNS service with our API, and realised that it actually makes the task unduly difficult. It can be done, but in order to change a record, you need to remove the old record, and to remove the old record you need to know what it is currently. This meant that you had to use the LIST command, grep out the old record, and then issue the necessary DELETE and ADD commands. Aside from being hassle, it introduces an unavoidable race condiition between the LIST and DELETE commands.

We’ve now implemented the obvious fix: a REPLACE command, which replaces all existing records for the specified host and type, and replaces them with the one provided. Obviously this doesn’t work if for some reason you want multiple records for a single host, but for the obvious use case it means that Dynamic DNS can be handled in a single command:

curl --data "domain=MY_DOMAIN&password=MY_PASSWORD&command=REPLACE \
myhost 300 A 1.2.3.4" https://ctrlpanel.mythic-beasts.com/customer/primarydnsapi

The DNS API is a standard feature included with all Mythic Beasts domain registrations. Full documentation can be be found here.

All Mythic Beasts domain registrations include access to our DNS service, which includes a number of powerful features.  One of these is an API that allows you to programmatically make changes to the DNS records for your domain.  This has a number of uses including:

• Automating DNS-based failover
• Scripting the redirection of requests to another server to display a holding message during maintenance periods
• Implementing “dynamic DNS” so that you can have a static hostname for a dynamically allocated IP address

The DNS API provides a simple, HTTP-based API, that is designed to be trivial to script.  We’ve recently made some improvements to the API.  The first is that where multiple commands are included in a single request, they are treated as an atomic transaction, meaning that they either all succeed or all fail, and there is no risk of the changes being made live with only some of the commands completed.  This means that you can change a DNS record by including a DELETE and an ADD command in a single request.

The other change is that the API now makes use of HTTP error codes to reflect the results of the transaction.  If you get a 200 response, all of the commands succeeded.  If you get anything else, then none of the commands will take effect.

You can find full details of the API, including some sample Perl code on our support pages. It’s also pretty easy to script it on the command line. Here’s an example of how to change a CNAME record using cURL:

curl --data 'domain=example.com&password=s3cr3t&command=DELETE www 86400 CNAME live.example.com.&command=DELETE www 86400 CNAME backup.example.com.'   https://secure.mythic-beasts.com/control/fcgi/customer/primarydnsapi

Earlier this year we installed some new hardware for our Virtual Dedicated Server cluster.  This has allowed us to not just double, but quadruple the amount of memory available on each of our VDS options.  Our base level VDS now comes with 1GB of RAM and is priced at £15/month including VAT.  In fact, if you pay for a year up front you can get it for the equivalent of just £12.50/month.

Naturally we’ll be offering these upgrades free of charge to our existing customers, but in order to do this we do need to move some servers onto different hardware.  We’ll be in touch shortly with a migration plan, although if you’re particularly keen to get an upgrade please drop support an email.