After careful consideration, we have reluctantly taken the decision that the use of Cloudflare will not be supported on our Web and Email Hosting service. First two clarifications:

• This only applies to our Web and Email Hosting services. It does not, and will not, apply to virtual servers (VPS), dedicated servers or Raspberry Pi servers.
• We have never formally supported or encouraged the use of Cloudflare with our Web and Email service. Until now, we have generally discouraged it, but we’ve tried to accommodate users who chose to use it.

The problem

As we’ve written about previously, we’ve seen a huge increase in the amount of abusive bot traffic hitting our web servers — much of it badly behaved AI scrapers — and frequently the volume of traffic is such that it overwhelms the server, and makes websites unavailable. At times we’ve seen over 95% of traffic coming from AI scrapers. Our Web and Email Hosting service is what’s often referred to as “shared hosting” meaning that we have websites for many customers on a single server. This is a very cost effective way to provide web hosting, but it does mean that any load issues caused by traffic to one website can affect other sites on the same server.

Our primary tool for dealing with abusive traffic is to identify the source of traffic and block the IPs that it’s coming from.

Cloudflare provides a service that seeks to protect websites by blocking abusive traffic. It does this by operating a “reverse proxy”; rather than web traffic arriving directly at the web server, it is instead sent to Cloudflare’s servers. Cloudflare inspects the traffic, filters out the abusive traffic, and then forwards the legitimate traffic to the actual web server. This has the effect that all traffic arriving at the web server appears to come from Cloudflare’s IP addresses, rather than the actual client IPs.

Some of our customers have chosen to front websites hosted on our shared hosting servers with Cloudflare. The problem is that Cloudflare isn’t perfect; it doesn’t succeed in filtering out all abusive traffic. This is particularly true of the free tier that we tend to see used in conjunction with our Web and Email Hosting service.

Unfortunately, when we see a large volume of abusive traffic arriving via Cloudflare, we are faced with a choice: either we block Cloudflare’s IP addresses, knowing that this will take all websites using Cloudflare completely offline, or we accept the traffic, which potentially has an impact on all websites hosted on that server. With the growth in volume of abusive traffic, we are being forced to make this choice increasingly often.

We have now taken the decision that we will treat Cloudflare’s IPs like any other IPs: if we see abusive traffic from them, we will block them. In the future, we may introduce a permanent block, or redirect traffic to a support page on our site that explains why Cloudflare is not supported on the service in order to avoid customers inadvertently using an unsupported configuration, but we will contact customers who appear to be using Cloudflare prior to taking this step.

FAQs

Obviously these questions are not frequently asked as this is the first announcement of this change, but whatever; here are some questions and answers:

What have you got against Cloudflare?

None of this is a criticism of the service that Cloudflare provides. The same would apply to any reverse proxy service placed in front of our servers that prevents us from seeing the actual source IP, thereby removing our ability to effectively filter traffic ourselves.

We do have concerns about any service that breaks the end-to-end encryption of web traffic, but that’s unrelated to this issue. Cloudflare (and any other such service) relies on terminating the TLS connection from the client on their servers, inspecting the traffic, and then making a new secure connection to the target web server.

Why do you think you’re better at this than Cloudflare?

We’re not, but when a particular attack is affecting our ability to provide our web hosting service to hundreds of customers, we’ve got a much stronger incentive to resolve it quickly.

Cloudflare definitely has some very impressive technology, but we suspect that much of it isn’t provided in the free tier of their service. Upgrading to a paid-for tier, or careful configuration of the free tier, might yield better filtering, but in our case, we’re affected by the most permissive configuration; it only takes one customer to point Cloudflare at our server with minimal filtering and we’ve got a mix of legitimate and abusive traffic arriving from the same IPs and we’re back where we started.

Cloudflare includes the client IP address in an HTTP header – why don’t you filter on that?

Filtering based on the source IP address of a TCP/IP connection can be done very efficiently because you only need to look at the packet header. Filtering based on an HTTP header requires accepting the connection, setting up a TLS connection and decrypting the content, and then parsing the headers, which is a significant overhead, and in extreme cases, the load from doing this is prohibitive.

Can I use Cloudflare on my VPS or dedicated server?

Yes, absolutely. The issues described here that make Cloudflare such a problem for our Web and Email Hosting service don’t apply to VPSs and dedicated servers, so if you want to put Cloudflare — or any other reverse proxy — in front of your server, you are free to do so. Of course, dealing with any abusive traffic that slips through is your responsibility.

Can I use Cloudflare on my managed server?

If you have a managed server with us then we will attempt to resolve any load issues caused by abusive traffic as part of this service. Please do not install Cloudflare (or similar) in front of your managed server without discussing it with us first, as it will hamper our ability to respond to any incidents. A special case is our WordPress Shared tier of managed WordPress hosting. As the name suggests, this is implemented on our shared hosting platform, and so we cannot support the use of Cloudflare with this service.

Software-as-a-Service has transformed the way that we use computers. The ability to access services anywhere, on any device, without having to install, manage or upgrade applications has obvious advantages. You can be up-and-running on GitHub, Slack or Box in a matter of minutes, and very often there’s a free entry-level tier that requires no up-front payment at all.

Even when you move into the paid-for tiers, replacing the cycle of licence fees and upgrade fees with a simple ongoing service fee can also be attractive.

But SaaS comes with a major drawback: lock-in.

You’re at the mercy of a single provider, and your only recourse if you’re not happy with the service is a potentially complex and expensive migration to a completely different platform. It may not be easy, or even possible, to export your data from your old provider.

And those “free” entry-level tiers have to be paid for somehow. As your usage increases, costs can quickly become very expensive. Or the unprofitable cheap and free tiers get withdrawn or restricted as the business needs to move its income stream from investors to actual customers.

Self-hosted open source

Open source alternatives to many popular services exist, and the obvious alternative to SaaS is self-hosting: run your own server and install the software yourself. You’re completely in control, your data is on your servers and there’s no risk of lock-in.

But self-hosting loses many of the advantages of SaaS; you’re now responsible for running the servers, applying security patches, maintaining backups, and monitoring for issues.

Managed Open Source Services

At Mythic Beasts, we offer an alternative: MOSS – Managed Open Source Services. With MOSS, you get the convenience of Software as a Service, but without the lock-in. We manage the service, apply security updates, provide 24/7 monitoring and take regular backups, but your data remains under your control: we’ll happily give you full access to all the data on your server at any time.

By using open source software, you’re not tied to a single provider. If you’re not happy with our service you can migrate to another provider, or even switch to self-hosting, without the cost of having to familiarise users with different software.

And if you decide you do want to change software, open source puts you in a much stronger position for planning a migration.

Our pricing structure is straightforward, and tied to what it costs us to provide the service. With no free, entry-level tiers that need to be subsidised by the higher tiers, prices scale with your usage, rather than leveraging your lock-in.

We’ve been hosting exclusively on open source software for nearly 25 years, and provide Managed Open Source Services for a range of applications, including GitLab, Mattermost, and NextCloud. For more information, please see our Managed Applications page, or contact us to discuss your requirements.

Modern email involves a confusing array of different acronyms. Most of these are attempts to fix the problem of email being fundamentally insecure, with no way to authenticate the sender of an email. The remainder are attempts to fix the new problems created by attempting to fix the first problem.

This blog post tries to provide a concise glossary of these different technologies, and the associated DNS records and URLs needed to make them go.

SPF: Sender Policy Framework

Publish a list of servers that are allowed to send mail from your domain.

SPF is published as a DNS TXT record for the domain itself (an apex record), which states which servers are allowed to send mail from your domain. For example:

example.com. IN TXT "v=spf1 include:_spf.mythic-beasts.com ~all"

SPF breaks email forwarding, unless you use SRS. SPF only restricts the “envelope sender”, which is not normally visible to end users.

SRS: Sender Rewriting Scheme

Rewrite sender addresses when forwarding mail in order to avoid failing SPF checks.

SRS is a technique used when forwarding email that replaces the original sender address with an address in your own domain. SRS only affects the envelope sender, which is not normally visible to end users. SRS allows email forwarding to work with SPF.

DKIM: DomainKeys Identified Mail

Digitally sign email envelopes.

Public keys are published in DNS records, allowing recipients to verify that the email is authentic. Public keys are published as TXT records within the _domainkey subdomain. For example:

mythic-beasts-k1._domainkey.example.com. IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG...."

The first part of the hostname is a unique identifier for the key. The signature is added to the email as a DKIM-Signature header. The header includes a field (the s field) with the name of the key used to sign the message.

DMARC: Domain-based Message Authentication, Reporting and Conformance

Tell recipients to reject email from your domain if it isn’t DKIM signed and doesn’t pass SPF.

DMARC is a mechanism that allows a domain owner to assert that all email sent will pass either DKIM or SPF validation, and if it doesn’t recipients should reject it. Subtly changes SPF behaviour so that it binds the “From” address (which is the one that users usually see) rather than the envelope sender. Breaks email forwarding even with SRS unless messages are DKIM signed. Example DMARC record:

_dmarc.example.com. IN TXT "v=DMARC1; p=reject; rua=postmaster@example.com"

ARC: Authenticated Received Chain

Enable mailing lists to forward DKIM-signed emails.

ARC allows a system that forwards mail to provide a digitally-signed summary of the results of SPF, DKIM and DMARC validation at the point thta it was received by the forwarding server. This enables the intermediary system to make changes to the message (such as adding a mailing list footer) that will break the original DKIM signature, but still allow the final receiver to verify its integrity. ARC relies on recipients trusting the intermediary. ARC uses the same DNS-published DomainKeys as DKIM.

DANE: DNS-based Authentication of Named Entities

Publish TLS certificates in DNS, and require TLS when connecting to your servers.

DANE is not specific to email, but it can be used to enforce the use of secure TLS connections when mail servers talk to each other. In the absence of DANE, mail servers will generally try to use TLS if possible, but fall back on an insecure connection if it doesn’t work. By publishing a TLSA DNS record, domains can enforce that TLS is used when delivering mail to the servers listed in its MX records. DANE relies on DNSSEC, and also provides an alternative to Certificate Authority-based authentication of TLS certificates.

MTA-STS: Mail Transfer Agent Strict Transport Security

Require TLS when connecting to your servers by publishing a policy at a well known URL.

MTS-STS allows you to require secure TLS connections when mail servers connect to your server by publishing a machine-readable policy at a well-known URL (https://example.com/.well-known/mta-sts.txt). MTA-STS is an HTTPS-based alternative to the TLS-enforcement part of DANE.

As of yesterday, new support tickets are now being handled in our new support system, which is powered by Request Tracker (RT). Hopefully from the outside, things look exactly as they did before: you email support@mythic-beasts.com, your request gets assigned a ticket number, and you get a timely and helpful response from our non-dedicated support team.

From the inside, this change is pretty daunting. Our support system is absolutely critical to our day-to-day work, and learning a new tool, and adapting all of our customer-facing process to use it is a big change. There are lots of little things like the canned “snippets” that we use when composing replies to common questions, and the processes to be followed when contacting customers in response to automated alerts. Doing this from a clean slate would be hard enough, but we also have to provide continuity for existing tickets, which can remain active for weeks, months or very occasionally, years.

We’ve been tackling the migration as incrementally as possible. Most internal, and certain external tickets have been handled in RT for some weeks now, but yesterday was the day that we flipped the switch on the support@ fire hose.

Our previous system has served us well for the 16 years that we’ve been using it, but it suffers from being closed source. There have been various enhancements that we’d like to have made, but we couldn’t, and ultimately our migration away from it has been forced by the fact that the standalone product is no longer maintained, having been replaced by SaaS version – an option that is out of the question for us.

Of course, not having access to the source code has made migrating away from it harder.

Open Source or bust

We actually evaluated RT a bit over twenty years ago, and decided that it wasn’t right for us, based mostly on some requirements that in hindsight seem a bit odd (RT has also matured a bit since then!) In selecting RT this time, we carried out a thorough evaluation of available ticketing systems, with one absolutely non-negotiable requirement: must be open source and hosted in house. We simply cannot afford for such a critical and hard-to-migrate part of our business to be locked in to closed source software, let alone a SaaS service.

Our primary goal with the new system so far has been to replicate the functionality that we had before, but once it is bedded in, we plan to make further enhancements to improve integration with our other software. Whilst in the short term we hope that you’ll see no change, in the long-term we hope that customers will see improvements to how we handle support.

We’re considering adding Request Tracker to our list of managed applications. Please drop us an email if this is something that you might be interested in.

We’ve always taken a boringly old-fashioned approach to our domain registration business. We take the price that our supplier charges us, add a little bit more than it costs us to provide the service and sell them on. The price we charge covers the cost of providing support, DNS, and other things like foreign exchange and payment fees.

We don’t do introductory “loss-leaders” – selling domains at a loss for the initial period and then ramping up the price for renewals^*. Our domain business isn’t a loss leader for other services, and we don’t offer barely-profitable prices that need to be subsidised by high margin add-ons that you don’t really need.

In fact, our domain pricing structure is so boring, that we wrote a simple script to do it for us. It pulls in wholesale prices from our supplier via an API, takes the current exchange rate, adds our margin, and sets our list prices.

When we set our current price structure, there were only a small number of Generic Top-Level Domains (gTLDs) and they were mostly a similar price. Country-code TLDs (ccTLDs) are often more expensive, but they’re also often more bureaucratic and more expensive to support.

These days there’s a huge number of gTLDs with a huge range in price, with some gTLDs costing hundreds of pounds per year and the assumption that the more expensive domains are those with greater bureaucracy no longer holds.

In recognition of this, we’ve just rolled out a significant update to our pricing structure that better models our costs. The net result is that we’ve reduced the price of all non-UK domains with a one-year cost of more than £30+VAT and we’ve also increased the discount we offer for multi-year renewals and registrations across all domains.

You can view our updated prices at mythic-beasts.com/domains

The price reductions apply to renewals and transfers as well as new registrations, and as always, existing customers get the same prices as new customers.

All domain registrations include our DNS service, with a powerful DNS API, and support for DNSSEC (where supported by the gTLD).

Our sustainable pricing model means we have no need for nasty lock-in schemes. We’ve never charged transfer-out fees, and our DNS service makes it really easy to import and export your DNS records.

^* Registries do sometimes offer promotional first year pricing.  In this case, we will pass on these discounts, but make the normal price clear in our price list.

At the end of July, Debian 10 (Buster) reached end-of-life, and with it, all mainstream support for Python 2. The Python Software Foundation actually ended support for Python 2 on 1st January 2020, but it’s remained in Debian until now because a small number of important packages depend on it.

One of those packages is Mailman 2, a widely-used mailing list manager.  For various reasons, many projects using Mailman 2 have resisted upgrading to its successor, Mailman 3.  With Debian 10 reaching end-of-life, we’re seeing renewed interest in this migration.

One of the barriers to migrating to Mailman 3 is that the upgrade breaks links to messages in the mailing list archives.  There are links all over the internet to messages in Mailman 2 list archives, and for many projects, breaking these links would be a significant loss.

We’ve recently done some Mailman 2 to 3 migrations, and as part of this, we developed our own solution to preserving archive URLs.  We created a script that trawls Mailman 2 archives and creates a map of old URLs in the Mailman 2 archives to the corresponding URL in the Mailman 3 archives.  This can be used by Apache’s mod_rewrite to generate redirects for the old URLs.   The map can be converted to a DBM file for more efficient lookups.  This is important for archives containing many thousands of URLs.

We’ve made the mailman-archive-mapper script freely available on GitHub.

We offer mailman as a managed application.

 

It’s a popular meme that all mysterious internet problems are caused by issues with the Domain Name System (DNS). Like most memes, it gets over-used, but when it comes to transferring a domain between providers, the intricacies of DNS create some very real problems.

To make things easier, we’ve just rolled out a new feature to our DNS management system that allows you to fetch records from your old provider’s nameservers prior to transferring the DNS for your domain to us.

Why is this needed?

This functionality can help achieve a seamless transfer of your hosting, by working around an annoying feature of the DNS system.

DNS is the system that converts internet names (like “www.mythic-beasts.com”) into IP addresses (like “93.93.129.174”) that can be used to locate the server for a particular service. This conversion is done by nameservers, and each domain has its own nameservers, usually provided by your hosting provider.

When you transfer the hosting for your domain between providers, you’ll need to update your DNS records to point at your new web and email servers, but you will also typically change from using your old provider’s nameservers to your new provider’s.

The simple way to transfer your domain is to do these two things in one go.  Your old provider’s nameservers direct traffic for your domain at your old web and email servers, your new provider’s nameservers direct traffic at your new hosting service, so just change the nameservers for your domain from your old provider’s to your new provider’s and you’re done, right?

This approach works, but it’s not ideal for domains that are in active use because of the delays created by caching.

Caching and TTL

One of the things that makes DNS so confusing is caching. When you look up a name, you’re told to remember the answer for a set period of time. IP addresses don’t change very often, so looking up a name every single time you need it would generate a lot of unnecessary traffic, and slow things down.

All DNS records have a “Time To Live” (“TTL”). This is the number of seconds that you’re allowed to remember it for before you have to do a new lookup to see if it’s changed. In the past, TTLs were usually set to hours, days or even a week. As the Internet has become faster, the overhead of DNS lookups has become less of a problem, and TTLs of one hour or a few minutes are now common.

Although caching helps improve performance in normal use, it creates a problem when you need to make changes. When you make a change to the DNS records for your domain, it won’t be picked up immediately by all users, because some people will have the old value cached.

If you know you’re going to need to change a DNS record, you can lower the TTL in advance (for example to 60 seconds), and then, when you come to change the record, all users will pick up the change very quickly.

If you’re planning to change hosting provider, it makes sense to lower the TTL on your DNS records in advance, so that when you come to make the change, all traffic is switched from the old provider to the new provider quickly.

Changing nameservers

When you have your own domain, you need to have some nameservers to answer DNS queries. As described above, when you transfer the hosting for your domain, you will typically also switch from using your old provider’s nameservers to your new provider’s.

The domain name system keeps a record of which nameservers provide the DNS for each domain. For example, DNS for mythic-beasts.com is provided by our nameservers (ns1.mythic-beasts.com and ns2.mythic-beasts.com). The problem is that these records are also subject to caching and usually have a fixed TTL of 48 hours.

This means that even if you set a low TTL for your own records, when you change the nameservers for a domain, you have a two day period when queries for your domain might still end up at your old nameservers. If your old and new servers are serving different records, users will get a mix of different answers.

The trick to achieving a clean switch between hosting providers is to separate the move from your old provider’s nameservers to your new provider’s from changing the individual DNS records that control who provides your web and email hosting. In other words, get the old and new nameservers serving exactly the same records, so that during the 48 hour nameserver changeover period, it doesn’t matter which nameserver answers the query. Once that changeover is complete, you can switch your web and email hosting by updating low-TTL records.

Our new fetch live records feature makes it easier to copy the records from your old provider’s nameservers to ours, so that you can do a seamless nameserver handover before migrating your web and email hosting. Unfortunately, this tool can only check for commonly used records because there’s no reliable way to get a complete list. The best solution is to get an export of your current DNS records from your current provider, and use our import function, but many providers don’t have an export feature in their systems.

This stuff is hard – we’re here to help

Domain transfers, and DNS in general, are difficult and confusing. For many of our customers, changing providers is a once-per-decade thing, whereas we deal with domain transfers every single day.

We’re working hard to build tools that make the process easier, but our support team is always on hand to provide personalised help.

Yesterday, the following not-so-subtle notice appeared on the admin interface of all Mastodon instances:

The Mastodon team announced on Monday that this release was coming, so we were ready for it:

Details of the vulnerability are still limited, but from what we do know it sounds serious (“Remote account takeover“).

All our managed Mastodon instances were safely patched just over an hour after the new packages dropped. One instance gave us a bit of trouble, as the new version appeared to tickle a bug in Elasticsearch causing ES to consume all CPU on the server. After we eventually pinned down the cause, it was resolved by an upgrade of Elasticsearch. Turns out the ES upgrade didn’t fix it, and we’re still working with our customer to get this resolved.

Managed open source hosting

Open source software such as Mastodon, GitLab and Nextcloud can offer a great alternative to the lock-in associated with proprietary cloud equivalents, but the effort associated with hosting them can be significant: backups, monitoring, security patching, and the investigation and debugging required when a supposedly innocuous software upgrade leaves your CPU usage wedged at 100%.

Our managed open source hosting provides the best of both worlds: the convenience of a “cloud” solution, but without the lock-in. Your data is yours, and if you don’t like our service you can take your data and host it somewhere else (although we’re confident you won’t want to). And because there’s no lock-in, you get straightforward pricing based on the resources you’re using, rather than loss-leaders followed by price hikes once you’re hooked.

Read more about our managed hosting, or drop us an email at for more information.

We’ve just rolled out a price reduction for domain registration for the vast majority of the TLDs that we offer, including .com, .net and .org. We pay for most of our domains in US dollars, and thanks to the increasing strength of the pound against the dollar, we’ve been able to reduce our prices for all such domains by an average of just over 10%.

.ie domains

We’re also pleased to announce that we’re now able to offer .ie domain registrations. Unfortunately, ID requirements mean that we’re only able to offer these to corporate registrants, and standard .ie residency requirements apply. .ie domains have been a frustrating gap in our available TLDs for many years, so we’re very happy that we’re now at least partially able to fill it.

No-nonsense pricing

Our full price list can be found on our domains page.

We don’t offer loss-leading promotional pricing — we charge the same for new and existing customers alike, don’t ramp up pricing on renewals, and never charge transfer-out fees.

We offer small multi-year discounts for registration or renewals in advance, and pride ourselves on offering a good service for a reasonable price.

Other domains

We’re also a JISC registrar, meaning that we can provide .ac.uk and .gov.uk domains. We can provide credit accounts (subject to checks), allowing organisations to pay for domains via PO and invoice, if required.

DNS, APIs, DNSSEC, IPv6

Domain registrations include DNS with API access as standard. We also support DNSSEC, and naturally, our nameservers are IPv6-enabled. If you’re migrating existing domains to us, you can import zone files directly, via our control panel or the API. We also provide a Domain management API.

We’ve been offering IPv6-only hosting for eight years now, and have demonstrated that many websites can forego the expense of an IPv4 address pretty easily. You can read more about how we do this on this blog post from 2020. This blog post itself is being served from an IPv6-only server!

A key part of this is our IPv4-to-IPv6 proxy. This listens for incoming traffic on a shared IPv4 address and forwards it to your IPv6-only server. In order to use the proxy, you need to tell it which hostnames to listen for, and which server or servers to forward traffic to. This can be done using our control panel, and as of today, it can also be done via an API.

Having an API for proxy configuration makes it possible to automatically add or remove backend servers, allowing you to spin up additional servers, or take servers out of service for failover or maintenance.

You can also use the API to add and remove hostnames handled by the proxy, and so can be used to automate the provisioning of new services.

Fine-grained access controls

As for our DNS API and Domain API, the Proxy API provides fine-grained access control for API keys. For example, you can create an API key that only has access to a specified domain or hostname, or you can create a read-only API key if you only need to read the current configuration.

Getting started

Our IPv4-to-IPv6 proxy is available to all customers with a Mythic Beasts server, including virtual servers, Raspberry Pi servers, dedicated and colo. You can find more information on the proxy service, and the Proxy API on our support pages.