One of our most common support requests recently is for PHP 8 on hosting accounts. Until now, our policy has been to run our hosting servers on a stable release of the Debian operating system, and to only install operating system-supplied packages. The ensures that we have a reliable, stable platform that it is fully covered by Debian’s security updates process.

Our hosting servers are currently on Debian 10 (Buster) which means PHP is stuck on version 7.3. Debian takes a pretty conservative approach to updates. Not so much “if it ain’t broke, don’t fix it” but more like “if it’s broken, but not a security hazard, still don’t fix it”. This is an excellent way to manage a stable, reliable operating system.

On the other hand, PHP 8 was released at the end of 2020, and it seems that an increasing number of developers are now dropping support for PHP 7 in their products. We find it odd that developers would drop support for a current stable version of what is probably the world’s most widely use server-side OS, but nonetheless we can’t ignore the increasing number of our customers who need a more recent version.

Choose your own version

We decided that if we were going to support newer versions of PHP, we’re going to do it properly and it’s now possible for users of our hosting accounts to select which version of PHP they use using our control panel.

The PHP version can be selected independently for each website hosted, and changes take effect immediately, making it easy to test migrations to a newer version, and roll-back if problems are encountered.

Our hosting accounts support unlimited hosted websites, so if you want to test whether your site will work with a newer version, you can always spin up a staging site on a sub-domain and switch the PHP version for just that site.

Supported versions

We currently support PHP 7.3, 7.4 and 8.1 on our hosting servers, and are considering adding support for 8.0. If you have a requirement for a specific version, please drop us an email.

deb.sury.org

The thing that makes this possible is the excellent work of Ondřej Surý, long-term maintainer of Debian’s PHP packages. In addition to providing the official Debian packages, Ondřej also provides deb.sury.org, a private repository providing Debian packages for multiple versions of PHPs, built and maintained to the same standards as the official Debian packages.

We’ve just rolled out a new addition to our range of APIs for managing services: the Domain Management API. This new API allows you automate management of your Mythic Beasts domain registrations.

Access to the API is controlled by API keys, which can be managed in our customer control panel. As for our DNS API, the keys provide fine-grained control over access, allow you to grant permissions on individual domains, or all domains on your account, and to restrict a key’s access to specific actions.

Fine-grained access control

The API gives access to information about your domains, such as the expiry date, nameservers, and domain status.

At present, the API only supports a small number of actions, although we intend to expand this in the near future. At present, the following actions are supported:

• Setting nameservers
• Setting DS records
• Locking/unlocking domains (where supported)

The ability to set DS records makes it possible to automate DNSSEC key roll-over, although it’s worth noting that we offer a free managed DNSSEC service which takes care of this for you, so you’ll only need to use this if you particularly want to control your DS records yourself.

The API is currently in public beta, and documentation can be found on our support site. We’d very much welcome feedback on the API, including suggestions for operations that you’d like to see supported. If you have any feedback, please contact us on support@mythic-beasts.com.

Dormant mode means your VPS can have a nice snooze.

We’ve recently rolled out some new features that provide more flexibility to our VPS platform.

On-demand billing

Last year we added on-demand billing to our Raspberry Pi Cloud and we’ve now rolled this out to our VPS services, allowing you to add and remove VPSs at any time and pay by the second for the time that the server is provisioned. We continue to offer monthly, quarterly and annual billing options, with discounts for longer billing periods, allowing users to choose between the best pricing for long term usage and the convenience of on-demand, pay-as-you go pricing.

Dormant VPS mode

We’ve also added the ability to make an on-demand VPS dormant, so that you’re only charged for the server’s storage space (and any allocated IPv4 addresses) until you want to reactivate it. Dormant VPSs can be reactivated at any time, although it is not guaranteed that you will be able to re-provision to the same specification of server immediately. The RAM and CPU previously allocated to your server may have been reallocated, and a move to a different host server may be required.

VPS management API

We have also added an API for managing on-demand VPSs, allowing the creation and deletion of servers to be automated. The API is very similar to our API for managing Raspberry Pi Cloud servers. To get started, see our API docs.

Cloud-init user data

We use cloud-init to automate operating system installation when provisioning a new VPS. The installation can be customised using cloud-init user data, which can provide additional installation steps to be performed after the first boot. User data can be provided through both the control panel and the API. It also possible to store and re-use user data snippets in the control panel, making it easy to repeatably spin up new servers with your applications already installed and configured.

More capacity

We continue to add capacity to our cloud to keep up with customer demand with the most recent expansion being in our London Meridian Gate (MER) zone.

Private cloud improvements

Our Private Cloud service gets you the features and convenience of our public VPS platform, but provided on your own dedicated servers. We’ve recently rolled out improvements to our Private Cloud platform, allowing Private Cloud servers to be provisioned and managed via the API and control panel.

Earlier this month we reported that we’d signed up to the Public Benefit campaign to reform Nominet, the company responsible for overseeing UK domain registrations.

The campaign was seeking 5% of Nominet’s membership in order to call an EGM to replace Nominet’s non-elected directors. The campaign quickly achieved this, the EGM request was delivered, and Nominet have now set the date for the EGM as 22nd March 2021. Members representing more than 17% of Nominet voting rights have now signed up to support the campaign. Typical AGM voting turnout is well under 10% suggesting that the vote is pretty much certain to succeed, at least according to The Register’s analysis.

If there was ever any doubt about the need for reform, Nominet’s response to the EGM letter has completely removed this.

Nominet’s CEO rushed out a statement hoping that:

all constituencies will be able to engage in a constructive way

At the same time, Nominet responded to Public Benefit’s email requesting member information by providing 575 printed pages:

This would seem to be more obstructive than constructive.

The EGM request made two motions: (1) sack the current directors; and (2) appoint two interim directors to take over. Nominet are claiming that the second motion is illegal (contrary to legal advice received by Public Benefit) and are refusing to put it on the EGM agenda. They now have the gall to claim that the EGM request destabilises Nominet because it does not provide a credible plan to replace the current leadership.

Is this just about reducing UK domain fees?

It’s been suggested that this campaign is about Nominet members, who are mostly companies like us that resell domain registrations, trying to reduce the price that they pay for domains. This seems to ignore the fact that the domain market is very competitive, and UK domains are particularly easy to transfer between registrars. Provided that the price is the same for all members, what that price is doesn’t make much difference to us.

Nonetheless, we’re very happy to make a public commitment that if the EGM process results in a reduction in the price that we pay for domains, we will pass on that saving in the price that we charge.

The MagPi Magazine has published a new article on how to set up a web server using a Raspberry Pi hosted in our Pi Cloud.

The article walks through all the steps necessary from ordering a server on our website to getting WordPress installed and running.

It’s also a great demonstration of how easy it is to host a website on an IPv6-only server such as those in our Pi Cloud. In fact, it’s so easy that the article doesn’t even mention that the Pi doesn’t have a public IPv4 address. An SSH port-forward on our gateway server provides IPv4 access for remote administration, and our v4 to v6 proxy relays incoming HTTP requests from those still using a legacy internet connection.

You can read the article on the MagPi site or order a server to try it out yourself.

We have Pi 3 and Pi 4 servers available now, and the option of per-second billing means you can try this without any ongoing commitment.

A customer was doing an IP address change on a server and wanted a quick way to find all references to the old IP address across all of their domains.

This seemed like a good job for our DNS API and a few UNIX utilities.

Finding matching records

Our DNS API makes it easy to find records with particular content:

curl -sn https://api.mythic-beasts.com/dns/v2/zones/example1.com/records?data=1.2.3.4

The -n assumes we’ve got a .netrc file with our API credentials. See our DNS API tutorial for more details.

This gives us a block of JSON with any matching records:

{
  "records": [
    {
      "data": "1.2.3.4",
      "host": "www",
      "ttl": 300,
      "type": "A"
    }
  ]
}

jq lets us turn the presence or absence of any matching records into an exit code that we can test with an if statement by piping into the following:

jq -e '.records | length > 0' 

This counts the number of members of the records array, and -e sets the exit code based on the output of the last expression.

Getting a list of zones

We want to check this across all zones, so let’s get a list of zones:

curl -sn https://api.mythic-beasts.com/dns/v2/zones

This gives us some JSON:

{
  "zones": [
    "example1.com",
    "example2.com"
  ]
}

What we really want is a flat list, so we can iterate over it in bash. jq to the rescue again. Simply pipe into:

jq -r '.zones[]'

and we get:

example1.com
example2.com

Putting it all together

Putting this all together with a for loop and an if:

IP=1.2.3.4
for zone in $(curl -sn https://api.mythic-beasts.com/dns/v2/zones | jq -r '.zones[]') ; do
  if curl -sn "https://api.mythic-beasts.com/dns/v2/zones/$zone/records?data=$IP" |\
      jq -e '.records | length > 0' >/dev/null ; then 
    echo "$IP found in $zone"
  fi
done

Gives:

1.2.3.4 found in example1.com

More than one way to do it

Another approach would be to use the zone file output format and check if the output is empty or not:

curl -sn -H 'Accept: text/dns' \
  "https://api.mythic-beasts.com/dns/v2/zones/$zone/records?data=$IP"

This give us matching records, one per line:

www         300 A 1.2.3.4

We can then test if we’ve got any matches using ifne (if-not-empty, part of the moreutils package in most distributions):

curl -sn -H 'Accept: text/dns' \
  "https://api.mythic-beasts.com/dns/v2/zones/$zone/records?data=$IP" \
  | ifne echo $IP found in $zone

Access to our DNS API is included with all domains registered with us. API credentials can be limited to individual zones or even records, can be either read/write or read-only.

ANAME records

Of course, it’s generally desirable to avoid including an IP address in lots of different DNS records in the first place. It’s preferable to assign the IP to a single hostname, and then point other records at that. Our DNS service supports ANAME records which allow the use of hostnames rather than IP addresses in places where CNAMEs cannot be used.

We recently announced our new DNS API which we’ve just moved out of beta and into production. 

One of the goals of the new API was better support for automating DNS-based challenges, such as those used by Let’s Encrypt to authenticate certificate requests. 

DNS-based challenges are needed to obtain wildcard certificates from Let’s Encrypt, and can be a convenient way to get certificates for hostnames that don’t a have publicly accessible web server, but can be tricky to implement due to delays in updating DNS records, and automatic requires having credentials capable of DNS records for your domain stored on your server.

The new API has a number of features to address these issues.

Restricted credentials

The DNS API allows you to create API credentials that are restricted to editing specific records within your domain.  Credentials can be restricted by hostname, record type, or both.

For example, you can create credentials that can only edit the _acme-challenge TXT record needed for Let’s Encrypt challenges. Access to the DNS API is potentially very sensitive, so it makes sense to limit access as much as possible.

Record verification

Updates made via the API do not become live immediately. There is a delay of up to a minute before they hit our master nameserver, and a potential further delay of a few seconds before the record propagates to our authoritative nameservers. When responding to a DNS-based challenge, you will typically want to ensure that the record is actually live before proceeding with verification.

Our DNS API provides a “verify” feature, that checks that records are live on all authoritative nameservers. For example, a GET request to the following URL would check that the nameservers have the latest update to the record:

https://api.mythic-beasts.com/dns/v2/zones/example.com/records/_acme-challenge/TXT?verify

This will return a 200 response if the nameservers are up-to-date, and 409 if they are not. This can be used to script a check after updating a record:

#!/bin/sh

ZONE=example.com
RECORD=_acme-challenge
TYPE=TXT

for i in $(seq 1 12); do
    RES=$(curl -n https://api.mythic-beasts.com/dns/v2/zones/$ZONE/records/$RECORD/$TYPE?verify -qs -w '%{http_code}' -o /dev/null)
    case $RES in
        200)    echo Records updated
                exit 0
                ;;
        409)    echo "Not yet updated ($i/12)"
                ;;
        *)      echo "Unexpected error: $RES"
                exit 1
                ;;
    esac
    sleep 10
done
echo Timed out
exit 2

Obtaining certificates the easy way

Our preferred Let’s Encrypt client is the excellent dehydrated, and we maintain a hook script for supporting DNS-based challenges in dehydrated. We haven’t yet updated the hook script to support our new API, but will be doing so soon and will post details here when it’s ready.

We’ve just launched our new DNS API, which provides a much cleaner and more powerful API for automatic management of DNS records.

Key features of the new API include:

• Configurable auth credentials – restrict access to individual records, or record types.  Ideal for Let’s Encrypt or other DNS-based challenges.
• Choice of JSON or zone file format for input and output.
• Atomic multi-record updates – update arbitrary sets of records in a single transaction.
• Form parameters for record creation – record creation can be trivially scripted using curl.
• Broad record type support – CAA, SSHFP, TLSA, SRV and many more.

For a walk through of the features of the new API, please see our DNS API tutorial , or for more details see the reference documentation.

To get started with the API, use the API keys section of the control panel to create some credentials.

Restricted API credentials for Let’s Encrypt challenges

The new API is currently in public beta, meaning that we reserve the right to make last minute breaking changes to the API, although we expect any such changes to be minor, and we would very much like to hear any feedback you may have.

Our forthcoming new DNS API seeks to make automating common DNS management tasks as simple as possible. It supports both JSON and “zone file format” for inputs and outputs, which opens up some interesting options for making bulk updates to a DNS zone.

Bulk TTL change

Suppose we want to drop the TTL on all of our zone’s MX records. We could get the records in zone file format and edit them using sed:

curl -n -H 'Accept: text/dns' https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX \ 
| sed -E 's/^([^ ]+) +([0-9]+)/\1 60/' \
| curl -n -H 'Content-Type: text/dns' -X PUT --data-binary @- https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX

Alternatively, we could get the zone in JSON format and modify it using the awesome command line JSON-wrangler, jq:

curl -n -H 'Accept: application/json' https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX \
| jq '.records[].ttl = 60' \
| curl -n -H 'Content-Type: application/json' -X PUT --data-binary @- https://api.mythic-beasts.com/dns/zones/example.com/records/@/MX

How does this work?

The most common response to seeing the second example is “cool, I didn’t know you could do that with jq!”. So what’s going on here?

The first line gets us the the MX records for example.com (the “@” means fetch records for the bare domain). This gives us some JSON that looks like this:

{
    "records": [
        {
            "data": "mx1.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 300,
            "type": "MX"
        },
        {
            "data": "mx2.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 300,
            "type": "MX"
        }
    ]
}

The next line modifies this JSON. jq allows you to select values within the JSON. .records selects the “records” property of the response. [] gets us all of the members of the array, and .ttl selects the “ttl” property of each one. On its own, this would extract the values from the JSON, and give us “300 300”, but jq also allows us to modify the property by assigning to it. This gives us the following JSON:

{
    "records": [
        {
            "data": "mx1.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 60,
            "type": "MX"
        },
        {
            "data": "mx2.mythic-beasts.com.",
            "host": "@",
            "mx_priority": 10,
            "ttl": 60,
            "type": "MX"
        }
    ]
}

We can now pipe this into a PUT request back to the same URL. This will replace all of the records that the first request selected with our new records.

The first approach works in a very similar way, except rather than working on a block of JSON with jq, we use sed to modify the following zone file formatted text:

@                      300 MX    10 mx1.mythic-beasts.com.
@                      300 MX    10 mx2.mythic-beasts.com.

Choose your weapon

We think that the JSON based approach is neater, but let us know what you think on our Twitter poll. We’d also be interested to hear of specific DNS management operations that you’d like to automate, so that we can see how they’d be tackled in our new API.

We’ve got a new version of our DNS API under development.  One of the neat new features is the ability to accept input in bind zone file format (aka RFC 1035). 

One of the things that this makes very easy is adding SSHFP records to the DNS. SSHFP is a mechanism for lodging your server’s public SSH keys in the DNS so that your SSH client can automatically verify a server the first time you connect to it, rather than prompting you to confirm the host key.

Using our new API, you can add or update SSHFP keys by piping the output of ssh-keygen straight into curl:

ssh-keygen -r myhost | curl -X PUT -n https://api.mythic-beasts.com/zones/example.com/myhost/SSHFP -H 'Content-Type: text/dns' --data-binary @-

What’s going on here?

ssh-keygen -r outputs your server’s SSH public keys in RFC 1035 format:

$ ssh-keygen -r myhost
myhost IN SSHFP 1 1 e579ff6aabc2f0acf714deca53108a0c1ea7d799
myhost IN SSHFP 1 2 7c47d5dfb748ff1fd244b7289d815e83dad8c2c1652b92ac8aed8ff166733d07
myhost IN SSHFP 2 1 c5caf4cc8870acc7fd113e5a7c866822ec0d94de
myhost IN SSHFP 2 2 9f11843fa1d9da318aa4bc09bbcaacaf4a9868c4d83dfc4bad6853d0c9597a31
myhost IN SSHFP 3 1 eb8644f5fcfd555341f2063bd92044075e20da89
myhost IN SSHFP 3 2 60f3e9780f9b87e5b4d6344f2ab46decbf705123e96ef07c3247f714ca220fc4
myhost IN SSHFP 4 1 139426de48381ea46ad75dde4e412bf1c9b11e61
myhost IN SSHFP 4 2 6f094181b510bbb573048835665773eb1a2a65fd4341d95207479ed71296491b

We then pipe that into curl to make a request to the DNS API.

A PUT request to the /myhost/SSHFP endpoint replaces all existing myhost SSHFP records.

-n tells curl to get auth credentials from a .netrc file, and the “Content-Type” header tells our API that we’re providing the new records in zone file format.

What’s the point of SSHFP?

Having lodged these SSHFP records in the DNS, and provided that DNSSEC is enabled for your domain, it’s possible to connect to a server without being prompted to verify the server’s host key.

$ ssh -o VerifyHostKeyDNS=yes root@myhost.example.com 

You can avoid having to specify the -o by putting VerifyHostKeyDNS=yes in your ~/.ssh/config file.

Feedback wanted

The new DNS API isn’t quite ready for deployment, so if there are any features that you’d really like to see in the new API, now’s the time to tell us, either on Twitter or by email.