Some months ago we announced that we were planning an even cheaper version of our entry-level VPS Lite virtual servers.

It took a little longer than planned, but we’re now pleased to announce the launch of our IPv6-only VPS Lite, for the ludicrously low price of £5+VAT/month (or even less if you pay annually). This virtual server comes with 4 billion IPv6 addresses, but no IPv4 connectivity.

The world is running out of IPv4 addresses, and whilst we’ve got an allocation that isn’t going to run out any time soon, issuing an IPv4 address with every server we sell will ultimately be a barrier to our growth. If you’re happy to have a server without an IPv4 address, we’re happy to give you a discount, and that’s exactly what we’ve done with our IPv6-only VPS Lite.

IPv6 usage is currently running at about 3% and doubling roughly annually, so now is a great time to get familiar with IPv6.

Whilst these servers probably aren’t suitable for fronting a website just yet, we’ve already got a number of customers who run IPv6-only backend networks, with everything behind their load balancers on single-stack IPv6.

Although these Virtual Servers themselves don’t have IPv4 connectivity, the host server does, which means that you can get to the admin console, virtual serial console and VNC over an IPv4 connection.

If you’re a technology professional and have no idea what any of this means, we suggest you start your training by placing an order here.

As you may be aware, ICANN, the body responsible for domain names has introduced a lot more “Top Level Domains” (TLDs). This means that you’ve now got a lot more choice than the traditional .com / .net / .org / .co.uk, etc.

You can now have names under .bike and .gallery, and very soon you’ll be able to have names under .kitchen and .land. Many more new TLDs have been approved, and will start appearing online in the coming months. (Given the quantities we drink, we’ve been wondering whether we should rebrand as mythic-beasts.coffee.)

Below is a list of the new TLDs being released in February. You can find prices for these domains on our domains page.

Congratulations to http://mikaellelebreton.photography/, our first customer (that we know of) to get a website up and running on a domain released this week.

Update 2014-02-26. Four new domains will open up today: .diamonds, .enterprises, .tips, and .voyage.

We’ve just rolled out some improvements to our monitoring service. All server products, including virtual servers, get access to our basic ping monitoring service, allowing you to receive SMS and email alerts if your server goes off-line. For £5/month you can add enhanced monitoring, which allows you to confirm that individual services such as a web server are working correctly. Customers of our Managed Hosting service not only get access to enhanced monitoring, but also get the benefit of having our staff respond to the alerts for them.

The new features are:

Temporary silencing of alerts

You can now silence alerts for a set period. This is handy if you’re doing maintenance, and don’t want to be constantly pestered by alerts, but also don’t want the risk of forgetting to turn the monitor back on again afterwards.

Prowl notifications

Prowl is a notification system for iOS, allowing you to receive alerts on an iPhone or iPad. The advantage of Prowl notifications over SMS is that they’re not limited to 141 characters, so we can include a more verbose message, including direct links for silencing the alert. They’re also delivered over the internet, rather than the mobile network, so will work if you have a wifi connection, but no mobile signal.

Support for an Android equivalent (such as Notify My Android) is on the to-do list.

Improved email alerts

The email alerts previously included the same dense text that we use for SMS alerts. The new style notifications are now more verbose, and include links for silencing the monitor, and in the case of web alerts, a link to the page that failed.

Monitoring of arbitrary TCP ports

We provide monitors for most common services, including HTTP, SMTP, IMAP and POP3. You can now also monitor any TCP port. This check simply confirms that the host is accepting connections on this port, and then closes the connection.

Edit: we’ve now received a report from Telecity, so have updated this report to take account of this.

Further edit: explanation for extended outage in one rack added.

Summary

• A power interruption occurred at around 8:09am on Saturday 25th January, affecting multiple floors in Sovereign House.
• For the most part, the interruption was momentary (around 500ms), but long enough to cause a reboot of affected equipment.
• One of our racks was without power until 10:38am, due to a tripped circuit breaker.
• Our staff were onsite at 11:15am, and then worked to restore services that had not come back up cleanly. One such server was our SOV DHCP server which will have affected any virtual servers configured to boot via DHCP.

Details

The power outage was caused by an interruption to the external mains power supply, followed by a failure of the DRUPS (Diesel Rotary Uninterruptible Power Supply) system that is supposed to ensure that power to the data centre is maintained during such a power cut.

The DRUPS system contains three separate units with sufficient capacity to cope with the failure of any one unit. Unfortunately, in this event, the unit that failed did so in a manner that triggered a shutdown of the other two. From the Telecity report:

… one of the units on DRUPS System 1 experienced a fault on its synchronisation card. This fault caused the unit to go into overload which, in turn, had a direct impact on the remaining two units. During the overload condition, the faulty unit back-fed the other two units which, for protection and per design, automatically shut down.

At this point the system went into raw mains bypass mode (i.e. bypassing the UPS systems, and connecting the data centre load directly to the mains). This occurred around 2 minutes after the original mains supply failure, by which point the mains supply had been restored, but there was a 500ms interruption as the bypass occurred.

This much is consistent with our observations, which is that in all but one rack, the logs on our remote PDUs did not record an outage, but the vast majority of equipment attached to them did: the management interfaces in these PDUs draw very little electricity and are known to be able to survive very short power supply interruptions.

As noted above, one of our racks experienced a more extended outage. This was due to the circuit breaker on the power bar being tripped. This was noticed and rectified by data centre staff inspecting racks following the initial outage.

At this point, the faulty DRUPS unit is out of service, meaning that whilst the power supply is protected, there is no redundancy until the unit is repaired and tested.

Conclusion

Whilst we are certainly unhappy about the outage, at this point we have no cause to question our choice of data centre provider. Sovereign House is a major UK internet hub, and is a purpose-built 6 floor data centre, built to the highest industry standards. With the best will in the world, there will always be faults that can take an entire DC, or significant parts of it, off-line, and for this reason, we would always recommend that mission-critical applications are served from multiple sites. Independent routing ensured that our facilities at other sites were unaffected by the Sovereign House outage.

That said, the aftermath of the outage has revealed some areas in which we can improve. In particular, the extended outage of one rack had a knock on effect to connectivity of others. Following Sunday’s scheduled maintenance work, we’re now in a position to improve our network topology to make it more resilient. We are also planning improvements to our Virtual Server hosts and database servers to ensure that they can recover more quickly following such an outage, and we have already made changes to our support systems to make them more resilient.

Beyond directly fixing the affected units, Telecity are also planning improvements to their communications during such an incident. This will help us direct our efforts more effectively.

Notes

For the avoidance of doubt, this interruption was completely unrelated to the network upgrades scheduled for Sunday evening, which went ahead as planned.

Finally, thank you to all customers who monitored our status page during the outage.

You can now configure reverse DNS for IPv6 through our customer control panel. If you’ve previously been handling reverse DNS for your allocation through delegation and would prefer to use the control panel, then please get in touch.

If you’ve got a server with us and are interested in trying IPv6 and don’t already have an allocation then please email support and we’ll be happy to provide you with a block of addresses.

Last night we quietly upgraded the disks in our Sphinx shell server to a pair of SSD drives. Sphinx has been suffering under heavy I/O load for a while now, and it’s safe to say that the SSDs have resolved that problem for the foreseeable future.

The upgrade was without downtime, using the magic of LVM’s pvmove command.

It’s been upgraded with a pair of fiendishly expensive server-grade SSDs. We’re not normally ones to pay too much attention to whether kit is designated as “server-grade” but in the case of SSDs it really matters due to the limited number of write cycles on SSDs. The new disks are good for 8TB of writes per day for 5 years, whereas the equivalent consumer grade version is only rated for 20GB/day, which wouldn’t last very long in Sphinx.

Sphinx has a special place in our hearts as it’s the machine on which the company was founded nearly 14 years ago, and it’s been in pretty much continuous service ever since. Of course, the current hardware has absolutely nothing in common with the dual Celeron BP6 that we deposited in a Fulham datacentre back in 2000, and it now lives in Docklands, but it’s still the same machine (right?) which is why it still says:

[pdw@sphinx ~]$ rpm -q redhat-release
redhat-release-6.1-1

(don’t worry, that’s probably the only package from RH 6.1 that we’re still using…)

We’ve just extended our range of Virtual Servers to include a 512MB “VS Lite” option for just £7 + VAT per month – or the equivalent of £5.83/month if you pay annually.

We’ve been doing Virtual Servers for 10 years now, and we’ve always hosted them on machines with hot-swappable hardware RAID, meaning that we can replace failed disks without customers even being aware that there was a problem. One of the great things about virtualisation is that reliability shares really well – but it still adds a bit to the cost

A large proporption of our Dedicated Server customers use machines without hot-swappable drives, and we thought we’d extend the option of using cheaper hardware to our Virtual Server range in the form of the VS Lite, our most affordable Virtual Server yet.

The VS Lite host hardware still has monitored RAID, so we can cope with disk failures without data loss^*, but it requires downtime in order to replace the drive.

Our standard Virtual Servers start at £12.50 + VAT per month for a 1GB server, and all run RAID-10 on hot-swappable drives, and offer a choice of data centres, allowing you to spread multiple servers across geographically diverse sites.

In the unlikely event that even £7/month sounds expensive, then we’ve got an even more affordable alternative with a bit of a twist up our sleeves which we’ll be unveiling shortly. The first person to guess what we’re planning can have one free for a year, and don’t worry, as with all our Virtual Servers you can upgrade and downgrade easily, so don’t be put off by the prospect of an even better offer in the pipeline!

On the other hand, there are no prizes for guessing what hardware we’re using for the VS Lite host servers…

* RAID is not backup.

Last year we announced some improvements to the Mythic Beasts DNS API, and I asserted that this made it good for implementing a Dynamic DNS service. Dynamic DNS is simply a mechanism for programmatically updating a DNS record, typically used to provide a consistent name for a computer that is at the end of an internet connection with a dynamically assigned IP address.

Well, last weekend I had the opportunity to try implementing a Dynamic DNS service with our API, and realised that it actually makes the task unduly difficult. It can be done, but in order to change a record, you need to remove the old record, and to remove the old record you need to know what it is currently. This meant that you had to use the LIST command, grep out the old record, and then issue the necessary DELETE and ADD commands. Aside from being hassle, it introduces an unavoidable race condiition between the LIST and DELETE commands.

We’ve now implemented the obvious fix: a REPLACE command, which replaces all existing records for the specified host and type, and replaces them with the one provided. Obviously this doesn’t work if for some reason you want multiple records for a single host, but for the obvious use case it means that Dynamic DNS can be handled in a single command:

curl --data "domain=MY_DOMAIN&password=MY_PASSWORD&command=REPLACE \
myhost 300 A 1.2.3.4" https://ctrlpanel.mythic-beasts.com/customer/primarydnsapi

The DNS API is a standard feature included with all Mythic Beasts domain registrations. Full documentation can be be found here.

Today we have made a number of changes Onza and Yali.

• Each user can now read their own web logs, which are stored in /home/user/www/log
• PHP scripts no longer need the execute bit set to run
• All scripts in cgi-bin directories have been moved to /home/user/cgi-bin to conform with Apache’s security recommendations

We hope that these changes will make it easier for people to run and debug their code.

A couple of weeks ago, Apple unveiled the latest incarnation of the Mac mini.  Naturally, we dashed out to buy a few to see if they’re going to be any good as servers.  Externally, this is the biggest revision of the Mac Mini yet, with a thinner all-aluminium case.  We always get a bit nervous when Apple unveil a new Mac mini as there’s a chance that they’ll ruin the formula that make it such a great server in the name of creating a fun toy for your living room.

The most noticeable change, aside from the new case, is the removal of the power brick.  The old minis relied on an external power supply that really was the size and shape of a brick.  Getting rid of these will make racking them a lot simpler, as well as saving space.  We reckon we should be able to get 24 machines and 3 APC Masterswitches into 6U of rackspace.  The C7 power connectors should be more secure too.

The next thing that we approve of is the easily accessible RAM slots.  Traditionally, Apple have charged silly money for ordering machines with extra RAM, so we’ve always done the upgrades ourselves.  Upgrading the old minis was a real pain, requiring the cover to be prised off with putty knives, so this is a welcome change.

The most important factor for us, and the thing that lead us to use the original PPC minis, is power consumption, as power is the primary cost when hosting a machine.  There’s good news here too: according to our power meter, power consumption is down from 26W to 12W (idle) and 44W to 40W (max).

Unfortunately, the reduction in hosting cost is offset somewhat by they usual price hike for the newer hardware, but we’re pleased to see that Apple have retained the “server” version of the mini with two hard drives, allowing us to continue to offer machines running software RAID.

Overall, the new Macs seem like a decent improvement on the old ones for our purposes.  Apple make a fuss over the great variety of ports now available on the back of the new mini, but inexplicably they haven’t provided the one thing we’d want: a good old-fashioned serial port.

We still need to make the minis work with our custom net-booting bootloader, but once that is done, we’ll be offering them as dedicated servers.