WordPress is an excellent content management system that is behind around 25% of all sites on the internet. Our busiest site is Raspberry Pi which is now constructed from multiple different WordPress installations and some custom web applications, stitched together in to one nearly seamless high traffic website.

We’ve taken the knowledge we’ve gained supporting this site and rolled it out as a managed service, allowing you to concentrate on your content, whilst we take care of keep the site up and secure. In addition to 24/7 monitoring, plugin security scans, and our custom security hardening, we’re also able to assist with improving site performance.

We’re now hosting a broad range of sites on this service with the simpler cases start with customers such as Ellexus, who make very impressive technology for IO profiling, and need a reliable, managed platform that they can easily update.

At the other end of the spectrum we have the likes of Parallax Photographic, a co-operative in Brixton who sell photography supplies for people interested in film photography, using real chemistry to develop the photographs and a full analogue feel to the resulting prints. Parallax Photographic use WordPress to host to their online shop, embedding WooCommerce into WordPress to create their fully functional e-commerce site.

Parallax were having performance and management issues with their existing self-managed installation of WordPress. We transferred it for them to our managed WordPress service, in the process adding not only faster hardware but performance improvements to their WordPress stack, custom security hardening, managed backups and 24/7 monitoring. We took one hour for the final switch-over at 9am on a Sunday morning leaving them with a faster and more manageable site. They now have more time to spend fulfilling orders and taking beautiful photographs.

Purrmetrix monitors temperature accurately and inexpensively, and as you can see above with excellent embeddable web analytics. In addition to hosting their website and WooCommerce site for people to place orders, we are also customers (directly, through their website!) using their site to monitor our Raspberry Pi hosting platform. The heatmap (above) is a real-time export from their system. At the time of writing, it shows a 5C temperature difference between the cold and hot aisles across one of our shelves of 108 Pi 3s. The service provides automated alerts; if that graph goes red indicating an over temperature situation alerts start firing. During the prototyping and beta phase for our Raspberry Pi hosting platform, we’ve used their graphing to demonstrate that it takes about six hours from dual fan failure to critical temperature issues. This is long enough to make maintenance straightforward.

Also embedded in our Raspberry Pi hosting platform are multiple Power over Ethernet modules from Pi Supply who make a variety of add-ons for the Raspberry Pi, including some decent high quality audio adapters. With the launch of the Raspberry Pi 3 we had to do some rapid vertical scaling of the Pi Supply managed WooCommerce platform – in thirty seconds we had four times the RAM and double the CPU cores to cope with the additional customer load.

 

We host a wide variety of WordPress sites include Scottish comedy club Mirth of Forth, personalised embroidery for work and leisure wear and our own blog that you’re currently reading. So if you’d like to have us run your WordPress site for you, from a simple blog to a fully managed e-commerce solution or one of the busiest sites on the Web, we’d love to hear from you at sales@mythic-beasts.com.

One of our customers does embedded development and have some custom servers in their office as a build platform. This is hardware specific to the embedded designs they’re working on and they can’t locate it in a data centre as they require regular human attention. Unstable development drivers cause crashes and the root flash filesystems need to be re-imaged and replaced.

Recently they’ve moved office and their new office has a ‘server room’, ideal for putting their very expensive custom kit in, and a handful of other machines that they keep locally in the office. While doing the fit out, they noticed that their ‘server room’ is attached to the main sprinkler system. A fire in the building and whilst the bread may be saved from being overly toasted, their expensive hand built development boards are drowned.

They raised this with their landlords who billed them the best part of a thousand pounds to resolve the problem, see the picture on the right.

I’m not sure if it’s the belief that the plastic roof will help, the combustible struts to hold it up or the lack of guttering that really emphasises the mismatch between what a landlord things a server room looks like and what a real data centre actually provides.

We’re in further discussions to see if we can host their custom kit too, because our server room has non computer damaging halon as a fire suppressant and we will return the servers to them unwashed. If your office server room looks like this, please get in touch at sales@mythic-beasts.com.

If you’re anything like me, December involves a tour of parents who have retired to far flung corners of the land and who are now living in houses with perfectly serviceable wifi but absolutely no mobile phone coverage. This creates a problem if you’re supposed to be listening out for computers that go bleep in the night, as the SMS notifications don’t get through.

To address this, we’ve just implemented Notify My Android support in our monitoring service. As the name suggests, this allows us to push monitoring alerts to your Android phone. It’s pretty easy to set up:

1. Register for an account with Notify My Android (a free account will allow 5 notifications per day, a one-off payment of $5 will get you unlimited notifications).
2. Log in, and generate an API key
3. Download the app on your phone, and log in
4. Visit our control panel and add an entry to your notification list that looks like nma:API_KEY where API_KEY is the key generated above.

If you use the other kind of phone we have equivalent functionality using Prowl. This works the same, except you put prowl: before your API key.

All of our dedicated and virtual servers include basic ping monitoring as standard. Comprehensive monitoring of other services is available as an add-on, or as standard with any of our managed services.

We’ve just completed an upgrade to our backup services. We’ve relocated the London node into Meridian Gate, which means for all London hosted virtual machines your primary backup is now in a different building to your server. We’ve kept our secondary backup service in our Cambridge data centre 60 miles distant.

To further improve, we have taken the opportunity to enable disk-encryption, so that all data stored on the primary backup server is now encrypted at rest providing an additional layer of assurance for our clients and fewer questions to answer on security questionnaires.
We’ve also restricted the number of ssh ciphers allowed to access the backup server to further improve the security of data in transit. We’ve also increased the available space and provided a performance boost in the IO layer so backups and restores will complete more quickly.

Of course we’ve kept some important features from the old backup service such as scanning our managed customers’ backups to make sure they’re up to date and making sure that we alert customers before their backups start failing due to lack of space. Obviously all traffic to and from the backup server is free and it supports both IPv6 and IPv4.

If these are the sort of boring tasks on your todo list and you’d like us to do them for you, please get in touch at sales@mythic-beasts.com.

We’ve just added support to our control panel and DNS API for “ANAME” records. ANAME records, also known as ALIAS records, aren’t real DNS records, but are a handy way of simulating CNAME records in places where you can’t use a real CNAME.

It works like this:

You’ve got DNS for your domain managed with Mythic Beasts, and you want to host your website with some 3rd party service provider. They’ll tell you to point DNS for your website at their server. You create a CNAME record for www.yourdomain.com and point it at server.3rdparty.com. So far so good.

You also want requests for your bare domain, e.g. http://yourdomain.com to be served by your provider, so you try to create a CNAME for yourdomain.com and get told you can’t. This is because you will already have MX, NS and SOA records for your bare domain, and CNAMEs aren’t allowed to co-exist with other records for the same name.

The usual fall back is to create A or AAAA records that point directly to the IP address of server.3rdparty.com, but this sucks because their IP is now hard coded into your zone, and if they ever want to change the IP of that server they’ve got to try and get all of their customers to update their DNS.

The nice solution would be SRV records, standardised DNS records that allow you to point different protocols at different servers. Unfortunately, they’re not supported for HTTP or HTTPS.

This is where ANAME records come in. You can create an ANAME just like a CNAME, but without the restrictions on co-existing with other records. We resolve the ANAME and substitute the corresponding IP addresses into records into your zone. We then regularly check for any changes, and update your changes accordingly.

Naturally, our ANAME implementation fully supports IPv6: if the hostname you point the ANAME at returns AAAA records, we’ll include those in addition to any A records returned.

We don’t like to pre-announce things that aren’t ready for public consumption. It’s no secret that we’d love to offer hosted Raspberry Pis in the data centre, and in our view the blocker for this being possible is the unreliability of SD cards which require physical attention when they fail. So we’ve provided some assistance to Gordon to help with getting netboot working for the Raspberry Pi. We built a sensible looking netboot setup and spent a fair amount of time debugging and reading packets to try and help work out why the netboot was occasionally stopping.

This isn’t yet a production service and you can’t buy a hosted Raspberry Pi server. Yet. But if you’d be interested, we’d love to hear from you at sales@mythic-beasts.com.

This is a standard Raspberry Pi 3 with a Power over Ethernet (PoE) adapter. You have to boot the Pi once from a magic SD card which enables netboot. Then you remove the SD card and plug it in to the powered network port. PoE means we can power cycle it using the managed switch. At boot, it talks to a standard tftpd server and isc-dhcp-server, this then delivers the kernel which runs from an NFS root. It’s a minimal Raspbian Jessie from debootstrap plus sshd and occupies a mere 381M versus the 1.3G for a standard Raspbian install. The switch is reporting the Pi 3 consuming 2W.

The Raspberry Pi topple is just for fun.

We’ve upgraded our VPS Lite service to bring it onto the same hardware platform as our standard VPS offerings. Our base level virtual server is now the VPS 1 and has 1GB RAM and 20GB disk. It’s fractionally more expensive than the old VPS Lite, but offers double the disk and RAM. We’ve also introduced an SSD option, in addition to traditional spinning disks, for faster IO performance.

Customers of the existing VPS Lite service will be migrated onto the new hardware platform, and given the option of keeping their current spec on faster hardware, or upgrading to the new VPS 1 spec.

Naturally, the new VPS 1 comes with IPv6 as standard: IPv4 is an option, although our IPv4 to IPv6 reverse proxy, and the availability of NAT64 for outbound traffic means that most websites can be hosted easily on a IPv6-only server.

Full details of the current specifications can be found on the Virtual Servers page.

We recently announced support for PROXY protocol in our IPv4 to IPv6 reverse proxy, and happily linked to the instructions for making it work with NGINX. One of our customers has pointed out that they didn’t actually work, and we’ve now got to the bottom of why not.

NGINX version

First issue: you need NGINX >= 1.9.10, as there was a bug with using proxy_protocol on IPv6 listeners. If you’re on Debian Jessie, you can get a suitable version from Jessie backports.

PROXY protocol version

Second issue: NGINX only speaks PROXY protocol v1 and our proxy was attempting to speak v2.

v1 is a human readable plain text protocol, whereas v2 is binary. If you see something like this in the error log:

2016/05/09 11:11:30 [error] 6058#6058: *1 broken header: "

QUIT
!
 ]Y??.????PGET / HTTP/1.1

Then that’s a good sign that you’ve got a v2 reverse proxy talking to you.

We’ve now changed our proxy to only speak PROXY protocol v1 by default. We will look into making this a configurable option in the future. The Apache module seems happy speaking either version.

Whilst we’re here, here are some other failure modes you might see. This in the access log, is v2 PROXY protocol being spoken to NGINX which is not configured for PROXY protocol at all.

2a00:1098:0:82:1000:3b:1:1 - - [09/May/2016:11:08:55 +0100] "\x00" 400 172 "-" "-"

And this is v1 PROXY protocol being spoken to NGINX which is not configured for it:

2a00:1098:0:82:1000:3b:1:1 - - [09/May/2016:11:39:30 +0100] "PROXY TCP4 93.89.134.240 46.235.225.189 64221 80" 400 173 "-" "-"

We’re increasingly using our IPv4 to IPv6 reverse proxy to host websites on IPv6-only virtual machines. One of the downsides of proxying is that your server doesn’t get to see the client’s real IP address. For non-SSL connections, the proxy can insert an “X-Forwarded-For” header, but SSL is increasingly becoming the norm, and one of the nice things about an SNI-aware reverse proxy is that it doesn’t need to do SSL off load: we don’t need your certificates on our proxy and your traffic stays encrypted until it hits your server. Of course, this means that we can’t go inserting any headers into your connection either.

Fortunately, there is a solution: PROXY protocol. This is a protocol-agnostic mechanism for passing information from a reverse proxy to a server, including the client IP address.

We’ve just added support for PROXY protocol to our reverse proxy:

Turning this on allows your server to get the client IP address, but as it’s an additional protocol, not part of HTTP, your server must be expecting it: turning this on and pointing it at a standard HTTP server will result in a broken website.

Most web servers have support for this. NGINX has support built in, and just needs “proxy_protocol” adding after the listen directive:

server {
    listen 80   proxy_protocol;
    listen 443  ssl proxy_protocol;
    ...
}

You will probably also want some additional configuration to actually set the IP address that gets used for logs etc., and also to ensure that you only trust proxy information from the real proxy servers.

For Apache, support is provided by mod_proxy_protocol, which needs to be installed manually. Once done, configuration is easy:

<VirtualHost *:443>
  ...
  ProxyProtocol On

  CustomLog ${APACHE_LOG_DIR}/access.log "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

The CustomLog line instructs Apache to use the real client IP rather than the proxy. You should now see v4 addresses being happily logged on your IPv6 server:

root@vm1:~# tail -n 1 /var/log/apache2/access.log
93.93.130.44 - - [29/Apr/2016:14:05:32 +0100] "GET / HTTP/1.1" 200 321 "-" "curl/7.26.0"

Unfortunately the module doesn’t currently provide a way to restrict enablement to trusted proxies only. As such, you’ll probably want to install a firewall to restrict HTTP/HTTPS traffic to only come from our proxies, as otherwise clients could easily fake their IP address.

One thing to watch out for is that although this is applied within a VirtualHost configuration, it’ll actually apply to all virtual hosts on the same IP address and port. This is an unavoidable side effect of the fact that the proxy information is sent before we start talking HTTP. Of course, with IPv6, throwing another IP address at the problem isn’t an issue.

Last week at the UK Network Operators Forum Pete gave a talk about our IPv6 only hosting, progress we’ve made and barriers we’ve overcome.

It’s now available to view online