Raspberry Pi on Raspberry Pi

Question: Is the Raspberry Pi 4 any good?
Answer: It’s good enough to run its own launch website with tens of millions of visitors.

Raspberry Pi 4 with PoE mounting points already attached.

The Raspberry Pi 4 is out. It’s a quad core ARM A72 running at 1.5Ghz with 4GB of RAM and native 1Gbps ethernet. This means that according to our benchmarks (PHP 7.3 and WordPress) it’s about 2.5x the speed of the 3B+, thanks to the much faster core design and slight clock speed boost. The downside is that it uses more power. Idle power consumption is up slightly to about 3W, peak is now around 7W, up from 5W. It has some improved video features too and USB3.

We obtained an early sample and benchmarked it running the Raspberry Pi website. We used the main blog, which hosts the www.raspberrypi.org blog, and has historically been the most CPU-intensive site to provide. We now see complete page generation in about 0.8s, compared to 2.1s for the 3B+. Obviously in normal operation, most pages are served from a cache, and so the typical end user experience is much faster.

We were really excited by the Pi 4 and wanted to have them available in our cloud for launch day. Unfortunately, Eben had some bad news for us: netboot on the Pi 4 is only going to be added in a future firmware update. Netboot is critical to the operation of our cloud, as it prevents customers from bricking the servers. Our dreams were shattered.

Our standard Pi Cloud unit consists of 6x9x2 blocks of Pi 3B servers connected to PoE switches with just one wire per server. They all net boot and are controlled through our control panel and API for customer use. Since the lack of netboot means we couldn’t just drop the Pi 4 in as a faster version at this time, we went back to the lab and we built an alpha Pi 4 Cloud on a smaller scale: 18 Pi 4s that Raspberry Pi have very generously given to us, all connected with gigabit ethernet so we can try out the 2.5x faster CPUs, 3x faster Network and 4x RAM capacity. We deployed this to our Sovereign House data centre where it connects to our core network.

In full production, we’ll have six racks of Pi 4 stacked back to back.

What we needed then was a test application. We suggested running the main Raspberry Pi website, as we once did with the Pi 3. But with over twice the horsepower per machine we thought we’d dream bigger. How about hosting the Raspberry Pi website on the Raspberry Pi 4, on the Raspberry Pi 4 launch day?

We’ve set up 14 Pi 4s for PHP processing for the main website (56 cores, 56GB RAM), two for static file serving (8 cores, 8GB RAM) and two for memcached (8 cores / 8GB RAM). Late on Friday night we started moving production traffic from the existing virtual machines to the Pi 4 cluster, completing the move shortly after midnight. Every page from the blog after Sat 22nd June has been generated on a Raspberry Pi 4.

Unfortunately, this configuration isn’t yet ready to become the standard, production environment for the Raspberry Pi website. As noted above, the Pi 4s don’t yet support netboot, and so these ones have local SD card storage rather than netboot and network file storage. This means they can’t be remotely re-imaged and have comparatively unreliable storage. The configuration is also only deployed in a single data centre with all servers on a single switch, whereas in normal usage the Raspberry Pi website is simultaneously hosted in two different data centres for redundancy.

To make things more nerve wracking, Pi 4 requires Debian Buster which is a pre-release version of the operating system (full release July 6th). So it’s a cluster of brand new hardware, with a pre-release operating system and a single point of failure. We very strongly advise our customers not to use this for a mission critical super high profile website under-going the most significant production launch in their history. That really isn’t a very good idea.

We once advised Eben that Raspberry Pi probably wouldn’t sell very many computers. He didn’t listen to us then either.

We haven’t moved the entire stack to the Pi 4. The front-end load balancers, download and apt servers are still on non-Pi hardware, split across three data centres (two in London, one in Amsterdam). The Pi 4 hardware looks well-suited to taking over these roles too, although we’ve kept the current arrangement for now, as it’s well tested and allows us to switch back to non-Pi 4 back-ends quickly if needed.

We haven’t moved the databases to the Pi 4 yet either. We’re not going to do that until we can have nice reliable mirrored storage on enterprise SSDs with high write reliability and long write lifetimes attached to the Pis.

Where do we go from here?

Once netboot on Pi 4 is available, we’ll be adding 4 core A72 / 4GB servers to our Pi Cloud, at a slightly higher price than the existing Pi 3 servers, reflecting the higher hardware and power costs. We are also planning to investigate virtualisation as 1 core / 1GB Raspberry Pi VMs may be of interest to existing Pi3 users. 64 bit kernel support and potentially a 64 bit userland would also now be worth investigating.

If you like the idea of Pi 4 in the cloud, a Pi 4 VM in the cloud or 64 bit ARM in the cloud, tell us your plans at sales@mythic-beasts.com.