Blissfully unaware that his TV is DDoSing the internet.

As many other folks have reported in the last few weeks, we have also been seeing a huge increase in the amount of traffic from abusive web crawlers.

Automated blocking of abusive traffic has long been a necessary evil. We already block a number of badly behaved SEO and AI crawlers on our shared hosting servers and, on-request, some customer servers. We also have a number of automatic tools to block abusive clients. These are typically attempting to brute force passwords or run web security scanners, and we firewall IP addresses out after a number of suspicious requests. These crawlers and bad-actors can already outnumber real organic traffic but the scale of the recent activity, along with the lengths taken to frustrate automated blocking, are on another level.

This new attack comes from a great many IP addresses, each making a tiny number of requests – often just one – from viable-looking but randomly generated User-Agents. We’ve had some success detecting and blocking these but this has not been without some problems. There have been periods where some of our servers have been struggling under the sheer number of connections they’ve had to deal with and some of the blocks we’ve put in place have impacted some legitimate users, especially those on very old computers. If this is you then we’re sorry you’ve been caught up in this.

To give you some idea of the scale, one of our shared hosting servers has in the last month been averaging over 1.5 million fraudulent requests from 290,000 unique IP addresses per day. These are addresses that we have a very high confidence are not making legitimate requests. We’ve identified 5.1 million unique IP addresses during this period and 3.4 million of those have only made a single request, which has made it very difficult for us to block them proactively.

From these IPs, we’ve seen 2.4 million unique User-Agents, and again 1.9 million of these have only been seen once. We’d certainly be surprised if there are as many Windows 95 users left as we’ve seen in these logs. Especially with the ability to talk to a modern TLS-enabled website.

The vast majority of these requests are from consumer ISP networks from a wide variety of countries with Brazil being the biggest contributor by far. UK networks only make up around 2% of the attacks we’ve seen but it’s the same pattern – it’s the big consumer ISPs we see the most. We’ve been careful to exclude any IP addresses that also show what looks like legitimate activity here, so it’s possible this is undercounting, especially with the growth of CGNAT. Around 5% of the requests are from IPv6 addresses.

The rumours that this is a botnet mostly made up of compromised Android SetTop Boxes that’s been leased out to an AI crawler that’s trying to avoid being blocked seem likely to us, but we’re not impressed. This has been a significant waste of staff time over the last few weeks as we’ve worked to mitigate the impact on our customers.

If you’re an ISP that wants to know if you’re part of this botnet please check to see if your ASN is in this file then contact support for a copy of our logs. There are over 22,000 distinct ASNs in our data, with more than 200 of those networks based in the UK.

Thanks to bgp.tools for the network and country data.

Don’t get caught napping when it comes to WordPress updates!

Installing updates is an important part of keeping your computer secure. This is also true when running a website based around popular publishing tools such as WordPress, which have vast communities of plugin and theme developers of varying experience. Plugins often contain security vulnerabilities that can lead to a compromised site and it can be difficult to tell if a new version is a security update or just adding features.

For our managed WordPress customers we have been using the excellent WPScan API for some time to check installed plugins and themes against their list of security vulnerabilities. Dealing with this report was a time-consuming manual process once or twice a week which we wanted to improve.

Helpfully WPScan have recently introduced a feature which allows us to receive these updates in real-time. Now, when a new security update for a plugin or theme is announced we automatically check within a few minutes if a vulnerable version is present on any of our managed WordPress installs, and then generate a support case to ask the customer when they’d like us to install the update. Some customers prefer to perform the updates themselves, which is also fine – the important thing is that the vulnerability gets fixed.

Where a security issue is dangerous and likely to be exploited then we apply our standard zero-day vulnerability process of deploying an update immediately and notifying customers afterwards. A good example of this would have been the recent Loginizer SQL Injection vulnerability, had the WordPress team not already decided this was too dangerous and invoked their rarely-used forced update process.

Now we can respond much more quickly to WordPress vulnerabilities, helping us keep our customers’ websites secure.

Our managed WordPress service includes a number of features that help keep your site secure and protect your data:

• Daily backups, mirrored to multiple sites
• 24/7 monitoring
• Custom security hardening
• Notification and installation of security updates
• You can ask us for help if something goes wrong!

If this sounds interesting then you can order managed WordPress, see details of our other managed applications or contact us if you have questions.