Teaching our network some MANRS
We’ve recently rolled out software upgrades to our networks that enable improved routing security and we have joined the MANRS (Mutually Agreed Norms for Routing Security) initiative.
Our MANRS certification for our EU and US networks confirms that we block spoofed traffic, drop incorrect routing information and work with others to maintain routing security.
This is beneficial for any customer using our transit and IP services, which includes all dedicated server and virtual server customers.
Resource Public Key Infrastructure (RPKI)
Amazingly, up until the advent of RPKI the entire internet worked on a trust relationship. When another network told us that they were responsible for a range of internet addresses we’d believe them. Border Gateway Protocol (BGP) is how networks communicate routing data to each other and it had no mechanism to confirm that the route and address space being advertised to you were genuine.
Incorrect advertisements result in network traffic being delivered to the wrong destination and incidents, both deliberate and accidental, are common and can cause real harm. For example, $17m in crypto currency was stolen in 2018 via an IP address hijack aimed at Amazon. Youtube has been taken offline as have large parts of the Cloudflare network.
RPKI seeks to address this by providing signed proof that a network operator (identified by their Autonomous System Number) is permitted to originate a specific range of IP addresses. Once a range of IP addresses is signed you know that any announcement of the address space from any other provider is invalid and should be dropped.
Our transit providers are also certified by MANRS for further protection.
An RPKI example
RIPE Labs have created a deliberately invalid routing announcement that can be used to demonstrate and test RPKI. RIPE Labs have published a Resource Origination Authorisation (ROA) that says only AS0 is permitted to announce the prefix 209.24.0.0/24. They then announce that prefix under AS15562.
With RPKI we see that the network listed in the ROA does not match the network announcing the route, so that route is considered invalid and rejected as being a hijack.
Ripe Labs have published a checker that runs in your browser and detects whether you can see this invalid route on your ISP’s network.
From our network, we now get the big smiley face:
Internet Resource Registry (IRR)
RPKI complements another approach to routing security: filtering based on Internet Resource Registry (IRR) data. RPKI allows us to verify if a network is a valid ultimate destination for a particular IP range. Most networks we don’t see directly, we go through another transit providing network. IRR allows us to verify that the network advertising a given route is authorised to originate or transit that route.
The Regional Internet Registries (RIR) allow network providers to register a link between their network and an IP block. Various tools exist (e.g. bgpq3) to create a list of all the internet addresses that a network can originate or transit from their downstream customers. This is be used to generate a filter list that restricts what routes we will accept from peers and downstream customers.
These lists can be very long and change frequently – the list for our network (AS-MYTHIC) is usually 5000 or so records with tens to hundreds of changes per day.
Best Common Practice 38 (BCP 38)
Another issue with insecure routing is “spoofing” — sending IP packets with a fake source address. This is widely used by attackers to cause denial of service attacks. An attacker sends packets with a sender IP address faked to be that of the target machine. The recipient of these packets will send replies to the target machine instead of the originator. This makes it very easy to create distributed denial of service attacks.
BCP38 is a Best Common Practice which requires that networks filter packets that aren’t either to or from an address within their network.
Part of MANRS is not only to implement BCP 38 but also to host an active spoofer. This means if we drop our BCP38 filtering our non-compliance will be published including regular mailings to network operator groups.
Having good MANRS
By combining all these methods routing security is significantly improved. RPKI provides dynamic checking that doesn’t rely on us adding static route lists to our routers. This also provides excellent protection against accidental hijacks from a “route optimiser” gone wrong. IRR forces accurate routing data to generate filters. BCP38 reduces risks to other networks from spoofed packets. Combining all of these means we have much better MANRs at the price of terrible acronyms.
RPKI filtering is now fully deployed on our US and European network and they both now pass Cloudflare’s “Is BGP Safe Yet” test.