IPv4 Anycast with Linux and Quagga

“DNS is down and nothing is working!” is not something anyone ever wants to hear at 3am. Virtually every service on a modern network depends on DNS to function. When DNS goes down, you can't send mail, you can't get to the Web, you can't do much—hopefully, your coffeemaker is not Web-enabled! Administrators do a lot of things to mitigate this risk. The traditional safeguard is to establish multiple DNS servers for a given site. Each DNS client on the network is configured with each of those servers' IP addresses. The chances of all of those servers failing in a catastrophic way are fairly small, so you have a margin of safety.

On the other hand, many stub resolvers will take only two DNS servers, making it nearly impossible to have any meaningful geographical dispersion in your DNS topology. DNS stub resolvers generally use the first of two configured DNS servers exclusively. Consequently, you end up with one server taking the entire query load and one idling, waiting for a failure. Not optimal, but hey, that's the price of redundancy...right? It doesn't have to be.

DNS redundancy and failover is a classic use case for anycast. Anycast is the concept of taking one IP address and sharing it between multiple servers, each unaware of the others. The DNS root nameservers make extensive use of anycast. There are currently 16 root nameserver IP addresses, only eight of which make use of anycast. There are 167 servers that respond to those 16 IP addresses.

Of course, anycast is not limited to DNS. It can be used to provide redundancy and failover for any number of stateless protocols and applications. Anycast might sound a little like multicast, but aside from the one-to-many, IP-to-endpoint relationship, they have very little in common. Multicast takes packets from one sender and delivers them to multiple endpoints, all of which subscribe to a single multicast address using a number of multicast-specific routing technologies. Anycast takes packets from one sender and delivers those packets to the “closest” of a number of possible endpoints using nothing more than standard unicast routing.

Let's start with some terminology:

Anycast endpoints participate in whatever internal routing protocol is being run on your network. All endpoints for a given anycast IP advertise a host route (also known as a /32) for the anycast IP to the router. In other words, each endpoint announces that the anycast IP can be reached through it. Your routers will see the advertisements coming from the various servers and determine the best path to that IP address. Therein lies the magic. Because the IP address is advertised from multiple locations, your router ends up choosing the best path to that IP address, according to the metric in use by that routing protocol—meaning either the path with the fewest hops (RIP), the highest bandwidth path (OSPF) or some other measurement of network goodness. When you send a request to an anycast IP address, it will be routed to the single server with the best metric according to the routers between you and the server.

What if that server fails? If the host fails, it will stop sending out routing advertisements. The routing protocol will notice and remove that route. Traffic then will flow along the next best path. Now, the fact that the host is up does not necessarily mean that the service is up. For that, you need some sort of service monitoring in place and the capability to remove a host from the anycast scheme on the fly.

Naturally, myriad other details need to be worked out when designing an anycast scheme. The general concept is pretty simple, and small implementations are easy to set up. However, no matter what size implementation you're dealing with, proper IP address architecture is a must. Your anycast address should be on its own subnet, separate from any other existing subnets. The anycast subnet must never, ever, be included in a summary.