WIX: a Distributed Internet Exchange
In order to make route changes simpler for the hundreds of WIX users, Citylink has deployed two route servers. Rather than having to peer with every router on WIX, the preference is to peer with only the route servers, dramatically simplifying route table maintenance for everyone. Each client router, then, has to maintain only two BGP sessions—one to each of the two route servers rather than to hundreds of routers.
The route servers (Figure 1) don't carry any traffic at all. They simply reflect routes from one peer to all of the rest. I was surprised to find that the servers are based on the same small footprint diskless 266MHz Pentium board made by Soekris Engineering that is used for the routers. LEAF is installed on compact Flash for reliability and fast booting. The Quagga Routing Software Suite is used to provide BGP services, and the kernel handles packet routing.
Two routers are used for redundancy, for IPv4, and a second set provides services for IPv6. Citylink maintains its own route registry and uses the Routing Policy Specification Language (RPSL) to manage the IPv4 routers. A set of shell and Perl scripts has been developed that use RTCONFIG to construct configuration files for the Quagga software. All this gives Citylink tight control over what peers can announce to the servers and ensures that a replacement can be deployed quickly if required. The whole process is managed with Revision Control System (RCS) to allow backups to be made and to ensure consistency.
Participants on WIX may advertise only those address spaces that are within their own individual network boundaries. The route servers re-advertise what they learn and filter out addresses that should never be routed, which are called Bogons. Examples of these are the loopback address, 127.0.0.1/8; other addresses allocated for private networks, such as 172.16.0.0/12; and non-assigned addresses.
At present, the IPv6 routers are maintained manually. When RPSL for IPv6 is standardised, however, and the amount of work required increases sufficiently, scripts will be used.
One interesting technique deployed on the WIX and its sister exchange, APE, in Auckland, is anycast routing. A good example of anycast routing is the recent addition of a mirror of a root nameserver at WIX. Because of the way BGP and anycast works, a query to the root server goes to the nearest mirror automatically. If an ISP peers at WIX, it can get a 2ms ping time. International paths are over 200ms, so this a huge improvement.
Local media companies also use anycasting to provide content on the exchange at a low marginal cost to ISPs. Rather than having to bring content requested by their customers across expensive international circuits, the ISPs can get it locally.
Anycast also can be used to limit the distribution of the traffic to only local networks. One example of this is The Return of the King premiere parade, which was Webcast using anycast routing from downtown Wellington. Over five hours, about 12 terabytes of data was requested by New Zealand customers, and this content was provided at no cost to ISPs. A mirror of the stream also was provided from a server in the USA for international viewers. Another example is the provision of software mirrors such as one for Debian, the distribution used by Citylink.
One of Citylink's biggest innovations is the provision of wireless Internet connections in cafés and some business premises in Wellington. The first access point was installed in June 2002, and the service was launched officially in November of that year. Currently, more than 200 access points are in operation (Figure 5).
One good example of how this Wi-Fi technology can benefit the community is the recent installation of a wireless access point at the Mary Potter Hospice for terminally ill patients. Two laptops on mobile trolleys are used to allow patients to stay in touch with their families or simply to read material on-line in their own time.
Citylink also operates the Auckland Peering Exchange (APE), which has about 40 peering participants. A recent addition was the Palmerston North Internet Exchange (PNIX), which, although it has only one participant right now, serves as a place for content providers to mirror servers. Other exchanges are planned for other parts of New Zealand in the near future.
Citylink has found that Linux readily is adaptable to whatever the company needs it to do. Having “intelligent” routers running Linux has meant that deeper firewalls can be deployed; it also has given staff access to better debugging tools. For example, TCP Dump can be used to examine traffic through a router in real time if required.
The use of Linux and other open-source software has been a key enabler in creating an affordable public Ethernet and a low cost-of-entry distributed Internet exchange. It will continue to allow the number of exchanges to grow and to fuel innovation and collaboration in other centres around New Zealand.
Resources for this article: /article/8265.
Richard Hulse is a broadcaster based in Wellington, New Zealand. He currently is working on a number of IT projects that involve using Linux to bridge between disparate systems.
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?
|Dynamic DNS—an Object Lesson in Problem Solving||May 21, 2013|
|Using Salt Stack and Vagrant for Drupal Development||May 20, 2013|
|Making Linux and Android Get Along (It's Not as Hard as It Sounds)||May 16, 2013|
|Drupal Is a Framework: Why Everyone Needs to Understand This||May 15, 2013|
|Home, My Backup Data Center||May 13, 2013|
|Non-Linux FOSS: Seashore||May 10, 2013|
- RSS Feeds
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Using Salt Stack and Vagrant for Drupal Development
- Dynamic DNS—an Object Lesson in Problem Solving
- New Products
- Validate an E-Mail Address with PHP, the Right Way
- Drupal Is a Framework: Why Everyone Needs to Understand This
- A Topic for Discussion - Open Source Feature-Richness?
- Download the Free Red Hat White Paper "Using an Open Source Framework to Catch the Bad Guy"
- Tech Tip: Really Simple HTTP Server with Python
- Keeping track of IP address
14 min 49 sec ago
- Roll your own dynamic dns
5 hours 28 min ago
- Please correct the URL for Salt Stack's web site
8 hours 39 min ago
- Android is Linux -- why no better inter-operation
10 hours 55 min ago
- Connecting Android device to desktop Linux via USB
11 hours 23 min ago
- Find new cell phone and tablet pc
12 hours 21 min ago
13 hours 50 min ago
- Automatically updating Guest Additions
14 hours 59 min ago
- I like your topic on android
15 hours 45 min ago
- This is the easiest tutorial
22 hours 21 min ago