Point-to-Point Linux

This financial firm decided to build its own redundant WAN routers. Here's a no-nonsense look at the tricky parts and how it all worked.
The Future

We are satisfied with the basic architecture, but a number of improvements need to be made. Given the annoyances of managing multiple T1s in a bonded interface, we now are planning on upgrading the T1s to a second T3. When we do that, we may drop the circuit splitting entirely. Circuit splitting adds a whole new level of complexity to the entire system, and we are unsure if it is worth it.

We have to continue to improve our monitoring of both line status and line quality. It is difficult to complain to circuit vendors about performance if you don't have historical data to back up your assertions.

It would have been convenient to use off-the-shelf servers for the router boxes. We have been investigating the latest 1U rackmount from a major manufacturer, but for several reasons it is unsuitable. The showstopper is that the BIOS does not allow booting from any Flash IDE device. The vendor knows of this limitation but will not fix the BIOS. Thus, we see ourselves building our own systems for the foreseeable future.

We will be building additional internal router boxes for handling LAN traffic, based on the WAN router model we have developed—1U systems with Flash drives running a minimal Fedora kernel.


Although this project is not complete, I feel we've accomplished enough to take a moment to evaluate its success. The key question is: would we do it again? The answer is a qualified yes. Our WAN routers perform the task of providing redundant connections between our office and backup sites. The usefulness of splitting the WAN circuits for redundancy is a wash as it adds so much complexity to the design.

This project has taken significantly longer to complete than we anticipated, a general symptom of developing your own solutions. The answers are there, but you expend more time finding them. Having a sharp, dedicated team (as I did) is crucial to making it all work. Just make sure to budget extra time for all the annoying little problems that are sure to arise.

Resources for this article: /article/7703.

Phil Hollenback is a system administrator at Telemetry Investments in New York City. When he's not upgrading Linux servers or skateboarding, Phil spends his time updating his Web site, www.hollenback.net.