Linux Advanced Routing Tutorial

For years, we used to have a plain-old ADSL in the office—fast download speeds, slow upload, high latency—all that at the cost of $1/GB. We have had so many problems with performance and reliability that after a few years of struggling, we decided to get a second upstream link—SHDSL 5M/5M symmetric link—low latency, consistent speed during the day. It's simply awesome.

Figure 1. Network Overview

However, SHDSL is expensive! Although national traffic is free of charge, international traffic is $5/GB. Compare that with ADSL at $1/GB for both national and international (Table 1).

Table 1. ADSL vs. SHDSL
Speed 7M down/1M up 5M down/5M up
Price national $1/GB Free
Price international $1/GB $5/GB
Public IP addresses 1 1 + /28

There are clear pros and cons to both. That made me think (and I don't do that often), what if we kept ADSL for international traffic that's not that critical for us, and use SHDSL primarily for national traffic—fast and cheap?

Figure 2. Split Traffic between International and National

We also got a /28 subnet with the SHDSL to use for a DMZ, and obviously, we want all the traffic to and from the DMZ to go via the SHDSL link, regardless of whether it's national or international.

Figure 3. Traffic to/from the DMZ

Routing 101

For my plan to work, the company core router needs some way to tell whether a packet from a workstation in the office to an IP.AD.DR.ES in the wild open Internet is international or national and send it accordingly through ADSL or SHDSL. Routers use their routing tables for deciding the fates and paths of the packets they forward. A simple routing table for an office workstation with an address of may look like this:

[workstation] ~ # ip route show dev eth0 proto kernel  scope link  
default via dev eth0

This says that all traffic to all other addresses in the range is to be sent straight to the local network segment attached to interface eth0. All other traffic follows the "default" route and is handed over to the router with IP to handle it. Let's assume we're sending a packet to, Google's public DNS server. For starters, we're trying to "ping" and are observing the traffic on the workstation's eth0 interface with tcpdump:

[workstation] ~ # tcpdump -i eth0 -n -s0 -e
listening on eth0, link-type EN10MB (Ethernet), 
 ↪capture size 65535 bytes
[1] 17:53:59.615650 00:16:17:ec:5c:6c > ff:ff:ff:ff:ff:ff, 
 ↪ethertype ARP (0x0806), length 42:
        Request who-has tell, 
        ↪length 28
[2] 17:53:59.615775 00:14:c2:5b:4f:2c > 00:16:17:ec:5c:6c, 
 ↪ethertype ARP (0x0806), length 60:
        Reply is-at 00:14:c2:5b:4f:2c, length 46
[3] 17:53:59.615796 00:16:17:ec:5c:6c > 00:14:c2:5b:4f:2c, 
 ↪ethertype IPv4 (0x0800), length 98: > ICMP echo request, 
        ↪id 3082, seq 1, length 64

My workstation consulted the routing table for the destination IP and realized it should send the packet to the default router For that, it needs the router's low-level Ethernet address (also known as a MAC address), and the first packet in the above tcpdump output, marked [1], is doing exactly that—asking for the MAC address of IP by "broadcasting" the request to all nodes on the network segment. The second packet, marked [2], is a reply—IP is at MAC address 00:14:c2:5b:4f:2c. Finally, the PING packet can be dispatched, with the destination IP to the router's MAC address 00:14:c2:5b:4f:2c (see the line marked [3]).

All good, so now we can assume our router got the packet and will forward it further toward the destination. Let's see what happens on the router.

We've got both the ADSL and the SHDSL links configured, but all traffic is, by default, sent through ADSL. The ADSL modem is at For now, the SHDSL link sits idle. Here is the routing table:

[router] ~ # ip route show
[1] dev vlan-shdsl  proto kernel  
 ↪scope link  src
[2] dev vlan-adsl  proto kernel  
 ↪scope link  src
[3] dev vlan-office  proto kernel  
 ↪scope link  src
[4] default via dev vlan-adsl

The first line [1] is the SHDSL link—our router's IP on that link is The second line is the link to the ADSL modem; the third line [3] is the network segment with my workstation, and finally, the fourth line [4] is the default route. All packets that don't match any of the local subnets on lines 1, 2 or 3 are sent down to the ADSL modem at that then will forward them to the ISP 2. That's also the fate of our packet to Let's quickly verify what is going to happen to it by calling ip route get:

[router] ~ # ip route get from 
 ↪iif vlan-office from via dev vlan-adsl

As you can see, it will be sent "via", which is the ADSL modem. A simple way to check the full network path from my workstation to any given destination address is the traceroute command. It shows all the routers ("hops") along the way to the destination:

[workstation] ~ $ /usr/sbin/traceroute
traceroute to (, 30 hops max, 
 ↪40 byte packets using UDP
 1 (  
    ↪0.156 ms  0.126 ms  0.124 ms
 2 (  
    ↪0.853 ms  0.831 ms  0.830 ms
 3  core-adsl.isp2 (218.101.x.y)  
    ↪11.765 ms  19.173 ms  19.066 ms
 4  core-xyz.isp2 (203.98.x.y)  
    ↪16.052 ms  15.515 ms  17.153 ms
[... some more hops ...]
13  64.233.x.y (64.233.x.y)  193.826 ms  194.230 ms  194.412 ms
14  * * *
15 (  
    ↪196.086 ms  195.909 ms  195.816 ms

As you can see, the first hop is our router The second hop is the ADSL modem The third hop is one of the ISP2's core routers, and so on and on, passing 11 more routers before the packet finally reaches the destination, aka


Michal Ludvig works for Enterprise IT Ltd in New Zealand as a senior Linux engineer. He's got root access to some of the largest New Zealand corporations, but since he tends to forget his passwords, he is, in general, pretty harmless.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.


bahce00's picture

Coverage of the 2007 MWRC has been fairly comprehensive.

very good, post more please

florian's picture

very good post... more on routing with bird/quagga and or advanced routing with linux is welcome. thanks.

nice read! bird config file

Moosa's picture

nice read! bird config file killed the purpose of obscuring AS no. in the table.(just saying)

Very nice and informative.

khappieinstein's picture

Very nice and informative. Inspired me to start Experimenting again. Thanks again. Keep writing. It been long i have touch base with LINUX.

Geek Guide
The DevOps Toolbox

Tools and Technologies for Scale and Reliability
by Linux Journal Editor Bill Childers

Get your free copy today

Sponsored by IBM

Upcoming Webinar
8 Signs You're Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
11am CDT, April 29th
Moderated by Linux Journal Contributor Mike Diehl

Sign up now

Sponsored by Skybot