Linux Advanced Routing Tutorial

Feeding the Bird

BGP is the routing protocol of today's Internet. Each and every ISP on the planet tells all other ISPs on the planet what IP addresses it has in its network and what other IP addresses can be reached through that ISP's network. The protocol to exchange this kind of information is called BGP (Border Gateway Protocol).

I'm not going to dive in to the details of BGP. All we need to know is that our ISP1 can use BGP to send us the list of all the national prefixes. Instead of entering them manually, we will listen to the "BGP feed" and update our routing table from it. This part needs coordination with the Internet provider, so I asked our ISP1 to "please assign us a private ASN and send us a BGP feed with national prefixes". BGP usually runs between Autonomous Systems identified by ASN (Autonomous System Number). Any big enough organization can apply and pay for its own official ASN, but for our purpose, it's enough to use an ASN from a private range assigned by the ISP. Table 2 shows the information I received from them.

Table 2. Info from ISP1
ISP Company
ASN 177XY 6452X
Router IP 203.0.113.37 203.0.113.38
Prefixes advertised all national ones none

Now we have the routers' IPs on both sides of the link, both ASNs and confirmation of what prefixes they will send us ("all national ones"). That's all the info we need.

A number of BGP dæmons are available for Linux. I chose Bird (at the time, the most recent version was 1.3.4) and installed it from an RPM.

Bird's configuration is very simple. The config file is at /etc/bird/bird.conf and may look like this:


log syslog all;
protocol kernel {
        import none;
        export all;
}
protocol device {
        scan time 10;
}
protocol bgp {
        description "ISP1 National Routes";
        local as 64526;
        neighbor 203.0.113.37 as 17746;
        source address 203.0.113.38;
        import all;     # Accept all prefixes from our neighbor
        export none;    # Don't send the neighbor any prefixes
}

Essentially, the Bird process talks to the BGP neighbor and imports all the advertised prefixes into its internal routes list. It also talks with the kernel and "exports to the kernel" all the routes it knows—that is, all the ones learned over BGP. That way, it feeds the kernel routing table from the BGP.

With the BGP routes imported, the kernel routing table now has more than 4,000 records:


[router] ~ # ip route show
   [1] 203.0.113.36/30 dev vlan-shdsl  proto kernel 
   ↪scope link  src 203.0.113.38
   [2] 192.168.128.0/24 dev vlan-adsl  proto kernel  
   ↪scope link  src 192.168.128.1
   [3] 192.168.130.0/24 dev vlan-office  proto kernel  
   ↪scope link  src 192.168.130.1
   [4] default via 192.168.128.254 dev vlan-adsl
   [5] 14.1.32.0/19 via 203.0.113.37 dev vlan-shdsl  proto bird
   [6] 27.252.0.0/16 via 203.0.113.37 dev vlan-shdsl  proto bird
   [7] 58.28.0.0/16 via 203.0.113.37 dev vlan-shdsl  proto bird
 ....
[4509] 203.97.0.0/17 via 203.0.113.37 dev vlan-shdsl  proto bird
[4510] 222.153.128.0/18 via 203.0.113.37 dev vlan-shdsl  proto bird
[4511] 222.155.96.0/19 via 203.0.113.37 dev vlan-shdsl  proto bird

Lines [1] to [4] are the same that you've seen before—all of the directly connected subnets and the default route. Lines [5] to [4511] are the routes received over BGP.

Now, let's query the newly populated routing for a path to our favourite f.root-servers.net:


[router] ~ # ip route get 192.5.5.241 from 
 ↪192.168.130.100 iif vlan-office
192.5.5.241 from 192.168.130.100 via 
 ↪203.0.113.37 dev vlan-shdsl        # SHDSL

Excellent, that's what we wanted! A couple traceroute runs from the workstation to both national and international destinations verifies that the new path is used as expected.

The DMZ Challenge

One of the requirements for our second Internet link was the possibility to get a block of public IP addresses for our DMZ (DeMilitarized Zone) where we plan to host some externally accessible services. From the ISP, we got block 203.0.113.208/28, and we want all the traffic to and from the DMZ to use the SHDSL link, regardless of whether it's national or international traffic. Routing the inbound traffic is the ISP's responsibility, but outbound is our job.

Let's see what happens with our current setup. On a test DMZ box, 203.0.113.222, the routing table looks like this:


[dmz-box] ~ # ip route show
203.0.113.208/28 dev eth0  proto kernel  
 ↪scope link  src 203.0.113.222
default via 203.0.113.209 dev eth0

All the traffic goes to the router's IP 203.0.113.209, and the router then decides what to do. Let's traceroute to the Google's public DNS server again and see how it goes:


[dmz-box] ~ # traceroute 8.8.4.4
traceroute to 8.8.4.4 (8.8.4.4), 30 hops max, 60 byte packets
 1  gw-dmz.e-it.co.nz (203.0.113.209)  0.175 ms  0.110 ms  0.110 ms
 2  gw-vlan-adsl.e-it.co.nz (192.168.128.254)  
    ↪8.707 ms  9.080 ms  9.522 ms
 3  core-adsl.isp2 (218.101.x.y)  11.899 ms  13.555 ms  15.585 ms
 4  core-xyz.isp2 (203.98.x.y)  15.114 ms  16.332 ms  17.142 ms
[... some more hops ...]
13  64.233.x.y (64.233.x.y) 195.061 ms  198.098 ms  197.589 ms
14  * * *
15  google-public-dns-b.google.com (8.8.4.4)  
    ↪203.621 ms  204.588 ms  205.792 ms

That's not quite what we wanted. The traffic flows through the ADSL link instead of the SHDSL link. But, it's understandable. Our router selects the network paths based only on the destination IP, disregarding the source IP. As soon as it sees a packet for 8.8.4.4, it sends it down the ADSL path, regardless of whether it comes from my workstation at 192.168.130.100 or from the DMZ box at 203.0.113.222.

Fortunately, the Linux kernel can have up to 255 independent routing tables and up to 32,768 different rules specifying which routing table to look up for each packet. The standard ruleset on a recent Linux machine looks like this:


[router] ~ # ip rule show
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

There are no constraints for the rules. All three rules may be consulted for packets "from all" addresses until a matching route is found. According to the rules, the "local" table is looked up first. The "local" table is maintained by the kernel and contains rules for broadcast addresses and similar special destinations. You can list its contents, but unless you know exactly what you're doing, you better not change it:


[router] ~ # ip route show table local
broadcast 192.168.130.0 dev vlan-office  proto kernel  
 ↪scope link  src 192.168.130.1
local 192.168.130.1 dev vlan-office  proto kernel  
 ↪scope host  src 192.168.130.1
local 203.0.113.38 dev vlan-shdsl  proto kernel  
 ↪scope host  src 203.0.113.38
broadcast 203.0.113.39 dev vlan-shdsl  proto kernel  
 ↪scope link  src 203.0.113.38
broadcast 127.0.0.0 dev lo  proto kernel  
 ↪scope link  src 127.0.0.1
local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1
[... a lot more routes ...]

If no match is found in the "local" table, the kernel moves on the the next rule that says "lookup main". The "main" table is the standard table where all the "normal" routes end up—all the routes for directly connected subnets, manually added static routes and also the routes from BGP. The command ip route show works by default with the "main" table and is equivalent to ip route show table main. If still no match is found, the last rule says "lookup default". The "default" table is seldom used and is usually empty.

In fact, the kernel tables have numerical IDs, and the words "main", "local" and "default" are just convenient verbose names defined in /etc/iproute2/rt_tables:


[router] ~ # cat /etc/iproute2/rt_tables
#
# reserved values
#
255     local
254     main
253     default
0       unspec

So, to recap our current situation, we've got the "main" table with a list of all national routes for SHDSL and one default route for ADSL for all international traffic. And, we've got a default rule that says "for all source addresses consult the main table".

______________________

Michal Ludvig works for Enterprise IT Ltd in New Zealand as a senior Linux engineer. He's got root access to some of the largest New Zealand corporations, but since he tends to forget his passwords, he is, in general, pretty harmless.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

cool

bahce00's picture

Coverage of the 2007 MWRC has been fairly comprehensive.
http://bahcemobilyam.blogspot.com

very good, post more please

florian's picture

very good post... more on routing with bird/quagga and or advanced routing with linux is welcome. thanks.

nice read! bird config file

Moosa's picture

nice read! bird config file killed the purpose of obscuring AS no. in the table.(just saying)

Very nice and informative.

khappieinstein's picture

Very nice and informative. Inspired me to start Experimenting again. Thanks again. Keep writing. It been long i have touch base with LINUX.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState