Linux Advanced Routing Tutorial
Solving the DMZ Challenge
What we need now is to set up a special routing table for all DMZ traffic and create a rule that will plug it in to the routing decision process.
The first step is to create and populate the new table and name it "dmz". We can choose any non-reserved table ID and add one line into the rt_table file to map the ID to our desired name:
[router] ~ # echo "121 dmz" >> /etc/iproute2/rt_tables [router] ~ # ip route show table dmz [router] ~ #
The dmz table is empty for now. Let's populate it. Apparently, we need just the default route—everything should be sent to ISP1's gateway:
[router] ~ # ip route add default via 203.0.113.37 ↪dev vlan-shdsl table dmz [router] ~ # ip route show table dmz default via 203.0.113.37 dev vlan-shdsl
All we need to do now is activate the table by adding a lookup rule to the routing decision process. The lookup rules can be based on many different parameters—the source and/or destination address or address range, the incoming interface, the TOS (Type Of Service) or an arbitrary "fwmark" value that is used in conjunction with iptables rules. For our purpose, the source IP and incoming network interface are the "selectors" that we need:
[router] ~ # ip rule add pri 1000 from 203.0.113.208/28 ↪iif vlan-dmz lookup dmz [router] ~ # ip rule show 0: from all lookup local 1000: from 203.0.113.208/28 iif vlan-dmz lookup dmz 32766: from all lookup main 32767: from all lookup default
All the packets coming from interface vlan-dmz with source address 203.0.113.208/28 will consult the dmz table. And the dmz table says to send them all to the SHDSL link by default. Let's see if it works:
[dmz-box] ~ # traceroute 184.108.40.206 traceroute to 220.127.116.11 (18.104.22.168), 30 hops max, 60 byte packets 1 gw-dmz.e-it.co.nz (203.0.113.209) ↪0.200 ms 0.124 ms 0.110 ms 2 rt-shdsl.isp1 (203.0.113.37) 2.676 ms 2.599 ms 2.632 ms 3 rt3.isp1 (121.98.ab.cd) 2.715 ms 2.680 ms 2.591 ms [... some more hops ...] 10 216.239.xx.yy (216.239.xx.yy) ↪172.955 ms 172.398 ms 172.339 ms 11 * * * 12 google-public-dns-b.google.com (22.214.171.124) ↪171.703 ms 171.621 ms 170.554 ms
That looks good. The international traffic from the box in DMZ goes over the SHDSL link, while at the same time, we can re-check that from the workstation in the internal LAN, it still goes over ADSL:
[workstation] ~ # traceroute 126.96.36.199 traceroute to 188.8.131.52 (184.108.40.206), 30 hops max, ↪40 byte packets using UDP 1 gw-vlan-office.e-it.co.nz (192.168.130.1) ↪0.170 ms 0.128 ms 0.125 ms 2 gw-vlan-adsl.e-it.co.nz (192.168.128.254) ↪0.861 ms 0.846 ms 0.825 ms 3 core-adsl.isp2 (218.101.x.y) 22.718 ms 22.109 ms 21.931 ms 4 core-xyz.isp2 (203.98.x.y) 20.417 ms 20.222 ms 19.769 ms [... some more hops ...] 13 64.233.x.y (64.233.x.y) 198.456 ms 197.824 ms 197.189 ms 14 * * * 15 google-public-dns-b.google.com (220.127.116.11) ↪194.318 ms 194.153 ms 194.159 ms
Magic, isn't it! It all looks good, and we're done, right? Well, not quite.
More Work for DMZ Routing
What happens when pinging the DMZ box from my workstation? First, the ping packet goes from 192.168.130.100 to the router. The router consults the "main" routing table, sees that the destination 203.0.113.222 is on a directly connected vlan-dmz and sends the ping request there. The DMZ box replies with a packet from 203.0.113.222 to 192.168.130.100—that's my workstation on vlan-office—and sends it to the router. The router sees that it arrived from the DMZ address range and consults the "dmz" table! The only record in there is the default route, so it takes it and dispatches the packet with destination IP 192.168.130.100 to the ISP where it eventually gets discarded. Oops!
Figure 4. Broken Traffic between DMZ and the Office
There are a number of solutions. Either we can copy all the non-BGP routes from the "main" table to the "dmz" table—that's indeed possible but not very elegant. A better option is to add a new routing entry to the "dmz" table that, when matched, will make the kernel move on to the next rule on the list:
[router] ~ # ip rule show 0: from all lookup local 1000: from 203.0.113.208/28 iif vlan-dmz lookup dmz 32766: from all lookup main 32767: from all lookup default [router] ~ # ip route add throw 192.168.128.0/20 table dmz [router] ~ # ip route show table dmz throw 192.168.128.0/20 default via 203.0.113.37 dev vlan-shdsl
What happens now with the ping reply packet is this: the kernel sees a matching rule 1000 and looks up the "dmz" table. There it finds that the best matching route for a destination of 192.168.130.100 is "throw". That tells it to choose the next available rule—in our case, "32766: from all lookup main" and looks up the "main" table. Here it finds a path to the 192.168.130.0/24 subnet and sends the ping reply to vlan-office. Problem fixed.
Another way, equally as good if not even better, is to go straight to the "main" table for all packets that arrive from the DMZ and are destined for our company's internal address range. Let's add a rule for it:
[router] ~ # ip rule add pri 999 from 203.0.113.208/28 ↪iif vlan-dmz to 192.168.128.0/20 table main [router] ~ # ip rule show 0: from all lookup local 999: from 203.0.113.208/28 to 192.168.128.0/20 ↪iif vlan-dmz lookup main 1000: from 203.0.113.208/28 iif vlan-dmz lookup dmz 32766: from all lookup main 32767: from all lookup default
With this rule in place, the router will see that the ping reply packet matches the rule "from 203.0.113.208/28 to 192.168.128.0/20" and will look up the "main" table instead of the "dmz" table right away. Personally, I like this solution the most as it explicitly says "lookup main" instead of a more vague "go to the next rule", whatever it may be, that the former solution does.
Figure 5. DMZ traffic works as expected.
Making the Changes Permanent
Many of the changes we have done so far are only in the router's memory and will disappear after a reboot. Let's make them permanent. Our core router is CentOS 6.2, and the following steps should work on any recent Red Hat Enterprise Linux or Fedora-based system.
A permanent name already has been given to the "dmz" table:
[router] ~ # cat /etc/iproute2/rt_tables # reserved values 255 local 254 main 253 default 0 unspec # local 121 dmz
To add the new ip rules, edit the file /etc/sysconfig/network-scripts/rule-<interface>. As soon as the given interface is configured, the rules will be loaded. The logical place is to load our rules once the DMZ interface vlan-dmz is configured:
[router] ~ # cat /etc/sysconfig/network-scripts/rule-vlan-dmz # ip rule add ... pri 999 from 203.0.113.208/28 to 192.168.128.0/20 ↪iif vlan-dmz lookup main pri 1000 from 203.0.113.208/28 iif vlan-dmz lookup dmz
Finally, populate the "dmz" table on reboot. This is done in the route-<interface> file in the same directory:
[router] ~ # cat /etc/sysconfig/network-scripts/route-vlan-dmz # ip route add ... default via 203.0.113.37 dev vlan-shdsl table dmz # throw 192.168.128.0/20 table dmz ## unless the rule 999 is in place
That's it. The routing now works as designed and survives a reboot.
The topics I explained in this article—multiple routing tables, rules for selecting them and a hint of BGP—should give you enough knowledge to build moderately complex Linux-based routers. In this trade, you will come across a lot more concepts—from OSPF through firewalls and NATs to VPNs and high-availability networks. None of these topics are within the scope of this article; however, in the end, the core thing that makes all this networking fun possible is routing. If you learn the few simple concepts of how IP routing works, you will know half of what networking is all about. Happy experimenting!
Michal Ludvig works for Enterprise IT Ltd in New Zealand as a senior Linux engineer. He's got root access to some of the largest New Zealand corporations, but since he tends to forget his passwords, he is, in general, pretty harmless.