Netfilter 2: in the POM of Your Hands
Following the publication of “Taming the Wild Netfilter” in the September 2001 issue of LJ [/article/4815], I received a number of e-mails, most asking for more detailed information on working with Netfilter. To satisfy those requests, this time I will delve a little deeper. For those of you who haven't read and tried out a basic setup, I suggest you do so. This article is slightly more advanced and intended for those who have a grasp, tenuous as it may be, on the basics as described in the aforementioned article.
In order to get the most out of Netfilter and the user-land component iptables, you'll need to upgrade both your kernel and iptables. While there's nothing wrong with the kernel and iptables that came with your distribution, the Netfilter code is under constant development. You also certainly might have no idea what patches your distribution saw fit to include in the iptables area (probably none). And, not all patches show up as Netfilter modules or iptables match extensions. I do, however, recommend you don't try what's in this article for the first time on your currently running firewall. Make sure you know what to expect by experimenting on a test system.
The final recommendation in the paragraph above brings up a very important point. This article is based on iptables-1.2.4 and the Linux kernel 2.4.17. Your results will almost certainly vary if you use different versions. The principles will be the same. Don't panic; just try to make some intelligent decisions about what you want. Also understand that just as oil and water don't normally mix, some of the choices you make regarding the modules you want will affect other modules in the same way—that is, some modules don't mix well with others. Looking at the 2.4.18-prepatches, some of the iptables patches applied for this article will be incorporated in 2.4.18. I suggest a look at the 2.4.18 Changelog when this kernel version is finally released to see what you now won't need to try to patch (the patch would fail anyway, detected or not—see below).
In this article, we'll be using Rusty Russell's patch-o-matic that will patch both the iptables and kernel sources. This patch-o-matic (POM) isn't completely automatic and will not attempt to patch anything without your approval. It also will test the patch to be applied first to see if it applies correctly. If it doesn't, you will be told and given several options. If a patch doesn't succeed, your best and safest bet is to continue without applying it. But we'll see that as we go along.
First, download the latest kernel version you want to use (available from www.kernel.org). It can be 2.4.16 or higher. I always recommend waiting at least a week after the latest stable version is out before trying it. That way, if some small bug has made its way into the latest kernel (2.4.15's shutdown filesystem corruption bug comes to mind), you'll probably know about it and avoid a potentially nasty situation.
Using whatever method appeals to you, open and configure your new kernel. This article won't cover kernel building, but a number of articles and sites can bring you up to speed on this (the definitive guide is found in your kernel source tree under Documentation/Changes). I suggest you configure as modules all the Netfilter code. For now, you'll need to choose at least:
1. Code maturity level options-->Prompt for development and/or incomplete code/drivers
2. Networking options-->Network packet filtering (replaces ipchains)
and from here also go ahead and enter
-->TCP/IP Networking-->IP: Netfilter Configuration (click to go to subpage)
3. On the IP: Netfilter Configuration subpage configure all modules.
If you want, select the IPv6 protocol, and you can then also configure the IPv6 Netfilter modules. You'll need to proceed at least as far as the “make dep” step with this kernel to get everything prepared.
By the way, if you read near the bottom of the Help that comes with the Network packet-filtering choice, you'll find you should choose Y if your system will act as a router; otherwise select N, and if unsure, select no. I don't know how sage this advice is. Even simple hosts often need the extra protection that can be afforded by Netfilter. You'll have to decide that question for yourself based on your best risk assessment for your network and how the host will be used. We'll see how Netfilter is, in fact, used on machines other than routers, below.
Once you have your kernel ready, download and open the latest iptables (available from netfilter.samba.org). Change into the iptables directory, and you're almost ready to start. If your kernel is not located in /usr/src/linux, then you'll need to tell iptables where to find it. Additionally, if you don't want to install iptables in /usr/local/, you'll need to specify where you want to install it. There's a reason for each of these parameters (listed below) to be included. Since iptables will be patching the kernel, it must know where to find the kernel, and the iptables binary must know where to find the extensions. The location of the extensions is hard-coded into the binary, so you can't arbitrarily move things around later—you'll have to rebuild and re-install.
The following arguments are available for iptables builds:
KERNEL_DIR=/path/to/kernel/source (default: /usr/src/linux)
BINDIR=/path/to/install/binaries (default: /usr/local/bin)
LIBDIR=/path/to/install/lib-extensions (default: /usr/local/lib)
MANDIR=/path/to/install/manpages (default: /usr/local/man)
At this point, I must note that I often work in a chroot environment, particularly when beating on the kernel sources, etc., so I don't inadvertently damage a working system. However, I've found that the patch-o-matic doesn't work properly chroot-ed. Normally, patch-o-matic will create a temporary directory just above the kernel source tree where it patches and tests, then replaces the kernel sources from there. In a chroot environment (at least on my systems), this directory is never created, and the directory above the kernel source tree becomes a mess as it fills with the kernel source. I've been remiss and not taken the time to look at the problem sufficiently to identify the root cause. But it's not important if you just back up your kernel source before continuing.
The first command you'll want to use (assuming the kernel source directory is located in your $HOME directory) is:
make pending-patches KERNEL_DIR=$HOME/linux
You should have no problems with this target. It will tell you what patches it wants to install. You should say “yes” to all these. If, for some reason, any patch doesn't apply (the program may tell you the patch failed to apply), don't worry. The patch already may be incorporated in the kernel source, but the patch logic was unable to detect it. Just tell the script “no” the second time around. Do not force the patch on. Although this is an option, it normally will result in the script aborting. Once pending-patches completes, it will tell you the kernel is ready for compilation. But we're not quite ready yet.
Patches applied or attempted on my system were ipt_LOG.patch (successful) and tos-fix.patch (failed). The tos-fix.patch failed because a fix was applied, but it did not correspond exactly to the patch in the patch-o-matic.
Once you have applied all pending patches to the kernel source, you're ready to take a look at new patches that have not been incorporated into the kernel.
Some time back, Rusty Russell, lead Netfilter developer, introduced the “make patch-o-matic target” to help folks incorporate new things in the kernel without having to know how to use patch. This target works fairly well, but don't expect it to work perfectly. Sometimes the patch logic is sufficiently old, and the kernel source sufficiently changed, that a particular patch won't work. In recent months, the patch-o-matic has grown quite a bit and some patches break others. So Rusty incorporated yet another target, “most-of-pom”, to allow new iptables builders to get access to as many of the patches as possible but reduce the possibility of failure.
My recommendation to you is to run make most-of-pom, first saying “no” to everything but noting those patches you're interested in. Then run make patch-o-matic, noting any new patches not in most-of-pom that you might be interested in. If no new patches interest you in patch-o-matic, stick to most-of-pom. If any new patches do interest you that are only available in patch-o-matic, take careful note of any other patches those new, interesting patches might break. The worst offender as of this writing seems to be the drop-table patch. We'll not look at that patch for this very reason. But if you need it, just read and heed the warnings with that patch and others that tell you they will be broken by it.
In some cases, such as with the H323-conntrack-nat patch, you will not be able to apply a patch to the kernels we use in this article. If you can't do without this particular patch, you probably won't be able to use the experimental make targets for patching (patch-o-matic or most-of-pom). If this is you (I had this need for one customer's system), you need to go to roeder.goe.net/~koepi. The patch there includes newnat5, h323, talk, ftp and irc nat helpers. This is a standard patch applied using the usual patch utility.
While running make KERNEL_DIR=$HOME/linux patch-o-matic, I selected the following patches:
NETLINK.patch (successful) NETMAP.patch (successful) iplimit.patch (successful) mport.patch (successful) pkttype.patch (successful) psd.patch (successful) realm.patch (successful) snmp-nat.patch (failed: already in kernel) string.patch (successful) tos-fix.patch (failed: already in kernel) ulog.patch (failed: incompatible with kernel or previously applied patch) LOG.patch.ipv6 (failed: already in kernel) REJECT.patch.ipv6 (successful)
While going through pending-patches and the patch-o-matic, you'll want to note a few things. The screen is divided by a line. Above the line is a welcome note and a warning. Below the line is the information you should look at.
First, you will have one or more lines that will tell you which patches already have been applied. You'll note as you go through patch-o-matic that it lists the patches applied in pending-patches. In fact, because we're running patch-o-matic, we don't need to run pending-patches; those patches also would have been applied here. You only need to run pending-patches if you don't also run most-of-pom or patch-o-matic, such as if you decided to use the newnat5/h323 patch mentioned above.
Below the already-applied lines comes a line:
Testing... Patchname.patch STATUS (comment)
The patchname.patch is the patch being tested. The STATUS will normally be NOT APPLIED. The comment will be one of (x missing files) or (x reject of y hunks). Missing files means the patch hasn't been applied (or the particular corresponding files wouldn't be missing), or the patch doesn't match what's in the kernel sources. In general, the reject means a patch has been applied, it just doesn't match the patch in the patch-o-matic for whatever reason. The most common reason is a small fix was made between the patch in patch-o-matic and the patch in the kernel. When you see reject, it is a foregone conclusion that patch won't apply. Not all patches will work. Note that the ulog.patch failed. This failed either because it was incompatible with a previous patch or with the (changed) kernel sources since the patch was originally created.
Third comes information about the patch, author, status of patch, what the patch is, what it does, often an example to clarify how to use it and perhaps a comment.
Finally, the question, do you want to apply this patch? The choices are No (default), yes, test, force, quit and help as indicated by [N/y/t/f/q/?].
Once we've added the patches we want, we're done. Now the kernel is ready for compilation. Or is it? Well, yes. However, we've added targets to the kernel. I suggest you return to the kernel tree, run make oldconfig and select the new Netfilter matches and targets we've incorporated (or what was the sense?). Now you can continue to compile the kernel. After you install the kernel and reboot into it, you're ready to put your new matches and targets to work.
|Designing Electronics with Linux||May 22, 2013|
|Dynamic DNS—an Object Lesson in Problem Solving||May 21, 2013|
|Using Salt Stack and Vagrant for Drupal Development||May 20, 2013|
|Making Linux and Android Get Along (It's Not as Hard as It Sounds)||May 16, 2013|
|Drupal Is a Framework: Why Everyone Needs to Understand This||May 15, 2013|
|Home, My Backup Data Center||May 13, 2013|
- Designing Electronics with Linux
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Dynamic DNS—an Object Lesson in Problem Solving
- Using Salt Stack and Vagrant for Drupal Development
- New Products
- Build a Skype Server for Your Home Phone System
- Validate an E-Mail Address with PHP, the Right Way
- A Topic for Discussion - Open Source Feature-Richness?
- Why Python?
- Tech Tip: Really Simple HTTP Server with Python
1 hour 24 min ago
- Reply to comment | Linux Journal
1 hour 32 min ago
- Understanding the Linux Kernel
3 hours 47 min ago
6 hours 17 min ago
- Kernel Problem
16 hours 20 min ago
- BASH script to log IPs on public web server
20 hours 47 min ago
1 day 22 min ago
- Reply to comment | Linux Journal
1 day 55 min ago
- All the articles you talked
1 day 3 hours ago
- All the articles you talked
1 day 3 hours ago
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi
It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?