Advanced Packet Data Testing with Linux

We wanted to do the hook as a loadable module. However, we found we had to modify the Linux 2.0.36 networking code. We modified the kernel to export the inet_proto_ops structure globally and tweaked the sock_register and sock_unregister routines to turn interrupts on and off correctly. The Linux 2.2 kernel appears to have made both these changes unnecessary and opens opportunities for loadable-module “over-binding” of the native IP stack operations.

When our module loads, we hook the socket creation process by calling sock_register with the inet_proto_ops structure's create function pointer pointing to our socket create function. From this point, whenever a socket is created, our routine is called.

While we were in the kernel, we applied kernel patches to provide a serial console and microsecond resolution time-stamping for received Ethernet packets. We also modified the netlink driver to increase its number of minor devices.

As mentioned before, when the hook is in place, our socket creation function searches the current task's environment table for the ND variable, and if found, binds the socket to that device.

The function first finds the environment table in physical memory (in which the kernel runs), but the environment table memory location is stored as user memory addresses under the task structure (current-->mm-->env_start to current-->mm-->env_end). We use get_phys_addr from fs/proc/array.c to convert from the user address to the physical address. Using this address translation function, our code searches the current task's environment for ND=.

If the ND variable is found, our code calls sock_setsockopt with the SO_BINDTODEVICE option and the specified device. However, sock_setsockopt copies the ifreq structure from user space to kernel space. Since our code lives in kernel space, we had to implement the device bind by setting sk->bound_device ourselves.

Once we had the /dev entries for the USN device and the SO_BINDTODEVICE mechanism compiled into the kernel, we were ready to begin testing our concept. The initial test was very straightforward; we used ifconfig to configure a USN device and then set the new kernel-supported ND environment variable to that device. From that point, any application we ran in that environment had its outbound IP packets sent into our USN device. To actually watch the packets, we executed tcpdump -i on our USN device (in this case, usn0).

The ping program was the simplest way we knew how to start sending IP traffic, so we executed it first. Everything proved to work the way we envisioned. The ping packets began showing under tcpdump. There was no one reading the device file at the time, so no response was made to the ICMP requests. But for our purposes, everything worked as planned.

We then tried ftp and telnet, another pair of IP applications that generate IP traffic. We saw the ftp and telnet traffic appear on our USN device, and that was sufficient proof to us that our idea was sound—until we decided to open an xterm.

Naturally, as for any X application, to see the xterm we had to set the DISPLAY environment variable to the host name of our desktop machine. Setting the display is simple, but when we tried to launch the xterm, we didn't quite expect what happened next—nothing.

Unfortunately, our kernel modifications and USN device worked a bit too well. After launching the xterm, a glance at the tcpdump output made it apparent what was happening: all of our packets, including the X packets, were being rerouted to the USN device. While that had been a very good thing for ping, FTP and TELNET, it was now a very bad thing. X packets were being written to the USN device, never reaching the X server on the host machine.

We were, to say the least, disheartened. We had several options at that point. The first idea was to note the transport protocol in use (TCP) and the X port number as packets flowed through the kernel, and route those packets through the Ethernet device. This idea was quickly dismissed due to its complexity and lack of elegance.

The second solution was to create a virtual X-user environment the tester could open, as opposed to using telnet or rlogin to access the test-case machine. This concept was fostered by our recent introduction to some of the interesting possibilities the VNC (virtual network computing) software package created. VNC's abilities are impressive, due to the fact that it is multiplatform. Unfortunately, the VNC approach proved to be too slow.

The concept was sound, and ironically, preferable to having the tester telnet in, export his display, etc. For all intents and purposes, we could have testers log in from Windows PCs and open VNC sessions to the Linux boxes. However, since the test community was based in X running on HP-UX, why couldn't we turn to an X server that allowed nesting?

The Xnest X server is both an X client and an X server. It is a client to the X server on the host display (in our case, the HP), and it is a server to X applications you execute on the remote machine (the Linux box). The procedure to getting an Xnest session up and running is quite simple. First, you log in to the remote machine, export your display back to the host display, then run Xnest with a local display string. This display string is then used to run remote applications in the Xnest session.