Playing with Binary Formats

This article explains how kernel modules can add new binary formats to a system and show a pair of examples.
The Binary Parameters

In order to implement a binary format that is of some use, the programmer must have some background information about the arguments that are passed to the loading function, i.e., format->load_binary. The first such argument contains a description of the binary file and the parameters, and the second is a pointer to the processor registers.

The second argument is only needed by real binary loaders, like the a.out and ELF formats that are part of the Linux kernel sources. When the kernel replaces an executable file with a new one, it must initialize the registers associated with the current process to a sane state. In particular, the instruction pointer must be set to the address where execution of the new program must begin. The function start_thread is exported by the kernel to ease setting up the instruction pointer. In this article I won't go so deep as to describe real loaders but will limit the discussion to “wrapper” binary formats, similar to binfmt_script and binfmt_java.

The linux_binprm structure, on the other hand, must be used even by simple loaders, so it is worth describing here. The structure contains the following fields:

  • char buf[128]: This buffer holds the first bytes of the executable image. It is usually looked up by each binary format in order to detect the file type. If you are curious about the known magic numbers used to detect the different file formats, you can look in the text file /usr/lib/magic (sometimes called /etc/magic).

  • unsigned long page[MAX_ARG_PAGES]: This array holds the addresses of data pages used to carry around the environment and the argument list for the new program. The pages are only allocated when they are used; no memory is wasted when the environment and argument lists are small. The macro MAX_ARG_PAGES is declared in the binfmts.h header and is currently set to 32 (128KB, 256KB on the Alpha). If you get the message “Arg list too long” when trying to run a massive grep, then you need to enlarge MAX_ARG_PAGES.

  • unsigned long p: This is a “pointer” to data kept in the pages just described. Data is pushed to the pages from high addresses to low ones, and p always points to the beginning of such data. Binary formats can use the pointer to play with the initial arguments that are passed to the program being executed, and I'll show such use in the next section. It's interesting to note that p is a pointer to user-space addresses, and it is expressed as unsigned long to avoid an undesired de-reference of its value. When an address represents generic data (or an offset in the memory “array”) the kernel often considers it a long integer.

  • struct inode *inode: This inode represents the file being executed.

  • int e_uid, e_gid: These fields are the effective user and group ID of the process executing the program. If the program is set-uid, these fields represent the new values.

  • int argc, envc: These values represent the number of arguments passed to the new program and the number of environment variables.

  • char *filename: This is the full pathname of the program being executed. This string lives in kernel space and is the first argument received by the execve system call. Although the user program won't know its full pathname, the information is available to the binary formats, so they can play games with the argument list.

  • int dont_iput: This flag can be set by the binary format to tell the upper layer that the inode has already been released by the loader.

The structure also contains other fields that are not relevant to the implementation of simple binary formats. What is relevant, on the other hand, are a pair of functions exported by exec.c. The functions are meant to help the job of simple binary loaders such as the ones I'll introduce in this article.

unsigned long copy_strings(int argc,char ** argv,
        unsigned long *page, unsigned long p,
        int from_kmem);
void remove_arg_zero(struct linux_binprm *bprm);

The first function is in charge of copying argc strings, from the array argv into the pointer p (a user space pointer, usually bprm->p). The strings will be copied before the address pointed to by p (argument strings grow downwards). The original strings, the ones in argv, can either reside in user space or in kernel space, and the array can be in kernel space even if the strings are stored in user space. The from_kmem argument is used to specify whether the original strings and array are both in user space (0), both in kernel space (2) or the strings are in user space and the array in kernel space (1). remove_arg_zero removes the first argument from bprm by incrementing bprm->p.


White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState