Playing with Binary Formats
In order to implement a binary format that is of some use, the programmer must have some background information about the arguments that are passed to the loading function, i.e., format->load_binary. The first such argument contains a description of the binary file and the parameters, and the second is a pointer to the processor registers.
The second argument is only needed by real binary loaders, like the a.out and ELF formats that are part of the Linux kernel sources. When the kernel replaces an executable file with a new one, it must initialize the registers associated with the current process to a sane state. In particular, the instruction pointer must be set to the address where execution of the new program must begin. The function start_thread is exported by the kernel to ease setting up the instruction pointer. In this article I won't go so deep as to describe real loaders but will limit the discussion to “wrapper” binary formats, similar to binfmt_script and binfmt_java.
The linux_binprm structure, on the other hand, must be used even by simple loaders, so it is worth describing here. The structure contains the following fields:
char buf: This buffer holds the first bytes of the executable image. It is usually looked up by each binary format in order to detect the file type. If you are curious about the known magic numbers used to detect the different file formats, you can look in the text file /usr/lib/magic (sometimes called /etc/magic).
unsigned long page[MAX_ARG_PAGES]: This array holds the addresses of data pages used to carry around the environment and the argument list for the new program. The pages are only allocated when they are used; no memory is wasted when the environment and argument lists are small. The macro MAX_ARG_PAGES is declared in the binfmts.h header and is currently set to 32 (128KB, 256KB on the Alpha). If you get the message “Arg list too long” when trying to run a massive grep, then you need to enlarge MAX_ARG_PAGES.
unsigned long p: This is a “pointer” to data kept in the pages just described. Data is pushed to the pages from high addresses to low ones, and p always points to the beginning of such data. Binary formats can use the pointer to play with the initial arguments that are passed to the program being executed, and I'll show such use in the next section. It's interesting to note that p is a pointer to user-space addresses, and it is expressed as unsigned long to avoid an undesired de-reference of its value. When an address represents generic data (or an offset in the memory “array”) the kernel often considers it a long integer.
struct inode *inode: This inode represents the file being executed.
int e_uid, e_gid: These fields are the effective user and group ID of the process executing the program. If the program is set-uid, these fields represent the new values.
int argc, envc: These values represent the number of arguments passed to the new program and the number of environment variables.
char *filename: This is the full pathname of the program being executed. This string lives in kernel space and is the first argument received by the execve system call. Although the user program won't know its full pathname, the information is available to the binary formats, so they can play games with the argument list.
int dont_iput: This flag can be set by the binary format to tell the upper layer that the inode has already been released by the loader.
The structure also contains other fields that are not relevant to the implementation of simple binary formats. What is relevant, on the other hand, are a pair of functions exported by exec.c. The functions are meant to help the job of simple binary loaders such as the ones I'll introduce in this article.
unsigned long copy_strings(int argc,char ** argv, unsigned long *page, unsigned long p, int from_kmem); void remove_arg_zero(struct linux_binprm *bprm);
The first function is in charge of copying argc strings, from the array argv into the pointer p (a user space pointer, usually bprm->p). The strings will be copied before the address pointed to by p (argument strings grow downwards). The original strings, the ones in argv, can either reside in user space or in kernel space, and the array can be in kernel space even if the strings are stored in user space. The from_kmem argument is used to specify whether the original strings and array are both in user space (0), both in kernel space (2) or the strings are in user space and the array in kernel space (1). remove_arg_zero removes the first argument from bprm by incrementing bprm->p.
|Speed Up Your Web Site with Varnish||Jun 19, 2013|
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
- Speed Up Your Web Site with Varnish
- Containers—Not Virtual Machines—Are the Future Cloud
- Linux Systems Administrator
- Non-Linux FOSS: libnotify, OS X Style
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- RSS Feeds
- Reply to comment | Linux Journal
3 hours 2 min ago
- Yeah, user namespaces are
4 hours 18 min ago
- Cari Uang
7 hours 49 min ago
- user namespaces
10 hours 43 min ago
11 hours 9 min ago
- One advantage with VMs
13 hours 37 min ago
- about info
14 hours 10 min ago
14 hours 11 min ago
14 hours 12 min ago
14 hours 14 min ago
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?