Kernel Korner - Allocating Memory in the Kernel

In this article, Robert offers a refresher on kernel memory allocation and how it has changed for the 2.6 kernel.

Unfortunately for kernel developers, allocating memory in the kernel is not as simple as allocating memory in user space. A number of factors contribute to the complication, among them:

  • The kernel is limited to about 1GB of virtual and physical memory.

  • The kernel's memory is not pageable.

  • The kernel usually wants physically contiguous memory.

  • Often, the kernel must allocate the memory without sleeping.

  • Mistakes in the kernel have a much higher price than they do elsewhere.

Although easy access to an abundance of memory certainly is not a luxury to the kernel, a little understanding of the issues can go a long way toward making the process relatively painless.

A General-Purpose Allocator

The general interface for allocating memory inside of the kernel is kmalloc():


#include <linux/slab.h>

void * kmalloc(size_t size, int flags);


It should look familiar—it is pretty much the same as user space's malloc(), after all—except that it takes a second argument, flags. Let's ignore flags for a second and see what we recognize. First off, size is the same here as in malloc()'s—it specifies the size in bytes of the allocation. Upon successful return, kmalloc() returns a pointer to size bytes of memory. The alignment of the allocated memory is suitable for storage of and access to any type of object. As with malloc(), kmalloc() can fail, and you must check its return value against NULL. Let's look at an example:

struct falcon *p;

p = kmalloc(sizeof (struct falcon), GFP_KERNEL);
if (!p)
  /* the allocation failed - handle appropriately */

Flags

The flags field controls the behavior of memory allocation. We can divide flags into three groups: action modifiers, zone modifiers and types. Action modifiers tell the kernel how to allocate memory. They specify, for example, whether the kernel can sleep (that is, whether the call to kmalloc() can block) in order to satisfy the allocation. Zone modifiers, on the other hand, tell the kernel from where the request should be satisfied. For example, some requests may need to be satisfied from memory that hardware can access through direct memory access (DMA). Finally, type flags specify a type of allocation. They group together relevant action and zone modifiers into a single mnemonic. In general, instead of specifying multiple action and zone modifiers, you specify a single type flag.

Table 1 is a listing of the action modifiers, and Table 2 is a listing of the zone modifiers. Many different flags can be used; allocating memory in the kernel is nontrivial. It is possible to control many aspects of memory allocation in the kernel. Your code should use the type flags and not the individual action and zone modifiers. The two most common flags are GFP_ATOMIC and GFP_KERNEL. Nearly all of your kernel memory allocations should specify one of these two flags.

Table 1. Action Modifiers

FlagDescription
__GFP_COLDThe kernel should use cache cold pages.
__GFP_FSThe kernel can start filesystem I/O.
__GFP_HIGHThe kernel can access emergency pools.
__GFP_IOThe kernel can start disk I/O.
__GFP_NOFAILThe kernel can repeat the allocation.
__GFP_NORETRYThe kernel does not retry if the allocation fails.
__GFP_NOWARNThe kernel does not print failure warnings.
__GFP_REPEATThe kernel repeats the allocation if it fails.
__GFP_WAITThe kernel can sleep.

Table 2. Zone Modifiers

FlagDescription
__GFP_DMAAllocate only DMA-capable memory.
No flagAllocate from wherever available.

The GFP_ATOMIC flag instructs the memory allocator never to block. Use this flag in situations where it cannot sleep—where it must remain atomic—such as interrupt handlers, bottom halves and process context code that is holding a lock. Because the kernel cannot block the allocation and try to free up sufficient memory to satisfy the request, an allocation specifying GFP_ATOMIC has a lesser chance of succeeding than one that does not. Nonetheless, if your current context is incapable of sleeping, it is your only choice. Using GFP_ATOMIC is simple:

struct wolf *p;

p = kmalloc(sizeof (struct wolf), GFP_ATOMIC);
if (!p)
    /* error */

Conversely, the GFP_KERNEL flag specifies a normal kernel allocation. Use this flag in code executing in process context without any locks. A call to kmalloc() with this flag can sleep; thus, you must use this flag only when it is safe to do so. The kernel utilizes the ability to sleep in order to free memory, if needed. Therefore, allocations that specify this flag have a greater chance of succeeding. If insufficient memory is available, for example, the kernel can block the requesting code and swap some inactive pages to disk, shrink the in-memory caches, write out buffers and so on.

Sometimes, as when writing an ISA device driver, you need to ensure that the memory allocated is capable of undergoing DMA. For ISA devices, this is memory in the first 16MB of physical memory. To ensure that the kernel allocates from this specific memory, use the GFP_DMA flag. Generally, you would use this flag in conjunction with either GFP_ATOMIC or GFP_KERNEL; you can combine flags with a binary OR operation. For example, to instruct the kernel to allocate DMA-capable memory and to sleep if needed, do:

char *buf;

/* we want DMA-capable memory,
 * and we can sleep if needed */
buf = kmalloc(BUF_LEN, GFP_DMA | GFP_KERNEL);
if (!buf)
    /* error */

Table 3 is a listing of the type flags, and Table 4 shows to which type flag each action and zone modifier equates. The header <linux/gfp.h> defines all of the flags.

Table 3. Types

FlagDescription
GFP_ATOMICThe allocation is high-priority and does not sleep. This is the flag to use in interrupt handlers, bottom halves and other situations where you cannot sleep.
GFP_DMAThis is an allocation of DMA-capable memory. Device drivers that need DMA-capable memory use this flag.
GFP_KERNELThis is a normal allocation and might block. This is the flag to use in process context code when it is safe to sleep.
GFP_NOFSThis allocation might block and might initiate disk I/O, but it does not initiate a filesystem operation. This is the flag to use in filesystem code when you cannot start another filesystem operation.
GFP_NOIOThis allocation might block, but it does not initiate block I/O. This is the flag to use in block layer code when you cannot start more block I/O.
GFP_USERThis is a normal allocation and might block. This flag is used to allocate memory for user-space processes.

Table 4. Composition of the Type Flags

FlagValue
GFP_ATOMIC__GFP_HIGH
GFP_NOIO__GFP_WAIT
GFP_NOFS(__GFP_WAIT | __GFP_IO)
GFP_KERNEL(__GFP_WAIT | __GFP_IO | __GFP_FS)
GFP_USER(__GFP_WAIT | __GFP_IO | __GFP_FS)
GFP_DMA__GFP_DMA
______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Kernel flags

Anonymous's picture

Hi, i see in my aircraft linux system flag 7 and flag 0 from 2 different kernel versions. What are they? Pls help. Thks

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState