CPU Affinity

Bind specific processes to specific processors with a new system call.
Getting Really Crafty

In the previous example, we bound Quake to one of the two processors in our system. To ensure top-notch frame rates, we need to bind all the other processes on the system to the other processor. You can do this by hand or by writing a crafty script, but neither is efficient. Instead, make use of the fact that CPU affinity is inherited across a fork(). All of a process' children receive the same CPU affinity mask as their parent.

Then, all we need to do is have init bind itself to one processor. All other processes, by nature of init being the root of the process tree and thus the superparent of all processes, are then likewise bound to the one processor.

The cleanest way to do this type of bind is to hack this feature into init itself and pass in the desired CPU affinity mask using the kernel command line. We can accomplish our goal with a simpler solution, though, without having to modify and recompile init. Instead, we can edit the system startup script. On most systems this is /etc/rc.d/rc.sysinit or /etc/rc.sysinit, the first script run by init. Place the sample bind program in /bin, and add these lines to the start of rc.sysinit:

/bin/bind 1 1
/bin/bind $$ 1

These lines bind init (whose PID is one) and the current process to processor zero. All future processes will fork from one of these two processes and thus inherit the CPU affinity mask. You then can bind your process (whether it be a real-time nuclear control system or Quake) to processor one. All processes will run on processor zero except our special process (and any children), which will run on processor one. This ensures that the entire processor is available for our special process.

Kernel Implementation of CPU Affinity

Long before Linus merged the CPU affinity system calls, the kernel supported and respected a CPU affinity mask. There was no interface by which user space could set the mask.

Each process' mask is stored in its task_struct as an unsigned long, cpus_allowed. The task_struct structure is called the process descriptor. It stores all the information about a process. The CPU affinity interface merely reads and writes cpus_allowed.

Whenever the kernel attempts to migrate a process from one processor to another, it first checks to see if the destination processor's bit is set in cpus_allowed. If the bit is not set, the kernel does not migrate the process. Further, whenever the CPU affinity mask is changed, if the process is no longer on an allowed processor it is migrated to one that is allowed. This ensures the process begins on a legal processor and can migrate only to a legal processor. Of course, if it is bound to only a single processor, it does not migrate anywhere.

Conclusion

The CPU affinity interface introduced in 2.5 and back-ported elsewhere provides a simple yet powerful mechanism for controlling which processes are scheduled onto which processors. Users with more than one processor may find the system calls useful in squeezing another drop of performance out of their systems or for ensuring that processor time is available for even the most demanding real-time task. Of course, users with only one processor need not feel left out. They also can use the system calls, but they aren't going to be too useful.

Resources

email: rml@tech9.net

Robert Love is a kernel hacker involved in various projects, including the preemptive kernel and the scheduler. He is a Mathematics and Computer Science student at the University of Florida and a kernel engineer at MontaVista Software. He enjoys photography.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Evan Plaice

Anonymous's picture

Thank you...
I have worked in the past on commercial flight simulators where everything relies on real time processing architectures (essentially 80's workstation multi-processor /w shared memory bus tech). I have always wondered what it would be like to leverage real-time processing capabilities on a PC without the OS getting in the way.

Not only did you give the best description of the application of affinity I've seen so far, you answered my 'how do you limit the os to one processor' question.

I can think of so many applications where this would be useful. Ex, performance testing that, for the first time, can be accurately measure without the margin of error typically included by os task switching on the same core.

I really hope that an affinity tool will eventually be added to the vanilla kernel. It would be hugely beneficial to have a tool like yours (with added capability to view affinity) that could easily manage processor affinity.

Check?

Robbienes's picture

How can you determine on which processor a process is running?

htop

Anonymous's picture

htop

setting the cpu affinity for 128 processors

Swamynath's picture

HOw to set the affinity for 128 processors. Since thhe bit pattern is unsigned long only 32 bits can be set. I am not sure how to do it for number of cpus greater than 32.?

Help

Anonymous's picture

Hi
i want to Linux to bind one or more processes to one or more processors using Java code.but i dont know is it possible thru java.Any help?

Re: Kernel Korner: CPU Affinity

Anonymous's picture

Hi,

I was just wondering if there's a similar way to bind IRQ's to a specific processor? That way drivers designed for single CPU can be used safely on a SMP machine as well.

Mark.

Re: Kernel Korner: CPU Affinity

Anonymous's picture

Just look at /proc/sys/irq/xx/smp_affinity
You can read about it in Documentation/filesystems/proc.txt

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState