New Linux “Steal Governor” Targets CPU Contention in Overcommitted Virtual Machines

New Linux “Steal Governor” Targets CPU Contention in Overcommitted Virtual Machines

Linux kernel developers are considering a new “steal governor” designed to improve performance when multiple virtual machines compete for limited physical CPU resources. The proposal uses the amount of CPU steal time observed inside a guest to dynamically reduce or expand the number of virtual CPUs on which that VM prefers to schedule work.

The feature is primarily aimed at heavily virtualized servers where administrators deliberately assign more virtual CPUs than the host can physically execute at once. Under heavy load, that overcommitment can lead to frequent vCPU preemption, lock-holder delays, cache disruption, and ultimately lower overall throughput.

The latest v11 patch series was posted on August 25, 2026, and its developer has proposed consideration during the Linux 7.3 development cycle, potentially targeting Linux 7.4 for inclusion. This means the feature is still under review and is not part of a stable Linux kernel yet.

What Is CPU Steal Time?

CPU steal time is a concept specific to virtualization.

Imagine a virtual machine has eight vCPUs. From inside that VM, the operating system behaves as though those eight CPUs are available. But those virtual CPUs ultimately need to run on the host's physical processors.

If several VMs are competing for the same physical CPU resources, the hypervisor may temporarily prevent one VM's vCPU from running so another VM can use the processor.

The time during which the guest wanted to execute but couldn't because the hypervisor was using the underlying CPU elsewhere is known as steal time.

High steal time is therefore a useful indication that the physical host is experiencing CPU contention.

The “Noisy Neighbor” Problem

The steal governor is designed primarily to address what virtualization engineers commonly call the noisy neighbor problem.

Consider a server hosting several VMs:

  • VM A has 32 vCPUs.
  • VM B has 32 vCPUs.
  • VM C has 32 vCPUs.
  • The physical server has only 64 CPU threads available to those workloads.

That configuration can work perfectly well when the VMs aren't simultaneously busy.

If all three suddenly become heavily loaded, however, they may collectively request more CPU time than the physical machine can provide.

The hypervisor then has to constantly switch between vCPUs.

Those interruptions can become particularly expensive if a vCPU is preempted while holding a lock or executing another latency-sensitive section of code. Other threads may then wait for a vCPU that isn't currently being allowed to run.

The result can be counterintuitive: giving the VMs more virtual CPUs can sometimes make the combined workloads slower.

Linux Could Respond Automatically

Administrators can already monitor steal time and adjust VM configurations themselves.

The new Linux governor attempts to automate part of that process.

The proposed steal_governor driver periodically measures system-wide steal time inside the guest. Based on those measurements, it adjusts a new concept introduced by the patch series called preferred CPUs.

These CPUs aren't being physically removed from the VM.

Instead, Linux treats a smaller subset of its active vCPUs as preferred scheduling targets when contention becomes high.

The scheduler then tries to consolidate workloads onto those CPUs.

How the Steal Governor Works

The basic policy is deliberately straightforward.

By default, the governor checks steal time every 1,000 milliseconds, or once per second.

Two thresholds determine what happens next.

If steal time rises above the default 5% high threshold, the governor reduces the preferred CPU set by one core.

If steal time falls to or below the default 2% low threshold, it adds one core back to the preferred set.

The process repeats gradually.

In simplified form:

High contention → use fewer preferred CPUs

Low contention → restore preferred CPUs

The governor always keeps at least one core preferred and never allows the preferred CPU set to exceed the VM's active CPU set.

Preferred CPUs Are the Key

The other important part of the proposal is the introduction of a new scheduler concept called preferred CPUs.

Linux already maintains CPU masks describing processors that are online and active. The proposed patches introduce an additional cpu_preferred_mask, which is always a subset of the active CPU mask.

When the steal governor isn't being used, the preferred CPU set simply matches the active CPU set.

When contention rises, the governor can shrink the preferred set.

The scheduler then attempts to move ordinary workloads toward those preferred CPUs.

The proposed implementation affects several scheduler decisions, including wakeups, periodic scheduler ticks, and load balancing.

Linux Doesn't Actually Disable the Other CPUs

This distinction is important.

The governor does not hot-unplug vCPUs when contention appears.

Instead, it gives the scheduler a strong hint about which CPUs should be used.

That approach avoids some of the complexity associated with dynamically removing and restoring CPUs.

The proposed preferred CPU list is also exposed through:

/sys/devices/system/cpu/preferred

This read-only sysfs interface allows administrators and tools to see which CPUs the kernel currently considers preferred.

CPU Affinity Is Still Respected

The kernel also won't override explicit CPU affinity settings.

If an administrator or application pins a task specifically to a CPU that the steal governor considers non-preferred, Linux will continue honoring that affinity configuration.

That is an important design decision for enterprise workloads where administrators deliberately pin particular processes or virtualized workloads to specific processors.

Preferred CPUs act as a scheduler policy rather than an absolute restriction.

No Communication Between VMs Is Required

One particularly interesting part of the design is that the VMs don't need to communicate directly with one another.

Each guest observes its own steal time.

When contention increases, each participating VM independently notices the additional steal time and begins consolidating its workload onto fewer preferred CPUs.

As contention decreases, the VMs gradually expand again.

In effect, steal time becomes a shared indirect signal produced by the hypervisor.

That allows multiple guests to respond cooperatively without introducing a dedicated communication protocol between them.

It Works Best When All VMs Participate

There is an important limitation.

The approach works best when all competing VMs enable the feature.

If one guest ignores steal time while its neighbors voluntarily reduce their CPU usage, the nonparticipating VM could potentially retain access to more processing resources.

The patch documentation therefore recommends enabling the governor consistently across VMs sharing the same CPU pool.

This cooperative nature is one reason the proposed configuration builds CONFIG_STEAL_GOVERNOR as a module rather than automatically activating the feature everywhere.

Early Benchmarks Show Large Gains in Some Workloads

The most interesting part of the proposal is its early performance data.

Testing has been conducted across PowerPC, x86, and s390 virtualization environments.

In one PowerPC configuration involving two VMs sharing a constrained physical CPU pool, Hackbench testing showed increasingly significant benefits as contention grew.

Reported combined improvements included roughly:

  • 10.6% with 10 Hackbench groups
  • 37.8% with 20 groups
  • 44.3% with 40 groups

Those figures shouldn't be interpreted as a universal Linux performance improvement. They represent specific virtualized configurations deliberately experiencing CPU contention.

Still, the results illustrate why reducing vCPU competition can sometimes increase total throughput dramatically.

Some Workloads Can Become Slightly Slower

The developer also acknowledges an important tradeoff.

Workloads that simply need as much raw CPU time as possible can experience small regressions when the governor is enabled.

That makes sense because the governor deliberately attempts to consolidate work onto fewer preferred CPUs.

The technique is most useful when vCPU preemption itself creates additional costs beyond simply losing CPU execution time.

Examples include workloads involving:

  • Heavy locking
  • Critical sections
  • Cache-sensitive processing
  • TLB-sensitive operations
  • Databases
  • Mixed OLTP and OLAP workloads

For these applications, reducing costly vCPU preemption can outweigh the apparent disadvantage of scheduling work across fewer preferred processors.

Databases Could Be an Important Use Case

Database servers are specifically mentioned as workloads that may benefit.

Modern databases frequently combine large numbers of threads, synchronization mechanisms, memory-intensive operations, and latency-sensitive transactions.

If the hypervisor preempts a vCPU while that CPU holds an important lock, other database threads may stall while waiting for it.

Consolidating the workload onto fewer vCPUs can reduce those situations and potentially increase overall throughput, even though fewer virtual processors are actively preferred by the scheduler.

The Governor Is Still Being Tested

Development remains active.

A dedicated testing harness for the steal governor was proposed on August 19, adding hundreds of lines of testing code designed to exercise the feature using virtualized environments.

Meanwhile, the patch set itself has gone through numerous revisions. Version 8 appeared in July, version 10 followed in August, and the latest v11 series arrived August 25.

That level of iteration is normal for substantial scheduler changes, particularly ones that can affect performance across multiple processor architectures and virtualization platforms.

Not in Linux 7.2

Despite appearing shortly after Linux 7.2's release, the steal governor should not be confused with a Linux 7.2 feature.

It is still a proposed kernel patch series.

In the latest v11 submission, developer Shrikanth Hegde asked scheduler maintainers whether the series could be queued during the Linux 7.3-rc development cycle with Linux 7.4 as the potential inclusion target.

That timeline is not guaranteed. Kernel maintainers may request further revisions, testing, or architectural changes before accepting it.

Why the Steal Governor Matters

CPU overcommitment is fundamental to modern virtualization.

Cloud providers and enterprise data centers rarely want physical processors sitting idle simply because every VM has been allocated enough theoretical CPU capacity to handle its maximum possible workload.

Overcommitting CPUs allows infrastructure operators to achieve much higher utilization.

The tradeoff appears when many VMs become busy simultaneously.

The steal governor proposes an interesting solution: rather than requiring the hypervisor or administrator to continually reorganize CPU allocations, the Linux guest itself responds to contention by voluntarily reducing its effective CPU demand.

Conclusion

Linux's proposed steal governor takes an unusual approach to improving virtual machine performance: when physical CPUs become overloaded, the guest voluntarily tries to use fewer of its available vCPUs.

By monitoring steal time and dynamically adjusting a new preferred CPU mask, Linux can consolidate workloads when contention is high and gradually expand them again when additional CPU capacity becomes available. Early testing shows that the strategy can produce substantial throughput gains for some heavily contended workloads, although simpler CPU-bound workloads may see smaller benefits or even modest regressions.

The feature is not merged yet, but the latest v11 patches are now being proposed for further scheduler testing, with Linux 7.4 identified as a possible target. If it ultimately lands upstream, the steal governor could give Linux virtual machines a new way to automatically adapt to one of the most persistent problems in heavily consolidated servers: too many virtual CPUs fighting over too few physical ones.

George Whittaker is the editor of Linux Journal, and also a regular contributor. George has been writing about technology for two decades, and has been a Linux user for over 15 years. In his free time he enjoys programming, reading, and gaming.

Load Disqus comments