Kernel Build Speedups Reach v3: No-Op Builds 74% Faster, and AI Found the Bottlenecks

Kernel Build Speedups Reach v3: No-Op Builds 74% Faster, and AI Found the Bottlenecks

Arm engineer Lorenzo Stoakes posted the third revision of his kernel build speedup series on September 17: 20 patches, rebased against current upstream, on track for Linux 7.4.

The numbers are large enough to be worth stating plainly.

The measurements

On Apple M2 running Linux:

Build typeImprovement
Full build (Clang)~10% faster
Incremental build (Clang)55% faster
Incremental build (GCC)45% faster
No-op build (make on an already-built tree)74 to 76% faster

Earlier revisions measured 36% faster on a full allmodconfig build and around 70% faster incremental on x86_64. AMD EPYC and Threadripper results sit in the same territory as the Apple Silicon ones, which is the useful signal here: the gains are not an artifact of one microarchitecture or one core count.

The no-op figure is the one that changes daily life. Running make on a tree you have already built should do almost nothing, and the fact that it was spending most of its time on work it did not need to do is exactly the sort of thing nobody profiles because everybody assumes it is already fast.

What was actually wrong

The origin of this series is the part worth sitting with: the bottlenecks were found with AI assistance, and they turned out to be single-threaded chokepoints in the build system that had been there for years.

That is a specific and unglamorous category of problem. The kernel build is massively parallel, so a serial stage in the middle of it is invisible in aggregate timings and obvious only if something walks the whole pipeline looking for places where 63 of your 64 cores are idle. It is the same shape as the error-path bugs language models have been turning up throughout the 7.3 cycle: mechanical, exhaustive, and boring in a way humans reading code are not reliably good at.

It is a useful counterweight to the governance arguments running alongside it. The ecosystem has spent this month arguing about AI-generated contributions, from Debian’s vote on responsible use to the origination rule that cost Void Linux 113 packages to the Document Foundation declining AI features outright. This patch series is the other column of the ledger, and notably it is not an AI-authored patch dump. A named engineer at a named company used a tool to find problems, then did the work and put his name on the result. That is the model the kernel has been asking for.

What changed in v3

Mostly consolidation rather than new wins:

  • Rebased on the latest upstream Git
  • Addressed warnings reported by Sashiko
  • Two patches from the original series were already merged
  • The Rust compiler parallel front-end was made opt-in rather than default

That last item is the cautious call. The parallel front-end in rustc is genuinely faster and genuinely newer than the rest of the toolchain, and defaulting a kernel build to it would tie build correctness to a compiler feature still settling down. Opt-in keeps the speedup available to people who want it without making it everyone’s problem.

Who this actually helps

Not most people. If you install kernels from your distribution you will never notice.

It matters a great deal for:

  • Kernel developers, where the incremental and no-op numbers are the entire feedback loop
  • CI systems building many configurations per commit, where a 36% reduction is a straight infrastructure cost cut
  • Distributions and hardware vendors building kernels continuously
  • Anyone maintaining out-of-tree modules, which our DKMS guide covers, since those rebuilds ride the same machinery
# time a no-op build on your own tree
time make -j$(nproc)

# and a full one
time make -j$(nproc) allmodconfig && time make -j$(nproc)

Timing

Stoakes has described the changes as largely uncontroversial, and the v3 patches are on the list now. Linux 7.4 opens for merging after 7.3 ships in October.

Long-standing joke in the kernel community is that a build gives you time to make coffee. These patches close that window a little further, which is a small loss and a considerably larger win.

Background reading

Explainers for the concepts behind this story.