Branch predictor: How many "if"s are too many? Including x86 and M1 benchmarks!

The Real Cost of an if in Hot Code

Look at any performance-critical loop long enough and you'll find something like a debug flag check that never fires in production. It's tempting to assume such a branch is free. But what happens when "just a few" becomes dozens, hundreds, or thousands of predictable branches? At what point does the branch predictor start costing you real cycles?

Modern CPUs are complex. A branch instruction can take anywhere from one to twenty cycles depending on whether it's taken, predicted, and how the predictor handles it. There are four main kinds of control flow instructions: unconditional jumps (jmp on x86), calls and returns, taken conditional branches, and not-taken conditional branches. Taken branches are the tricky ones: without prediction, they stall the pipeline while the CPU figures out where to go next.

The branch predictor unit (BPU) tries to solve this by guessing the target of a branch very early, using almost no context—just the instruction pointer and some past history. It keeps this data in structures like the Branch Target Buffer (BTB), which remembers where previously taken branches lead. This prediction happens before the decoder stage, which is why it has so little information to work with.

Why Prediction Matters

To see why branch prediction is essential, consider a simple ARM program with an unconditional branch. In a naive CPU pipeline, the fetch unit grabs the next instruction in memory while the branch is still being decoded. That guessed instruction is wrong. Only when the branch finishes executing does the CPU realize its mistake, roll back, and fetch the correct target. Those wasted cycles are called a frontend bubble.

Static prediction and branch delay slots were historical fixes, but today's CPUs use dynamic branch prediction. The BPU predicts the next address even for branches that haven't completed execution, avoiding most bubbles entirely. The BTB is central to this: every taken branch needs a hit there to avoid a stall. The question is how many hits the BTB can handle.

Stressing the BTB

An experiment to test BTB limits is deceptively simple: chain a series of unconditional jmp +2 instructions, each jumping to the next. This forces every instruction to be a taken branch, and each one needs a BTB hit to avoid a pipeline bubble. By measuring cycles per instruction across runs with warm and cold predictors, you can isolate the BTB's behavior.

Run on an AMD EPYC 7642, a cold sequence of 1024 dense jmp instructions took 10.5 cycles per jump. Once the BTB was warmed, subsequent runs dropped to ~3.5 cycles per jump. The difference—about 7 cycles—is the cost of an unpredicted taken branch, even an unconditional one.

Code density matters too. The experiment can pad instructions with nop opcodes to spread branches further apart. This doesn't change execution count, only the code's physical size and how addresses index into the BTB. Varying block size reveals how the predictor handles layout effects.

Results on AMD EPYC 7642

Testing across different numbers of branches and block sizes produces a clear pattern:

• Under 256 branches in a very small working set (under 2048 bytes), fully predicted branches can run at ~1.5 cycles each.
• Up to 4096 branches, the cost settles around 3.4 cycles per fully predicted branch, regardless of code density.
• Past 4096 branches, the BTB overflows and cost jumps to ~10.5 cycles per jump—the same as an unpredicted branch.

The 4096 threshold is the BTB's size. Performance counters confirm this: BPUCLEARS.EARLY and BACLEAR.CLEAR events spike only above that limit. Notably, replacing jmp with an always-taken conditional branch produces nearly identical curves, with the conditional variant about 2 cycles slower when predicted.

Not-taken branches behave entirely differently. A sequence of jne instructions that never fire costs a flat 0.3 cycles per block, no matter how many there are. Their conditional branch not-taken doesn't occupy BTB entries, so you can have as many as you like for free. This answers the original question about debug flags: never-taken branches don't affect performance.

Calls and Returns

Calls and returns also need BTB entries for best performance. A test issuing callq and ret pairs—each call hitting a unique function so returns can be predicted—shows costs rising after the 2048 mark on the EPYC 7642. This makes sense: each call/ret pair consumes two BTB slots. Fully predicted call/ret sequences cost about 7 cycles, roughly matching two unconditional jumps.

The practical advice is to keep hot loops under 2048 function calls and under 4096 taken branches on this CPU.

Newer AMD: EPYC 7713

The EPYC 7713 (Milan) behaves differently. Always-taken jmp sequences can run at below 1 cycle each when the hot loop fits in 32KiB and has fewer than 1024 branches. Performance degrades with noise past 4096 jumps and collapses entirely around 6000, suggesting a secondary prediction mechanism picks up some slack before failing. Call/ret pairs start degrading after 2048 and stop predicting beyond ~3000 pairs.

Intel Xeon Gold 6262

The Xeon's behavior is more predictable but has its own quirks. A predicted taken branch costs about 2 cycles, with dense 4-byte blocks paying ~3 cycles due to a documented penalty for closely packed branches. The 4096-jump threshold holds, confirming the 4096-entry BTB. Interestingly, 64-byte blocks show performance breaking at 512 jumps, not 4096. This stems from the BTB's 8-way associativity: at 64-byte intervals, only half the slots can be used.

Call/ret tests fail similarly past 2048 blocks (4096 total flow-control instructions), consistent with the 4K BTB. The 64-byte alignment penalty seen on AMD appears here too, but not on the EPYC chips.

Avoid placing jumps, calls, and returns at regular 64-byte intervals on Intel.

Apple Silicon M1

The M1 tells a different story. For a small hot loop under 4096 bytes of code, predicted jmp instructions run at 1 cycle each—excellent. Beyond that, the cost settles at 3 cycles per predicted jump until the working set grows past ~192KiB, at which point prediction stops working entirely. This matches the M1's 192KB L1 instruction cache, hinting that the BTB is tied to L1 state.

Flushing the M1's branch predictor is notoriously hard, but the data suggests unpredicted taken branches cost more based on jump distance—a behavior not seen on x86. For small code, a miss runs about 8 cycles versus 3 for a hit. Large working sets converge at ~8 cycles.

The call/ret chart on M1 (using bl/ret in ARM terms) again shows the benefit of keeping hot code under 4096 bytes, with 4-6 cycles per call/ret sequence otherwise. Direct comparisons with x86 are unfair because ARM's calling convention differs substantially.

Practical Takeaways

Whether you add that extra if depends on what kind of branch it is. The evidence says:

• Never-taken conditional branches are free. They don't consume BTB entries or predictor resources.
• Taken branches and function calls each need a BTB slot. On x86 server CPUs, that budget is roughly 4096 entries total.
• Keep hot code under 16KiB for x86 to stay within safe prediction territory.
• On M1, try to fit hot code within 4KiB for the fastest branch handling; the prediction budget tracks L1 cache.
• Avoid regular 64-byte spacing of branch instructions on Intel.

For real code, the implication is straightforward: don't waste taken branches, but don't sweat dead-flag checks. The predictor handles never-taken branches effortlessly, no matter how many you add.

The research behind these tests is related to the Spectre v2 attack. Spectre-v2 exploits the fact that BPU state could survive across context switches, allowing an attacker to train the branch predictor and trigger speculative execution of privileged code, leaking data through a cache side-channel. The fix involved preventing BTB state from being shared across isolation boundaries.