Why Go's Benchmarking Feels Effortless

Go's standard library makes micro-benchmarking nearly frictionless. A dot product implementation, for example, can be timed by adding a Benchmark* function to a test file:

const benchArrSize = 1000000

func BenchmarkDot(b *testing.B) {
  aa := make([]float32, benchArrSize)
  bb := make([]float32, benchArrSize)

  for b.Loop() {
    dotProduct(aa, bb)
  }
}

Running the benchmarks is then a single command:

$ go test -bench=.
goos: linux
goarch: amd64
pkg: example.com
cpu: 13th Gen Intel(R) Core(TM) i7-13700K
BenchmarkDot-24           3136            381506 ns/op
PASS
ok    example.com     1.199s

The runner handles loop counts automatically, and any one-time setup simply sits before the loop. The experience is clean, but it doesn't translate directly to Python.

The Python timeit Module

Python's closest equivalent is the command-line interface of timeit. Given a module like dot.py:

def dotProductLoop(a, b):
    result = 0
    for i in range(len(a)):
        result += a[i] * b[i]
    return result

You can time a snippet with:

$ python3 -m timeit -s "import dot; a = [1]*1000000; b = [2]*1000000" "dot.dotProductLoop(a, b)"
10 loops, best of 5: 20.7 msec per loop

The -s flag supplies setup code, and the positional string is the benchmarked expression. This works, but it requires spawning a separate Python process, making nontrivial setup code unwieldy. Sharing code between benchmarks is awkward, and maintaining these invocations in a shell script is not a pleasant development experience.

The programmatic API offers more control. A basic invocation looks like:

import dot
import timeit

a = [1] * 1000000
b = [2] * 1000000

N = 10
print(timeit.timeit("dot.dotProductLoop(a, b)", globals=globals(), number=N))

The globals parameter simplifies variable access. This call runs the benchmark for a fixed number of iterations and returns total runtime in seconds, but per-loop time requires manual division. For automatic loop-count selection, the Timeit class provides the autorange method:

print(timeit.Timer("dot.dotProductLoop(a, b)", globals=globals()).autorange())

It returns a tuple of the number of loops and total time:

(10, 0.20622027607)

For multiple trials, the repeat function exists:

print(timeit.repeat("dot.dotProductLoop(a, b)", globals=globals(), number=N, repeat=5))

With output:

[0.2064882309, 0.20689259003, 0.2068122789, 0.2074350470, 0.20825179701]

Notice that repeat forces you back to an explicit number=N, discarding the convenience of autorange. You also receive per-repetition times and must manually find the minimum. Combining repeat with autorange through standard parameters isn't possible — a notable gap.

A Utility to Fill the Gap

To bridge that gap, a small utility function replicates the command-line behavior programmatically:

def autobench(stmt, globals=None, repeat=5):
    # Find the number of iterations to run
    timer = timeit.Timer(stmt, globals=globals)
    num, _ = timer.autorange()
    raw_timings = timer.repeat(repeat=repeat, number=num)
    best = min(raw_timings)
    print(f"{num} loops, best of {repeat}: {best/num:.3f}s per loop")

Usage is straightforward:

autobench("dot.dotProductLoop(a, b)", globals=globals())

Output closely matches the command-line mode:

10 loops, best of 5: 0.021s per loop

This autobench function handles automatic loop counting and repetition, covering what the timeit module's knobs miss. The only feature it omits is automatic time-unit scaling (e.g., switching between ns, us, and ms based on duration). For most benchmarking needs, that's sufficient; third-party packages exist for more advanced scenarios, but this simple function often gets the job done.