Why interviewing for algorithms still misses the point

Ask engineers at large tech companies why they run applicants through algorithms quizzes and a common answer is that at their scale, one accidental O(n^2) implementation could bring the site down. The underlying assumption is that these interviews ensure nobody writes pathological code in production. My own experience suggests the opposite: I fail most algorithms interviews, yet a large share of the value I've produced on the job has meant solving exactly these phone-screen-level problems for real.

Some of the worst inefficiencies I've found in production code would have been caught by the simplest interview questions. At one company, a core library contained a resizable array that grew its backing store by a constant number of elements on every overflow and then copied the old array into the new one. That's the textbook example of how not to implement a dynamic array — linear-time resizing instead of amortized constant-time. In an interview, anyone on that team would have written the correct version. In production, that flawed implementation accounted for roughly 1% of all GC pressure across all JVM code at the company and was the second largest source of allocations. Fixing it was worth more annually than my lifetime earnings.

The top allocation source was also mundane. Someone had taken a hash function designed for a single byte array and adapted it to take two inputs by concatenating two byte arrays in sequence. Callers who wanted to hash two long values had to convert each to a byte array using a utility that also reversed endianness — a step intended for network byte order. The fix was to change the interface to accept a pair of longs and let the hash function handle byte shuffling internally. Removing those allocations was similarly worth more than my lifetime earnings.

Even constant-factor optimizations that appear in interviews show up in real systems. I've twice been asked, as a follow-up, to speed up a solution by exploiting the fact that densely packed integer IDs can be stored as a bitfield. In production, the equivalent insight usually requires not a clever trick but simply noticing, for example, that two deployments of the same solution to "maximize a hit rate" were both far from optimal.

A better example came from BitFunnel, a search index used in Bing. The configuration of its bloom filters was being tuned by a black-box gradient-descent optimizer, which always produced strange configurations with non-idealities that were then worked around by making the filters less dense — spending more resources and money. The right approach was much simpler: since the core operation in BitFunnel is multiplying probabilities, any configuration could be scored by multiplying a few numbers. The configuration space is small enough to iterate over with a couple of for loops. That ignores some independence assumptions, much like naive Bayesian spam filtering did, but it worked in practice.

Interview solutions don't survive incentive structures

These problems share a few characteristics. Each could be phrased as an interview question. Most team members would have solved it in an interview setting. Each fix was worth more annually than my lifetime earnings. And each flaw persisted for years because nobody was looking for it.

If company-wide incentives aligned with efficiency, none of these would have survived undetected. People who profile their code would have noticed the hottest allocation sites in the most computationally intensive library. There's no standard profiling tool that points at the bloom-filter configuration problem, but exponential savings there came from applying high school math, not wizardry. The real trick was having the time and mandate to look.

I worked at one company that deliberately did not use algorithms quizzes in interviews but did incentivize engineers to do what was good for the company. In that environment, I found only one fix that nearly met the bar of the examples above. The reason is less that incentives made me less productive and more that the company had enough people who viewed company-wide betterment as part of their job, so easy high-value problems usually didn't exist — someone else would have already fixed them.

That company's interview loop — phone screen and onsite combined — was easier than the typical big-tech phone screen, and there was essentially no systems design interview. We briefly tried a hard algorithmic question with new grads but stopped because every one of them failed it. In current hiring discourse that's called "lowering the bar," but most of the job doesn't involve jumping over bars of any height. When it does, the bars are a couple of inches high and easily walked over. By output, that was the most productive company I've worked for.

A history of cargo-culting interview fads

Today's algorithms quizzes are not the first hiring fashion to look odd in hindsight. Before that, Microsoft's dominance made its interview style the template, and influential voices insisted that rivals needed to copy those practices to compete. During that era, questions could be brainteasers like "why are manhole covers round?" or Fermi problems, or purely behavioral interviews with zero technical content. Interviewees prepared with books like How Would You Move Mount Fuji? the way they now grind Cracking the Coding Interview.

People at the time rationalized brainteasers as a test of raw thinking ability. Today that rationale looks thin to a younger generation of engineers, who recognize that answering such questions correlates more strongly with interview prep than with job competence. None of the question-generation process appears to have changed, though — teams crib from the latest prestige company and no one validates their methods.

When I recently asked hiring managers how they verified their interview process was sound and fair, the most common responses were that they didn't, that they couldn't know, or that they just knew it worked. A few claimed they'd studied it but could relay no details about methodology. At a meta level, because interview techniques still spread by imitation rather than evidence, it would be surprising if the current obsession weren't another fad that looks ridiculous in a decade.

Training is also under-incentivized

The failure to catch million-dollar inefficiencies isn't just an issue of individual skill. Even when people would gladly fix such problems, their teams and orgs rarely have efficiency in their objectives. All deployments carry risk, and accepting a performance diff means doing work, taking on risk, and getting nothing measured in return. After enough work to get the top rating in a review cycle, the marginal return on further effort for the employee is negative — at one point, after a raise, I calculated that my compensation increase was 0.03% of the easily quantifiable profit my changes had brought the company. That was on a well-aligned team; most teams are worse.

Those who do try to produce training and documentation find it undervalued compared to feature work. I once wrote a 4,500-word runbook on finding JVM inefficiencies, covering profilers, GC tuning, and custom tooling, plus how to fix common config problems. A handful of engineers who used it could debug issues on their own, and I likely never heard about most of the successes. Had I spent that week doing "real" work instead, I'd have had a concrete, quantifiable result for my performance review. Training that reaches many people also doesn't monetize well — written technical content is effectively given away, while a well-paid corporate trainer can make roughly $100k in a day or two; a successful zine author like Julia Evans brings in about the same over a full year.

In the companies where I found glaring inefficiencies, no single root cause explains their persistence. It's a hedgehog defense of perverse incentive structures, compounded by the fact that engineers unaccustomed to efficiency work find it unfamiliar and therefore harder than it is. When a day's work can return $1M per year in savings, people still don't look — because they've never been rewarded for it, and few organizations train them to. Companies get what they incentivize, and most companies have decided, implicitly, that performance at scale isn't worth measuring. Algorithms interviews won't change that.