Why coding interviews aren't all that bad - Eli Bendersky's website

A Defense of the Coding Interview

Coding interviews are unpopular. Ask any developer about their experience with the whiteboard, and you'll likely hear complaints about unfairness, artificiality, and difficulty. Viral stories of brilliant engineers failing basic algorithmic questions reinforce the narrative.

But after nearly 20 years of interviewing software engineering candidates and reviewing hiring packets, I believe coding interviews are actually one of the best tools we have. They aren't perfect, but they are more effective than the alternatives.

By "coding interview," I mean a session where the candidate writes roughly 50 lines of code involving fundamental data structures: linked lists, arrays, binary search trees at most, plus discussion of recursion, parsing, or algorithmic complexity.

The best coding questions are simple in structure but require a solid grasp of programming fundamentals. Asking a candidate to invert a binary tree isn't testing whether they'll do that on the job; it's probing their understanding of recursion, data structures, and careful pointer handling.

The Objectivity Problem

Coding questions are the most objective way to evaluate candidates. Solving a well-defined problem removes many subjective factors—particularly demographic and background biases that plague other interview formats.

Consider the alternative commonly proposed: just talk to the candidate. Early in my career, I was paired with an experienced engineer to interview a candidate who was exceptionally articulate. Within 20 minutes, I was convinced they should be hired. My partner calmly asked a basic technical question from our domain—designing a low-pass filter. The candidate was immediately lost and struggled for an hour. I was shocked that someone so impressive could fail something so fundamental.

I've seen variations of that scenario repeatedly over the years. Conversational interviews are a reliable way to get a biased read on candidates. In a slate of interviews, it's fine to have one that's more personal to assess cultural fit. But the majority should be objective and technical.

Why Take-Home Projects Fall Short

Another popular alternative is the take-home project—a realistic assignment requiring anywhere from 2 to 20 hours of work. It sounds appealing, but it has three major problems.

First, candidates dislike them. Applying to multiple jobs means multiple long assignments, which discourages exactly the strong candidates companies are trying to attract.

Second, they burden the hiring team. Preparing and evaluating these assignments takes significant time, compounding the load on engineers already juggling their regular work.

Third, and most critically, take-home assignments are highly vulnerable to cheating, which is rampant in the industry. Dedicated services and forums sell solutions, and this isn't a new phenomenon—years ago, I found that most work on freelance platforms was either homework or "do my work for me" requests.

Coding interviews can also be gamed; you can find most common questions posted online. But in an interview, you're on your own. An interviewer can ask follow-up questions that expose someone who merely memorized answers. Enforcement is far harder with take-home exercises.

Diversity Considerations

There's active discussion about how interviews hurt diversity, and that matters. But coding interviews are significantly more objective than the alternatives when it comes to reducing bias.

"Tell me about yourself" interviews activate automatic rapport with people who share your background and subconscious antagonism toward those who don't. Take-home projects aren't much better: cheating is more accessible to those with money. When high stakes are involved, people will find ways to get ahead. A candidate from a less privileged background may not be able to pay for a solution, but others can.

Coding interviews, imperfect as they are, level the playing field more than the alternatives.

A Necessary Signal

Coding interviews shouldn't be the sole criterion for hiring, but they are an important signal that shouldn't be dropped. To paraphrase a well-worn quote:

Indeed it has been said that a coding interview is the worst form of interview except for all those other forms that have been tried from time to time.

All of this is my own opinion, of course.

Update (2022-03-29): shortly after this post was published, MIT announced it was reinstating its SAT/ACT requirement for admissions. Their reasoning—that these standardized tests, while imperfect, offer more objective criteria for disadvantaged groups than other measures—mirrors the argument for coding interviews. It's a useful parallel.