Interviewing engineers in the age of AI

Most engineering interview processes were built for a world that has quietly stopped existing.

The design was simple enough. A candidate turned up carrying a finite amount of knowledge in their head, you asked questions to work out how much of it was in there, and the strongest candidates knew more, thought faster, and produced better answers on the spot. For years this roughly tracked whether someone could do the job. Not because recall is the job, but because back then you couldn't (easily) fake it, and someone who'd genuinely internalised how a system worked usually had the scars to go with it. The knowledge was standing in for the experience behind it.

Then AI turned up and knocked the legs out from under it.

Today any engineer with a browser open can generate a REST API, explain Paxos, scaffold a React component, write the Terraform or produce a Kubernetes manifest that mostly works, all in the time it takes to read this sentence. The information was always the scarce thing the interview was pricing. It isn't scarce any more.

Plenty of interview processes still behave as though it is, and the result is exactly as daft as you'd expect. Companies ask questions designed to test memorisation. Candidates use AI to memorise the answers. Everyone shakes hands at the end feeling as though something meaningful was measured, when the only thing that actually got measured was who'd done their revision. Knowing something and being able to build something were never quite the same skill. They're now not even close.

The wrong question

A great many technical interviews still hinge on whether a candidate can produce an answer from memory.

What's the complexity of a red-black tree insertion? Explain eventual consistency. How does React reconciliation work? What guarantees does Kafka actually give you?

These aren't bad questions. They're just questions that have stopped telling you anything. A strong engineer retrieves that answer in seconds. So does a mediocre one. So does the assistant sitting in the next tab. When everyone can reach the same answer for free, the answer stops being a signal and starts being noise that only sounds like rigour.

The interesting part has always started once the answer is already on the table.

Can they tell whether it actually applies to the problem in front of them?

Can they name the trade-offs without being prompted?

Can they explain why this approach beats the other one here, in this system, under these constraints? Can they spot the moment the model is producing confident, fluent nonsense, and stop trusting it?

Those are the skills the job is made of, and not one of them shows up in a recall question.

The new scarcity is judgement

Engineering has always been a discipline of trade-offs. AI doesn't abolish a single one of them; it just makes generating the candidate solutions almost free. Which means the scarce skill was never producing options. It was choosing between them.

The strongest engineers I've worked with are rarely the ones who generate the most code. They're the ones who know which code shouldn't exist. They know when to turn down a framework that everyone's excited about. They know when a microservice should have stayed a module. They know when complexity is genuinely earning its keep and when it's just this season's fashion with a conference talk behind it. No model on the market replaces that, and an interview that doesn't go looking for it is measuring the one thing that's just been commoditised while ignoring the one thing that hasn't.

Let them use the AI

A fair number of organisations have responded to all this by trying to ban AI during technical assessments, which is usually a mistake and occasionally a revealing one.

In production, engineers use AI. On side projects, they use AI. During incident response, architecture work, code review, documentation and the long grim afternoons of debugging, they use AI. Building an artificial room where the candidate is forbidden the tools they'd reach for every other day of their working life tells you almost nothing about how they'll actually perform once they're hired. It tells you how they perform with one hand tied behind their back, which is a question nobody was asking.

The far more useful exercise is to watch how they use the thing. Hand them a realistic problem, give them the assistant, and pay attention. Do they swallow the suggestion whole, or do they check it? Do they verify the assumptions buried in the output, or assume the confident tone means it's right? Do they push back when the model is wrong? Do they catch the hallucination before it becomes a commit? Do they take the generated solution and make it genuinely better? You learn more from ten minutes of that than from an hour of watching someone reconstruct a binary tree on a whiteboard from memory - a skill they will use precisely never.

Hire for engineering, not for coding

A lot of companies say they're hiring engineers while assessing only coders, and the gap between those two words is where most of the bad hires live.

Coding is one activity inside engineering. Engineering is understanding what's actually being asked for, balancing constraints that won't all be satisfied, managing complexity before it manages you, keeping the thing alive in production, and making decisions when the information is incomplete and the clock is running. An engineer who can implement a clean solution in half an hour is worth having. An engineer who can look at the ticket and tell you the problem shouldn't be solved at all is often worth more, and you will never find them with a coding exercise, because the coding exercise has already conceded that the problem is worth solving.

Interview loops should reflect that: architecture discussions, how a system should evolve over the next two years, a real incident picked apart with the trade-offs explored out loud, technical strategy under genuine uncertainty.

For senior engineers especially, those conversations expose more in twenty minutes than an algorithm puzzle does in an afternoon.

Ask them to judge, not to recall

The cleanest fix is also the cheapest. Stop asking candidates to produce answers and start asking them to evaluate them.

Put two architectures in front of them and ask which one they'd back, and why, and what fails first, and where the operational risk is hiding, then how they'd evolve the design once the load triples. Hand them a lump of AI-generated code and ask them to review it. What worries them? What would they change before it goes anywhere near production? What assumptions are sitting unspoken inside it, waiting to go off? You can't memorise your way through any of that, and the assistant in the next tab can't sit the conversation for them. Judgement, experience and engineering maturity all surface fast once the question is "is this any good" rather than "can you recite this".

What the good ones actually look like

The best engineers I've interviewed rarely win the room with encyclopaedic recall. They win it with clarity.

They understand the trade-offs and can say them plainly. They explain complicated things simply, which is much harder than explaining them impressively. They're entirely comfortable saying "I don't know", and then telling you how they'd find out - the more useful half of the sentence. They challenge the premise of the question instead of rushing to answer it. They think about the system as a whole rather than the one component you waved in front of them. And most of all, they stay effective when the problem goes vague on them, because a real problem always does, eventually.

AI has changed an enormous amount about how software gets built. Ambiguity has stubbornly refused to join in. It remains as human and as awkward as it ever was, and the engineers who navigate it well will keep standing out long after the novelty of the tooling has worn off. This is the same pattern I keep running into elsewhere: AI accelerates the implementation and leaves the judgement exactly where it found it, and what AI adoption actually looks like on the ground is far less about the model and far more about the people deciding what to point it at.

The last move

AI hasn't made interviewing easier. It's made lazy interviewing easier, and those are not the same thing, however similar they look from the outside.

Any process that leans mainly on memorisation gets a little less useful every quarter, because the thing it measures keeps getting cheaper to acquire. The organisations that adapt will spend less effort on what a candidate knows and far more on how they think, because in a world where everyone can reach the answers, the answers stop being the test. What's left is judgement, which was always the hardest thing to assess, the hardest thing to fake, and the hardest thing of all to teach. Which is rather the point. It's the move you have to make yourself, and no amount of help in the next tab makes it for you.

Further reading

• Joel Spolsky, "The Guerrilla Guide to Interviewing": old now, and aimed squarely at a pre-AI world, but the core instinct (hire people who are smart and get things done, not people who memorised the trivia) has aged a great deal better than the questions most companies still ask.
• Will Larson, lethain.com on hiring: a steady stream of clear-eyed writing on designing loops that measure the job rather than the rehearsal, from someone who has run a great many of them.
• Conway's Law (Melvin Conway's site): not about hiring at all, but the reminder that organisations build what their structure allows them to build is exactly why judgement and communication keep outlasting every shift in tooling.