By Melanie Dabu, Managing Partner at Digital Knack
Today, as AI transforms virtually every aspect of work, technical screens stand at a fascinating crossroads. The technical assessment, aka the “coding challenge,” is a ubiquitous yet highly contested part of tech hiring that has been ripe for disruption. Having spent the greater part of my career helping engineering teams not only hire but hire better, I have a vested interest in how AI can help us get things more right than ever before.
The current landscape of tech screens
Anyone who has been on either side of the hiring seat for a technical role is no stranger to the “tech screen”, aka the “coding challenge”. For engineering roles that require coding skills, this
part of the interview process is usually administered after candidates pass an initial interview or resume screen.
There are Leetcode-style algorithmic and data structure challenges that aim to evaluate problem-solving aptitude and computational thinking. This method prioritizes thought process over implementation minutia. Another approach is to test for specific knowledge on programming languages, frameworks, and tools. These tests yield hard scores like “90% of proficiency in Python.”
Companies give candidates these assessments in various form factors: custom challenges created by their own engineers, standardized platforms like HackerRank or CoderPad, and via take-home or live programming sessions.
Why they work
While I doubt anyone will claim any method to be perfect, the tech screen has many proven benefits. At its most basic level, it provides the hiring team with data points to answer the grand question of: Can this candidate do the job? Going one step further, a well-executed live coding and problem-solving session can expose how well candidates can do the job, including how they are likely to interact with their future teammates, and vice versa. A well-thought-out tech screen can also be a selling point to candidates who are, in turn, assessing the caliber of people at the hiring company. Lastly, the argument for applying a uniform, repeatable, non-arbitrary step to the hiring process is strong.
Where they fail
Expecting tech screens to predict on-the-job performance remains flawed logic because interview challenges usually lack the practical aspect of complex, real-world context. At best, your tech screen tells you who has the core competencies to code and solve problems from Day 1, but enabling you to spot your next star engineer is another story. However, if it can weed out those who are truly unqualified, then what’s the harm?
Three things come to mind: speed, time, and motivation. A tech screening round adds, at minimum, another week to the interview process. Lags can cause companies to lose good candidates, either to a competing offer or deflated motivation. At least two to three of the hiring team’s engineers need to carve out time to administer these tech screens, often several per week. This lost productivity is very costly, especially to small companies. And we’re back to motivation. There’s the fatigue of candidates completing tech assessment gauntlets at multiple companies, the lack of interest or ability of candidates to prep for these tech exercises, and the lack of connection to the company that candidates can feel if the tech screen is given too early in the process.
Oh and there’s a fourth: just because tech screens are non-arbitrary and uniform, it is not immune to bias. Bias in how they were built, bias in access (practice can improve results), and bias in how they are administered and scored.
Enter, AI
To my surprise, there doesn’t seem to be any tools yet that can either conduct or augment a complex technical interview using AI.
What does exist right now belong in these 3 buckets –
- AI hiring agents that can conduct an interview but not a tech screen, or ones that seem to offer basic-scoring vs problem-solving type of tech assessments
- AI-assists like tech interviewing platforms with AI features that aid with interview analyses, or AI notetaking tools
- AI-powered interview prep tools for candidates, including mock interviews
Opportunities to Improve Tech Screens with AI
Do I expect AI to be a panacea for all the flaws of how tech screens are currently conducted? Not one bit. I am, however, excited about its potential to bridge the gaps in the following areas –
Adaptability of the tech screen. I see adaptive AI-powered technical screening tools aiding hiring teams in two ways: customizing tech screens and dynamically adjusting question difficulty based on real-time performance. This is already how top engineers conduct their interviews; AI has the potential to enable engineering teams to scale and standardize this level of screening more broadly and quickly. Companies can not only improve their ability to assess if candidates are a strong fit; they can spot talent that might be overlooked in one-size-fits-all tests.
Motivating candidates. Ok, so the prospect of an AI-conducted or powered tech interview may not increase someone’s motivation to go through the process, but it can mitigate the factors that cause candidates to lose motivation. With improved customization and adaptability, candidates can expect a less “canned” interview experience, one wherein they can be assessed more on their abilities versus their ability to prepare for the interview. A higher-quality tech screen and overall experience can reduce the fallout of sr-level candidates who are often turned-off by run-of-the-mill tech screens, especially when conducted by less-experienced screeners.
Speed and lost productivity. Engineering hiring teams lifted by AI-powered interviewing tools can spread their interview processes across more interviewers with less risk to quality and consistency. Or perhaps using an AI interviewing tool for the first technical screen can allow teams to focus their live time on fewer candidates. Both can improve speed and reduce lost productivity for the parts of their jobs that do not involve interviewing candidates. All of which can improve the candidate experience and the company’s ability to hire top talent more speedily. See where I’m going with this? The whole ecosystem benefits.
Reduce bias. And finally, the pièce de resistance – do we dare to dream that a tech screen powered by AI can reduce the earlier-mentioned biases? I do. While biases in training and flawed data sets can do more damage than good, I am optimistic about the positive impact that empowering engineering hiring teams with adaptive technical screening tools can have. At minimum, I believe it can expand the availability of what is already working well for many companies. It’s also reasonable to expect that an AI-assist would mitigate the variance in an interviewer’s skill level, preparedness, or even mood. It should also reduce the disadvantage of candidates who have less time to prepare or study for interviews. At best, I posit a more level playing field that doesn’t corner hiring teams into compromising on quality.
Let’s Augment, not Automate
So what is this dream of an AI tech-screening maven that I can’t wait for someone to build? One that can help hiring teams assess candidates beyond the algorithmic-solve with greater ease. One that doesn’t insult the intelligence nor miss the potential of a good engineer. Most of all, one that is customizable in a way that engineering teams can incorporate what makes their microculture unique, and find others who are well-fitting puzzle pieces to the full picture they are in the journey of building.
