As a software developer with over a decade of commercial experience, I've found myself on both sides of the hiring process over the years. Technical tests are commonplace these days, yet they are a controversial topic. At best, they can be a time-consuming endeavour for all parties involved, and at worst, they have been used to exploit potential candidates for unpaid work. I'm here to propose an alternative approach which aims to reduce the barrier to entry for both candidate and employer, whilst getting a more accurate representation of the candidate's proficiency.
The problem with the technical test
There are various issues with the tech test as we know it. I’ve picked out a few of the main ones below.
Bias
The "suggested" time limit on most asynchronous tech tests is often flexible, not a strict rule. Generally speaking, this means there is a bias towards candidates who have more time to spare. These might be unemployed people, students, or younger people who might have fewer responsibilities, for example. This is not ideal when you want to appeal to the broadest set of potential candidates possible. For instance, a parent in their 40s with two young children could be an excellent fit for a senior engineer position, but they might not be able to spare the 3-4 hours a tech test might take.
Time
As discussed above, tech tests can be quite a time sink for candidates, but they can also consume significant amounts of time from the hiring parties too. Checking out multiple codebases, potentially across various platforms (Github, Gitlab, maybe even the odd zip file!), familiarising yourself with the build tooling, and debugging whether an issue is with your machine or the candidate's implementation, all add up. And that's all before you've even got to the stage of reviewing their actual code.
Accuracy
With the recent advancements in generative AI, the standard asynchronous tech test is less reliable than ever for producing an accurate representation of a candidate's proficiency. AI tools such as Chat GPT, Copilot, and many more are allowing more people than ever to enter the world of software development. AI-generated code is not always easily spotted, especially in a smaller codebase like many tech tests. This opens up the potential for a candidate to pass a tech test without possessing the necessary skills for the role.
While AI has certainly intensified this issue, it has actually been a problem for as long as tech tests have been around. In the same vein as using generative AI to complete a tech test, a candidate could just as well allow a friend, or hire someone more skilled than themselves to complete the tech test. Even excessive use of Stack Overflow and copy-pasting could lead to a passing tech test without a true understanding of the underlying concepts.
Of course, a well-rounded hiring process would include a follow-up review and discussion with the candidate about their approach, which would hopefully flush out the most egregious deception attempts. Ideally, the process wouldn't include this potential waste of time at all.
A potential solution
My alternative to the conventional tech test is a process that every developer is already familiar with: the humble code review. The idea is that as part of the standard interview process, you can allow the candidate 20-30 minutes to complete a code review on a pre-prepared pull request in the source control tool of your choice. Then, meet back with the candidate to discuss their review comments, or even let them do it live with you and discuss as they progress through. You could even have multiple pull requests for different roles or levels. For example, a relatively simple pull request which has syntax errors, whitespace inconsistencies, variable name typos, unnecessary repetition, etc. These would all be great opportunities for a more junior developer to pick out improvements. A senior-level pull request might include more in-depth concepts such as ensuring that functions are pure and testable, components are appropriately broken down, the correct types are being used, and avoiding using loose typing or assertions, for example.
This approach respects the time of both the employer and candidate. It requires similar time commitment from both parties and can be completed synchronously. This closes the feedback loop and streamlines the hiring process, eliminating the wait for tech test results from candidates.
The risk of generative AI, copy-pasting, or outside help is significantly reduced, if not completely removed. This is due to the shorter time expectation and synchronous process, but primarily because it requires the candidate to understand the code and suggest improvements, rather than simply adding their own code to meet acceptance criteria.
The post-code review interview isn't all that different from the standard post-tech test interview, as the concepts are broadly the same. You can ask the candidate to explain their thought process and probe for any missed areas to determine if they don't know the topic or simply overlooked it during the interview.
Conclusion
Like much of software development, there are many solutions to the same problem. The concept proposed in this post presents an alternative to the standard tech test. If your current hiring process is time-consuming and doesn't always give the results you want, then perhaps it's worth trying the code review approach. It could lead to a broader hiring pool and a more accessible, well rounded process for everyone involved.
Top comments (2)
While I find asynchronous take home tests to be less stressful and thus enjoyable over one done in interview conditions, as you say its so difficult to write one that doesn't skew in favour of those who can spend most time on it.
I think the code reviews idea is a great way to scope it down and get to the more interesting discussion points faster with a candidate.
Some comments may only be visible to logged-in visitors. Sign in to view all comments. Some comments have been hidden by the post's author - find out more