Many people don’t understand that giving a good interview is a skill that can be learned and improved. There seems to be a tendency to use the same questions that you were asked when you were a candidate. It also seems like new interviewers often overestimate what can be accomplished in 30 - 60 minutes or where the hiring bar should be for a position. Whatever the reason, the result is that many technical interview questions are pretty awful at assessing anything. So I figured today I’d try to give some guidance on improving your interview questions.
KISS = Keep It Simple Stupid
The first tip I have is that the meat of your question should be a sentence or two. I learned this from Ryan Davis years ago when I was starting to give interviews.
A sentence probably seems too short. After all, most real coding problems and tasks take much more than a sentence to describe. It can seem impossible to ask something useful within that limit. It is difficult to distill problems down to a sentence or two, but it is worth it. Short questions give the candidate more time to work and give you more time to assess the candidate. Remember, any time you are talking is time that the candidate doesn’t have to show their skills. Concise questions also tend to be more realistic for the time frame and more readily adapted to different skill levels. If your question doesn’t fit in a sentence or two, consider breaking it up into multiple parts. That way if a candidate only has time to tackle part of it they still feel like they made progress.
Another advantage to short questions is they allow you to assess the candidate’s ability to think through a problem and identify edge cases or other ambiguities. When we get tasks or features to implement we usually have to find the edge cases and work with the product owner or our co-workers to figure out how to handle them. I want to work with people who can handle a bit of ambiguity in a problem statement and can ask questions to better understand what is going on.
Avoid Jargon
Another issue I see in many interview questions is the use of jargon. Jargon may be company specific terminology; we have plenty of that at Google. Alternatively, programming language or industry-specific terms may be jargon to junior candidates.
Language-specific jargon is often subtle enough we don’t realize we’re using jargon. For example, associative arrays go by different names in different languages. I’m a Rubyist and the associative array data structure in Ruby is called a hash. The same data structure is a dictionary in Python, a map in Go, and an object in JavaScript. If I asked a candidate to implement an algorithm using a hash and they aren’t familiar with Ruby they may not understand what I’m asking even if they know how to use an associative array.
Here’s another jargon example in a question I used for several years, “Write a function that converts an integer to expanded form.” Most adults don’t know what is meant by “expanded form” which makes this question much harder than it needs to be. I often had to define and explain expanded form in the interview which used up precious time. I ended up rewriting the question to remove the jargon which saved time and gave me more information about the candidate’s actual skills.
Give Examples
One of the easiest ways to eliminate jargon and shorten your question is to give examples. For the question expanded form I mentioned above, I rewrote it as “Write a function that takes an integer and prints out the number’s decomposition. For example, 372 would become 3 hundreds, 7 tens, 2 ones.” I’ve also heard of interviewers giving one example input and output and then asking the candidate specify the output for another input to verify understanding.
When you give examples beware of giving too many or showing all the edge cases. One or two is usually sufficient. More can be distracting. Be thoughtful when choosing examples. My example for the expanded form problem doesn’t show edge cases like 0s and 1s. I’ve found that if I specify all the edge cases up front, it is overwhelming for candidates. When I let the edge cases come up organically, the candidate and I can have a conversation about them. Should 703 be 7 hundreds, 0 tens, and 3 ones? Or should it be 7 hundreds and 3 ones? Both are reasonable answers, and a discussion with the candidate about which to pick gives me more insight into how they’ll contribute to the design process if hired.
Build in Ambiguity and Extensions
I have two coding questions I use right now. Both of them are delightfully underspecified. As I mentioned above, I don’t mention the edge cases up front. The questions I use have many reasonable solutions.
None of this is accidental. A problem with built-in ambiguity allows me to adapt the difficulty on the fly. If folks are struggling or we’re running low on time, I can reduce the number of edge cases I ask them to handle. I can tell a candidate what algorithm or data structure to use if they are struggling. If the candidate prefers a particular approach, like iterative instead of recursive, they can still be successful.
I also have several extensions written out for my problems. These are follow up questions that can make the problem more challenging. I might ask how the code would change if the input data is significantly bigger. Or I may ask how the work cloud be distributed across multiple machines or data centers. A relatively common extension I ask is “What’s the complexity of your solution? Can you do better?” All of these let me increase the difficulty and get more information about a candidate’s strengths, weaknesses, and willingness to keep improving or expanding on something after it has been “solved.”
Another reason I have flexible questions is to simulate real software development. No interview question comes close to resembling the real world. However, the problems we solve in the real world are rarely well-defined, and requirements frequently change after we’ve started coding. If a candidate gets frustrated that the problem isn’t well-defined or because I change the requirements that is a signal to ask them about how they work with capricious product owners.
Make It Easier Than You Think You Should
Interviews are an artificial environment. Candidates don’t have their tools and references. They are expected to work independently. They often have traveled and may be short on sleep. They are usually in an environment where they have no idea what to expect, where to find the bathroom, or if there will be a lunch break. It is unreasonable to expect folks to do production-quality coding in this situation. In the real world we don’t work alone, without our tools, in a short time frame while being judged continuously and potentially interrupted.
All these confounding factors mean that your interview questions should be easier than you think is reasonable. My questions are usually things I would have been comfortable attacking early in my career. I aim for things that my co-workers can solve in 10 minutes so that the candidate has a good chance of finishing during a 25-30 minute interview.
I use a rough rule of thirds to decide if my problem is appropriately tricky. With a good problem, I expect a third of candidates to be successful without much help. Another third will make small mistakes or require some hints. The final third is only successful with significant hints, help with an algorithm, or doesn’t finish during the time available.
Draw From Your Work if Possible
My last tip is to draw from your work if possible. In the very first interviewer training I took, the instructor encouraged us to not use problems off the internet or out of books. Instead, he recommended looking for problems in the tasks we did in our day-to-day work or something particularly interesting that came up recently. Primarily this reduces the chances that the candidate has seen your problem before and/or memorized the solution.
The number decomposition problem I wrote about above came directly from work I was doing at the time. One of the two questions I use right now comes from a task I did dozens of times in my career before Google. A co-worker uses a question based on a customer interaction he had. Finding these questions can be challenging, but they exist if you keep your eyes open.
I also feel better about questions based on my actual work. There’s a lot of valid reasons why coding interviews are awful. If my company’s hiring process requires me to ask them, I want them to assess whether the candidate can do the job. If the question comes from the work I do, that makes me more confident that I’m assessing relevant skills.
Conclusions
Coding interviews are awful. I’ve written about that before. However, as interviewers, there are many things we can do to make them less awful and more useful to the hiring manager. I’ve outlined some ways to improve, but I’m sure there are more. I’d love to hear your opinions about good and bad interview questions.
Top comments (0)