Originally posted on crunchingnumbers.live
Over the last month, I got to help with hiring developers for the first time. It was fun to use my teaching skills from graduate school and interpersonal skills from Toastmasters to test the candidates' technical and soft skills. I think the candidates also had a good time interviewing with me.
I believe that live coding, if we do it right, is a great way to test a candidate's technical and soft skills, and for the candidate to see if we are someone they want to work with and learn from. Unfortunately, it's hard to find tutorials for doing live code interviews and writing good problems in the first place. It's even harder to find actual problems because they are kept secret for future use.
In this article, I will explain how to set up a live code interview and how to write good problems based on my experience in teaching. I will reveal the 3 problems that I used in my interviews—yes, I will have to make new ones next time—and analyze why each made a good problem.
1. Setup
I recommend using JSFiddle and Google Hangouts to conduct a live code interview. JSFiddle allows you to write just 1 HTML, 1 JavaScript, and 1 CSS file, which forces you to limit the scope of your problem. It also allows you to write Sass and import libraries. Hangouts makes screen sharing simple. Screen sharing is great because I can check Console messages on the candidate's screen and help them debug their code.
Before I begin my interview, I give my candidate a Hello World project and help them navigate the page. I direct them to Settings and recommend the Tabs layout, a two-column layout that provides plenty of space.
I also explain to the candidate that, as the JavaScript shows, they can use console.log and debugger to solve problems, just like they might in a normal situation. I remind the candidate that they are welcome to Google or ask me what method to use (if they can recall from memory, even better). Lastly, they should talk aloud what they are thinking so that I can help them and assess their skills better.
After the setup is complete, I lay down the format of my 1-hour interview and get a nod from the candidate:
- For the first 5 - 10 minutes, I will ask you questions about yourself.
- Next, I have 3 coding problems. Ideally, each takes 15 - 20 minutes to solve.
- Finally, you get to ask me questions for the last 5 - 10 minutes.
- Does that sound all right with you?
This helps the candidate set right expectations and goals for the interview. I'm actually creating a ruse by revealing there are 3 problems, as I know most candidates will finish only 1 or 2. I want to understand how a candidate processes why they are out of time.
I won't cover interpersonal questions by revealing what I asked. Simply, I encourage you to create a set of questions common to all candidates and tailor the words to an individual's background. I think the goal of interpersonal questions should be to help candidates relax and practice talking to us.
It's time for live coding! Let's take a look at how to write good problems.
2. What Makes a Good Problem?
For an 1-hour interview, I recommend writing 3 real-life problems, each with 3 parts. By real-life, I mean something that you have encountered at work and the candidate will also.
For me, a good problem meets these 5 conditions. If your problem doesn't meet all 5, try writing it again.
a. Your starter code is easy-to-follow and runs without errors.
By writing a clean, simple starter code that runs without errors, we help all candidates start from the same place. Any feature or error that a candidate introduces afterwards can be solely attributed to them.
Writing a starter code is not easy and requires practice. As a rule of thumb, use clear names, add comments, introduce the 3 parts in a natural manner, and indicate where the candidate should write their code.
I recommend that you give the starter code to your coworkers to test the problem and use real-time feedback to iterate. (Make sure to buy them beer or coffee afterwards!)
b. Each part covers exactly 1 skill that you want the candidates to have.
We are being unreasonable if we expect our candidates to know how to do everything. If each part covers 1 skill and most developers will finish two problems, we can test at most 6 skills in a candidate. This forces us to think what skills are truly important for the candidate to succeed in their job.
c. Each part has more than 1 solution.
Our way should not be the only way to solve a problem. Encourage the candidates to pursue their solution. The type, conciseness, and readability of their solution will reflect a candidate's coding experience.
d. The 3 parts are fail-safe.
Each part is an island, entire of itself. If a candidate fails in one part, it's ok; they can still solve the other two and prove their skills.
Ideally, the 3 parts are independent (i.e. can be solved in any order), but this isn't always feasible. If a candidate fails in Part 1 and Part 2 depends on it, we help them complete Part 1 so that the candidate can work on Part 2.
e. The UI shows that you enjoyed creating the problem.
Interviews are stressful. Develop a UI that shows that we put careful thought into the problem because we care about learning a candidate's skills (both soft and technical). By doing so, we can help the candidate be at ease and bring out their best.
3. During Interview
Before a candidate starts a problem, I make sure to explain its purpose and application. I give them a couple of minutes to look at the starter code and ask me questions. While the candidate attempts at a solution, I quietly observe and take notes on what they have tried and when.
If there is a mistake in code, I don’t interrupt right away but give the candidate time to run the code to see its effect. With subsequent mistakes, however, I explain early how to fix them so that the candidate has more time to solve the actual problem.
Lastly, if the candidate has a hard time solving a part, I help them reason step-by-step and am supportive throughout. To iterate, one of the goals of a live code interview is to help the candidate see what it’s like to work with and learn from me.
4. Examples
Let's take a look at the problems that I created. In all 3 problems, I finished the HTML and CSS for my candidates so that they can focus on JavaScript. Furthermore, you will notice that I used native JavaScript methods in my starter code.
I had two reasons. (1) My team uses Ember, a framework most candidates haven't used before and will learn in their first few months by copy-and-paste. I wanted to see that, when a candidate sees an unfamiliar native method, they will use my code to come up with a new one. (2) Since native methods are common to all frameworks, I wouldn't be putting candidates who don't know a particular framework at a disadvantage.
a. Search and Sort Data
Starter Code:
Skills Covered:
- Part 1: Read arrays of nested objects
- Part 2: Filter arrays
- Part 3: Sort arrays
I really like this problem because it shows the bread and butter of my company's app (we deal with data a lot). I want my candidates to have the skills to read, search, and sort data.
My starter code handles the easy case when data is always present. I like to challenge my candidates to handle the real-life case when data is sometimes missing. I let them encounter their first error early in Part 1—if they simply copy-and-paste my code, they will see "undefined" in the table—so that they will tread more carefully in Parts 2 and 3.
Whether a candidate can complete Parts 2 and 3 (search and sort data) depends on completing Part 1 (read data). If the candidate fails Part 1, I make sure to help them complete it so that they can work on Parts 2 and 3. While I help them, the candidate gets to see my mentoring skills.
For sorting, I let the candidates choose an option so that they feel like they have some control over their interview. If a candidate passes, I like to give them bonus questions: How would you modify your code to allow sorting in descending order? To allow the user to toggle between ascending and descending orders? To save time, I ask them to simply describe their answers, not implement them.
b. Optimize Render
Starter Code:
Skills Covered:
- Part 1: Present code
- Part 2: Refactor code
- Part 3: Understand Big-O notation
I was inspired to write this problem because I had faced a performance issue in using D3 at work. When I drew thousands of elements in D3, adding and removing highlights (changing opacity) would cause the frame rate to drop significantly. I came up with a neat trick that I have yet to see documented elsewhere.
This time, my starter code has zero comments. I explain to my candidates that, in Part 1, I am testing their ability to read someone else's code and explain it back to me. I am interested in learning their presentational skills for code reviews.
In Part 2, I point the candidates to highlightListItem and resetHighlights methods, and explain that the code for calculating t0, t1, and timeElapsed and setting innerHTML have been repeated. How would they refactor the two methods?
In Part 3, I ask them to tell me the Big-O notation for highlightListItem. If they are not familiar with Big-O—no worries—I explain what it means by using what's available on their screen (recall fail-safe). As the number of list item grows, how will the time to change opacity grow? I encourage the candidates to explore the relation by changing the number on line 1 from 1000 to 2000, 4000, and 8000.
I follow up with: Do you think it's possible to make highlightListItem a constant time algorithm? That is, no matter how many list items there are, the time to change opacity remains the same? Here, I'm more curious about their ability to discuss and reason than to give the correct solution.
c. QUnit Test
Starter Code:
Skills Covered:
- Part 1: Find array length
- Part 2: Use Console
- Part 3: Understand asynchronicity
As a mathematician-turned-developer, I love writing tests. I'm doubly proud of how Ember treats tests as first-class citizens and even covers writing them in their tutorial. I don't know of any other framework that does this.
That said, the third and last problem isn't really about testing a candidate's ability to write tests. Rather, I want to check that they can use Console to understand what's going on in the code and to get information from the DOM. I also want to check that the candidate understands asynchronicity, something that they will often encounter when getting data from and sending it back to the backend.
5. What Now?
To all developers (both interviewers and candidates), I hope that I have shown you how to conduct a live code interview and the benefits that you get from doing so. I encourage you to try the 3 problems (please click on Fork to create a copy) and write your own based on what you do at work. Don’t forget to share them to help others with live code interviews!
Top comments (1)
Nick, thanks for sharing your experience and thoughts! I'm sorry to hear that your setup didn't go well.
I was glad to have done a test run of JSFiddle with my coworkers, because we discovered that its Collaborate feature (live update on both parties) was broken. Thankfully, Google Hangouts solved this problem, in addition to letting me see the candidate's Console to help with debugging.