How to hire engineers at an early stage startup

How to hire engineers at an early stage startup

One of the hardest and most important parts of starting and growing a company is hiring a great team of engineers.  It’s probably the most important thing you will need to get good at as a founder of a company, and, it’s one of those things that you’ll need to figure out on your own.

The reason it’s so important – in case it’s not obvious – is that the caliber of people you bring aboard is the biggest determinant of success at an early stage company.  Nothing will sink you faster than hiring (and retaining) B or C level talent. It will slow down the development process, lead to crappy product, cause management overhead, make joining you less appealing for A players, and is generally the worst thing you can do.

It’s easy to do hiring wrong (I’ve done it wrong).  Some of the dangers include

• Hiring the wrong people – which leads to having to fire them (or worse, retaining them), which is painful
• Not hiring enough people – which slows your velocity and can be deadly from a business perspective
• Wasting a ton of your time – you can easily spend all your time sourcing, interviewing, etc and not enough time building the actual business. But expect to spend a bunch of your own time on it at least to start
• Wasting a ton of your team’s time – this happens when you bring unqualified candidates too far down the funnel and other people get sucked into interviewing and grading bad fit candidates
• Spending a lot of money – recruiting is a huge business and once you start hiring you will be inundated with offers from recruiters and recruiting platforms to give you your business

Startups vs. Established Companies

As an aside, if you’re at Google or a place like it, this section won’t be as useful.  Google has unique advantages compared to a startup in terms of name-brand, amount of inbound applications, what they can pay and offer in perks.  

What Google or Facebook managers call “hiring” is actually a completely different thing from startup hiring.  It’s really “selection” from a highly curated applicant pool, without having to worry about compensation, applicant interest, sourcing, etc.   It has more in common with what the admissions committee at Harvard does. Most of the quality applicants come to you, and you sort, filter and select, and then, if someone meets the bar, you have to sell them on taking your (usually very competitive) offer.

Startup founders need to operate higher in the funnel, and typically have a much tougher pitch.  They need to go out and find quality engineers who already have other, more secure, higher paid jobs, cut through the noise to convince them to talk, sell them on a dream, convince them to do a real interview process (which might lead to their rejection), and ultimately hop ship to work on a much riskier, usually lower paid venture.  It’s hard.

I still have a lot to learn on how to do hiring well.  But I can share some of what I learned at my last company where I was really proud of the team we put together.

Hiring is a sales process

If you are at a startup, you should view hiring as a sales process.  You are selling the opportunity of working at your company. Engineers are the buyers, and it’s very much a buyers market.  The best candidates need to be sold – do not expect them to come knocking on your door.

You should think of bringing talent onto your team in terms of a sales funnel.  This particular funnel has three steps: 1) generating interest, 2) evaluation of fit, 3) offering and closing.

At the top of the funnel, there is a combination of marketing / branding, inbound traffic, outbound outreach and sourcing, referrals, recruitment, and other ways of getting candidates interested.

Once candidates are interested, they progress into an evaluation phase.  You should think of this in terms of finding whether the opportunity is a “fit.” Is the engineer right for your company and is your company right for the engineer.  It needs to be both.

Finally, for candidates you believe are good fit, you make an offer and go into closing mode to try to get them to sign.

What I describe below is what I think of as the “standard process,” which I used with success at my company.  I’m sure there are a lot of other ways of approaching this (and I’d love feedback on how to improve it).

Who do you want to hire & why should they work at your company?

You’ve got a startup that no one has ever heard of.  Your task is to get high-quality engineers excited to come work there.  You can’t compete in terms of salary or job security. How do you do it?

You need to start by deciding who you are looking for, the more specific the better.

Here’s what I care about:

General intelligence.  I don’t care about experience with particular technologies; I want the smartest people I can find. (I recently had someone I respect disagree here and favor hiring specialists at very early companies in order to cut out ramp up time – I don’t quite buy that strategy though)

Technical thinking.  I don’t require a CS degree (I don’t have one), but I do favor candidates who have demonstrated an ability to think rigorously.  For me this could mean a STEM degree, a philosophy or economics degree, past technical experience at a job, impressive side-projects, etc.

Team players.  No assholes.  I want people who put the good of the team and the product above their personal ambitions.

Product-first mindset.  See the section on Product-first vs. Code-first.  I have some questions below on how to suss this out.

For me this leads to a rubric like:

• Education: Ivy League or Stanford / MIT / UChicago or lower tier only if grades really good
• Experience: worked at a place with a known rigorous hiring process (google, fb, or a smaller, well known startup like Oscar, Paperless Post, Betterment, Vine, etc)
• Any undergrad major fine, but STEM a bonus
• Bootcamp fine, so long as they have pedigree above
• Enough CS knowledge to pass a Google-style interview

On the flip side, you need to consider what your target engineers are going to care about and how can you stand out.

As mentioned, you probably aren’t going to be able to compete based on:

• Name recognition.  You aren’t Google, Facebook or Slack.
• Salary and benefits.  Google pays its top engineers 7-figures and starting engineers good six figures.  You probably can’t.
• Perks.  Free catered food, on-site massage, volleyball courts, renting out the MOMA for holiday parties.  Maybe if you are realllllly well funded, but probably not.
• Number of users.  You aren’t scaling out to millions of users right at the start, so by that measure the impact of your product will be lower.

However, you have some advantages compared to bigger companies.

• Velocity.  Engineers at big companies tend to be frustrated by how long it takes to develop, how many meetings they have, how small of an area that are able to work on, how hard it is to launch, and so on.  At your startup you can (and must) move much faster. You should lean into this when pitching engineers.
• Product ownership and input.  Engineers at your small company can own a much bigger piece of the product.  I always sell this pretty hard because it also selects for the type of engineers I want – people who care about the customer and user experience.
• Culture.   Engineers at small startups have a chance to really shape the culture of the team.  I emphasize how smart everyone on the team is, how we have no assholes, how we self-direct and iterate on our process as a team, how we take ideas from everyone.
• Upside.  There’s not only the potential upside of startup equity, but upside in terms of growth and leadership if the company grows.  Generally the people who are there first are most likely to end up in leadership positions if the company succeeds.
• Technical challenges.  Emphasize the unique technical challenges that you need to execute on in order for your company to succeed.  Engineers want to stretch themselves and work on challenging tech problems. But be careful here not to oversell.  A lot of the challenge at an early stage startup is in figuring out what to build, not how to build for scale. You need engineers who are OK changing direction frequently and building something less than the perfect system.
• Vision and Impact. You have an opportunity to really sell the vision and mission of your company, probably in a way that a tech giant cannot at this point.

Branding & Marketing

You need to make your company as appealing as possible to an engineer who has never heard of you.  We underinvested in this at my last company and constantly ended up in a position where prospective engineers did not understand what the company did and why they should work there.

What this means in practice is that you need to invest in professional assets that:

• Clearly explain what your company does
• Explain the business opportunity
• Explain the mission – why does your company make the world better?
• Showcase the culture
• Talk about the tech you are building
• Spotlight the leadership
• Spotlight investors and strategic partners if you have any
• Make it clear in general why someone who has never heard of you should take a big risk to come join you

The assets you want in place are:

• A pro looking website.  It can be minimal, but should explain clearly the things above.  Here is a good example: https://www.cockroachlabs.com/careers/
• A “one-pager” that you can send someone as a pdf that explains the company, mission, opportunity, culture and tech.
• Bios of the co-founders and leadership.
• A great job description.

By professional, I mean that you should pay a designer to make them look legit.  You don’t need Red Antler, but you do need them to look good.

Writing a job description

At a minimum, the job description should succinctly describe what you are looking for and give a feel for the company culture and values.  Here’s an example from my last company.

---

We are looking for an outstanding software engineer to continue building and innovating on our customer-facing product.

This is an ideal opportunity for a full-stack engineer looking to work in a high-velocity, high-impact environment, owning the development process from design to implementation to launch.

YOU

• Write clear, self-documenting code
• Are a problem solver, a pragmatist, a fast-learner, not dogmatic
• Have experience building web and mobile apps (Javascript, Node, React a plus)
• Are confident with performance optimization, and identifying bottlenecks and scalability concerns
• Are familiar with a range of modern data stores (relational, document, key-value, search, etc.)
• Have a good background in algorithms, systems, and design
• Work well with a multidisciplinary team, and communicate actively
• Think independently and love sharing your technical knowledge with others
• Aren’t afraid of a little database administration or dev-ops work
• Probably have a technical degree and 1+ years of real-world development experience

WE

• Are a small, dedicated team of smart, experienced engineers, product folks, and creatives
• Love Github, Slack, Asana, AWS, Meteor, Node
• Hate useless meetings, value your time, and enjoy a good work/life balance
• Are data-driven, and take product ideas from the whole team
• Work quickly to test and launch new ideas
• Value design and beautiful user experiences

BENEFITS

• 100% covered Medical, Dental, and Vision Plans
• Health & wellness perks through HealthKick
• Flexible Working Hours
• 401K Plan
• Free Meals and Snacks with a Fully Stocked Fridge and Pantry
• Cold Brew Coffee in the Summer, Fresh Hot Coffee in the Winter, and Seltzer on Tap Year Round
• Subsidized Commuter Benefits for MTA & CitiBike
• Monthly Team Outings
• 20 Paid Vacation Days & Unlimited Sick Days
• Office Located in Little Italy

Expect a competitive application process that will include an initial coding assignment, phone screen, and in-person interview. Thank you for your interest. We look forward to hearing from you!

We are an equal opportunity employer and value diversity in our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

---

However, the job description and company branding surrounding it present an opportunity to go above and beyond and let you stand out.

Take a look at what Vanta does in their description: https://www.keyvalues.com/vanta

The thing I love about this is it leads with the culture and mission is likely to be appealing to exactly the kind of engineer they want to work there – someone who is very customer-centric and willing to do more than just engineering to make the product succeed.  The effort they put into the job description shows how much they care about hiring the right people.

The Recruiting Workflow

Once you have your assets in place, you need to start running a recruiting workflow.  You’ll want to track this workflow using some sort of Applicant Tracking System (ATS).  

There are a bunch of ATS solutions out there, and I don’t have strong opinions on what is best.  We used a product called Lever, which I don’t actually recommend. But there are others and the important thing is that you have one central database that effectively de-dupes candidates and stores where they are in your recruiting funnel.  This could be a spreadsheet to start if you don’t have budget for a real system, but there are some free options out there which are better than sheets for this purpose and I recommend you use something with more structure.

The above chart shows high level what the recruiting workflow looks like.  On the far left are the various places candidates come from. As you move right, there are various steps for progressing them through your funnel.  Let’s start by looking at the sources.

Referrals

Referrals are the best source.  They can come from your own network, from employees, or from investors.  They are best because (a) you have an easy in for contacting the candidate, (b) they are pre-vetted, (c) the pre-existing relationship can help you close the candidate.

For employees, you should offer a referral bonus to incentivize them.  There are different schools of thought here – I’ve heard people say that employees should love the company enough that they want to tell their friends to come work there and that therefore referral bonuses should not be necessary or create a bad incentive.  

I don’t really buy this.  Recruiting is so hard and referrals are so valuable that I would do everything in your power to motivate employees to take the time to reach out to their talented friends and co-workers.  There’s a reason Google offers referral bonuses even though everyone in the world wants to work there already.

You should set up times to sit down with engineers on your team and go over their linkedin networks and see if they have folks who might make sense for them to reach out to.

For investors, you should make sure the assets I mentioned above are strong and then ask them to distribute those assets to their network.  Good investors will have in-house people focused on talent since it’s such a huge determinant of success. Form good relationships with those talent folks and make it easy for them to sell your company to people in their network.

Sourcing candidates & Cold Outreach

After referrals, your best option is sourcing candidates yourself and doing cold outreach.  

Sourcing candidates is basically a manual process.  It entails compiling a big list of potentially qualified engineers, designers, etc along with your best way to contact them.  You can sometimes bootstrap this process by getting an existing list from an investor or fellow entrepreneur. These lists are very valuable and you should ask around and try to get one.

Even if you get a list of engineers and their emails, you’ll need a way of expanding it.  One of the “secret” ways I had of doing this at my last startup was paying people remotely via Upwork to source candidates off of LinkedIn.  The way this works is you create a Google sheet with fields like:

• Name
• LinkedIn
• Email
• Years of Experience
• Company
• Undergrad School
• Grad School
• Highest Degree

And then give instructions to a remote worker on the qualifications you are looking for.  I’d provide them with a list of schools, companies, job titles, and so forth, and set them to work reading LinkedIn looking for possible candidates.  Email is generally not listed on LinkedIn, so they will need to use a tool like EmailHunter to guess the email address.

As this list builds up, you start emailing potential candidates.  The ethics of this are a bit gray. Technically, it is not spam in the legal sense because the email was not automatically scraped and you are not selling a product, but it always felt iffy to me to reach out and try to get in front of someone using this method.  Up to you whether you go this route, but at a startup you need to be aggressive and I think it’s basically on the right side of the line. A lot of engineers were flattered to get the outreach. Some were annoyed.

On a technical note, I would create a separate email address to do the outreach from.

You can also try contacting people directly through LinkedIn, but I found that (a) it was very expensive and (b) not nearly as effective as directly getting into someone’s inbox.

Recruiting Platforms

Another option is recruiting platforms like Hired and Vettery.  These platforms aggregate engineers who are looking for jobs and make them searchable to employers.  It’s generally free for employers to browse the platform; they only pay if they actually make the hire.  

The fee is steep though, usually somewhere between $15k-25k, or 15-20% of the first year’s salary.  You can negotiate discounts as a startup, and can also get discounts for bulk usage. But overall it’s a pricey solution.

It’s also a relatively low quality solution in my experience.  The main issue is with the quality of talent on these platforms.  As mentioned above, the best engineers and designers typically have jobs.  If they are looking for new jobs, they often have connections to go through or will target their searches.

I found that the people on these platforms tended to be B-level talent.  We interviewed many of them, but literally zero of them made it though our interview process on the engineering side.

Still, there are probably some strong candidates on there and if I were hiring again, I would definitely look.

Inbound candidates

As a rule, inbound candidates (people who found your company and applied directly) tend to be the lowest quality.  They can still be good, so it’s worth posting your job description widely and seeing what comes in, but expect a high signal to noise ratio.

External recruiters

Outside recruiters are another option.  They generally work on contingency so that you only pay if you make a hire through them.  If you run a startup long enough they will start contacting you with potential placements.

The upside is that they do the sourcing work for you and sometimes have quality candidates.  

The downside is cost – like the recruiting platforms they take up to 25% of first year salary, although you should always try to negotiate a discount.  

Internal recruiters

If you have raised enough money and are hiring enough engineers, you should hire an internal recruiter.  Typically they cost around $60k / year. So if you are hiring more than 5-6 engineers per year, it will be beneficial to have someone on staff who can fully manage the hiring process.  It’s much more bang for the buck than paying an external recruiter 20k per hire – a good internal recruiter should be able to get you ~1 engineer per month depending on who you are looking for, and they will save you personally a ton of time and overhead.

The internal recruiter should be responsible for the whole hiring process, from writing and reviewing the JD, to filtering inbound applications, managing outbound, setting up phone screens and interviews, corralling feedback, and so on.  All of this overhead adds up quickly, and if you don’t have someone else handling it, you will find yourself as a founder, cto, or vp eng spending a ton of time setting up meetings and responding to emails. Far better to use your time on higher leverage activities like pitching high quality candidates and closing.

The Pitch

You need to practice a version of the pitch that is specific to engineers.  It’s a different pitch than for investors or random people you network with.

The specifics of the pitch are going to depend on your business, but some things that I like to hit on are:

• Mission – how is your company making the world better
• Culture and Values – what do you care about, what type of company is it
• Business Opportunity – how will the company grow and make money
• Traction – it’s working…
• Leadership – why are you and your cofounder and other leaders great
• Tech challenges – what are the most interesting things to work on
• Product process – here’s how we develop new products
• Team – the other engineers on the team are great because of X, Y, Z
• Customers – our customers are X, Y, Z and they need us because of Y
• Investors and Funding – We raised Xm from these great investors
• Product roadmap – here’s what we plan on building over the next year
• Personal growth and mentorship – if you work here, you will have the opportunity to grow in X, Y, Z ways
• Startups are great – emphasize the benefits of startups over big companies – velocity, culture, ownership

You should practice this pitch.  You’ll find that engineers ask the same questions over and over again, and every time they ask, you should make a note and try to adjust the pitch to explain again and again.

For my last company SelfMade, the pitch I gave went something like this:

• Mission: “We are trying to help small businesses grow”
• Details: “We do that by making them have awesome instagrams that drive traffic and conversions”
• Background: “The company is series A, we’ve raised 11.5m from great investors, we are based in NYC, have been around for 3 years”
• Opportunity: The opportunity is really big and growing – every month 20k new Shopify stores start.  Amazon does half its business through its marketplace. Instagram is THE channel that all of these businesses need to figure out, and we are helping them.
• Traction and growth: We have 750 paying customers and are doing about $5mm a year in revenue, up from $1mm at the start of the year
• Me: I was a principal engineer at Google. At SelfMade, I manage product and engineering.  My co-founder is a second time entrepreneur with an exit under his belt at a company you may have heard of.
• Product: The product is a platform for producing Instagram feeds and ads at scale.  Explain how it works….
• Tech challenges: The challenges are in optimizing the efficiency and quality of the content we produce.  We use ML for X, Y, Z
• Product roadmap: We are expanding to support paid media in addition to organic instagram content.  We are also expanding to support more platforms and higher end customers.
• Investors: We’ve raised 11.5mm in funding from top tier investors in NYC and Silicon Valley.
• Personal growth: I’ve mentored a lot of engineers, and would help mentor you.  You will have an opportunity to own major parts of the applicationOur startup is great: you’ll have ownership, be able to move quickly, meet new friends, and so on

Initial phone screens

If you have someone who is interested, the first step after email is to set up a brief phone call to pitch the company and hear about their background.

If you have an internal recruiter, they should set this call up and take it for most candidates.  The exception is if you have a particularly promising candidate who has come recommended, then you should consider taking the call and doing the pitch yourself.  And if you don’t have an internal recruiter, then, well, you’re taking the call.

The call should take 15-30 minutes max, and should cover:

• Intro
• Candidate’s background
• Candidate’s interests and desired role
• Company background and pitch
• Role description and pitchNext steps

I always have the candidates give me their background before giving the company pitch because it lets me tailor the pitch to what I think will resonate with them.  E.g. if they are interested in frontend, I pitch the opportunities to work on app frontend. If they are looking for mentorship, I talk more about my background managing engineers.  And so on.

The goals of this call are twofold:

1. Determine if you want to continue the process with the candidate.
2. If yes, then get the candidate to commit to the next step.

This call should be high-level and non-technical.  It’s purely a screen on

• Basic intelligence – is the person smart and articulate
• Basic background – do they have roughly the right background for the roleBasic culture fit – did they do anything that would make you think they are obviously not going to fit at your company

If all of these things are a “yes,” then you should try to get them to commit to moving forward on the call.  

If they have questions or concerns and seem high quality, you can offer to connect them with another leader or engineer at your company for more color.

Coding tests (optional but recommended)

The next step I recommend is a short online coding test.  We used a platform called Codility, but there are a number of platforms out there that work for this.

The idea is to make sure the candidate can code well enough that they won’t totally flail in an onsite interview.  This is not the place to ask hard coding questions – it’s just a sanity check. It should take the candidate no more than 1-1.5 hours.

That said, even on what I judged to be relatively easy online coding tests, this would filter out at least 50% of applicants who couldn’t get the coding test half right.   

We probably ended up eliminating some qualified candidates with this filter, but I don’t think the false positive rate was very high.  Programming is hard, and there are a lot of people who can sort of do it, but not really do it, and if you want a great team you need to aggressively weed out people who are not strong.  That said, you should take the tests yourself and make sure you can do them. If you can’t do them, they are too hard.

The real downside to this filter is not that qualified candidates don’t do well on the test, but that qualified candidates might not want to take the test at all.  It takes time, can be stressful, and, if they don’t do great, it hurts the ego. You don’t want qualified candidates dropping out of the funnel here, so if you see someone who has a great resume (top school or top company) who was good on the initial screen, then I would definitely skip this step.

That said, try hard to avoid having unqualified candidates in for onsites.  When someone comes in and starts bombing questions it puts both you and the candidate in an awkward position.  You either have to ask them to leave early or else waste a lot of valuable engineering time interviewing them. Being asked to leave an interview early is a bad experience for candidates; keeping unqualified candidates around for full interviews is a waste of time for your engineers.  So weed people out as early as you can.

Technical phone screens (optional but recommended)

Another possible (and recommended) step is a technical phone screen with an engineer.  This can be done in addition to or in lieu of the coding test.

The format is a 45-60 minute phone call or video hangout where an engineer on your team poses one or two coding questions to the candidate and has them work through them using google docs or some other collaborative editor.

Some advantages compared to the coding test are (1) that you can learn more about how someone thinks by observing them work through a problem (2) it creates a chance for the candidate and engineer to have a personal interaction and (3) can give another person on the team a chance to evaluate intelligence and culture fit in a way that you don’t get from someone taking a coding test.  It also makes cheating impossible (and yes, people cheat on the coding tests fairly often).

The disadvantage is mostly that it takes time from an engineer, and, if you are in rapid hiring mode and have a relatively small group of engineers to conduct the calls, these interrupts can add up to a lot of lost productivity.

Overall though I recommend this step for getting the most qualified candidates into the onsite.

On-site interviews

The onsite interview is the most important part of the process.

For a full-time engineering or product hire, you should plan on having 5-6 separate sessions with the candidate, plus some more informal time where the candidate gets to have lunch, meet some non-technical folks, and get a feel for the office.

Most of the engineering interviews should cover a mix of:

1. Coding
2. Algorithms and runtime analysis
3. System design

And there can be a secondary focus on

1. Some particular technology (e.g. iOS or React)
2. Live coding or debugging
3. Product & user focus

The engineers giving the interviews should coordinate beforehand to make sure there is enough coverage (e.g. at least one systems focused question instead of all coding).

An interview session should last about 45 minutes:

• 5 minutes on introductions, getting the candidate comfortable, explaining the format
• 35 minutes on one main question
• 5 minute wrap up at the end where the candidate has an opportunity to ask questions

Everyone on the team should be able to conduct an interview, but new hires will need time to shadow experienced interviewers before jumping in on their own.  I also encourage people to develop their own questions.

A good interview question

• Should
  • be easy enough that it can be solved in 35 minutes
  • be non-trivial in the sense that candidates will need to spend at least a few minutes thinking about possible solutions before diving in
  • lend itself to different possible solutions that have different performance characteristics (often there is a “brute force” solution and then a more efficient solution)
  • be “extensible” so that if a candidate gets the basic solution quickly you can make it harder by asking variations of it
  • Should be “hintable” so that if a candidate gets stuck early on you can nudge in the right direction without fully giving it away
• Should not
  • require any specialized knowledge beyond basic coding and algorithms (unless you are interviewing for a specialized role)
  • be an “aha” type puzzle – no questions where you need a flash of insight to get it right (how does an insect get out of a blender, or why are manhole covers circular type questions)

The point of the interview, and I always stress this prior to starting with a candidate, is not just to see if the candidate can get to the correct answer.  It’s much more about seeing how the candidate approaches a problem. I want to see whether the candidate:

1. Takes the time to understand the question
2. Asks clarifying questions before starting to solve the problem
3. Writes down example inputs and their corresponding outputs to think through various cases
4. Breaks the problem down into simpler subparts
5. Starts with the simplest solution even if it’s slower (I always ask them to do this)
6. Takes instruction well when they are heading down the wrong path
7. Asks questions when they get stuck
8. Outlines the algorithm before writing the code
9. Writes code that is at least basically syntactically correct in the language of their choice
10. Describes in words what they are doing and how they are thinking
11. Proves to themselves and the interviewer that their proposed solution works
12. Can engage in a discussion about different possible solution strategies
13. Can describe the performance of their solution and think through possible optimizations
14. Can describe test cases they would write to validate their solution

Sample Interview Question - Pascal's Triangle

For instance, say the question is to write a function that determines the value at a particular position in Pascal’s Triangle.  

I would give the candidate a function signature like:

function pascalVal(row, col) {
  // your code here
}

The first thing I would want to see is if the candidate understands the question.  Good questions would be:

• What is the indexing of row and column (0-based)?
• What are some example solutions?
  • pascalVal(0, 0) = 1
  • pascalVal(2, 1) = 2
  • pascalVal(3, 2) = 3

And then I would want to see if they could get to either a recursive or dynamic programmimg solution.

• If they got the recursive solution,
  • What’s the runtime performance (2^n)
  • How could they improve that performance (memoizing, taking advantage of symmetry, etc)
  • Can they also write up the iterative solution
• If they got the dynamic solution
  • What’s the runtime performance (n^2)
  • How could they improve memory usage
  • Can they also write up the recursive solution

There are various points in this question where candidates get stuck, and I have moves at each of them to help them keep going.

The goal of the interview is that the engineer is able to fill out a simple rubric about the candidate:

1. What was the question?
2. What was the code the candidate produced?
3. How would you rate their coding (if applicable)?
4. How would you rate their problem solving?
5. How would you rate their design (if applicable)?
6. How would you rate their knowledge and experience?
7. Are they a culture fit? Why or why not?
8. Would you hire them (1. strong no, 2. weak no, 3. weak yes, 4. strong yes)

Interview etiquette

A few notes on interview etiquette:

• Whiteboard interviews can be very intense for candidates and it’s important that the interviewer be mindful and understanding.  There is nothing gained by increasing the pressure – it’s already a very contrived setup so you should try your hardest to be sympathetic and put the candidate at ease.
• Always emphasize that getting the solution is not everything and that you care just as much about how they think through the problem, the questions they ask, whether they would fit the culture and so on.
• Do not check your phone or email during an interview.
• Have someone on the team who is dedicated to making the experience good – this person would typically be the recruiter, but it also could be an engineer.  They should greet the candidate, make sure all the interviews are running on time, make sure the candidate has everything they need.Remember that if you make an offer you will likely need to sell the candidate on joining, and how they were treated during the interview will make a difference.

Post interview

Hiring meetings

After everyone has submitted their feedback, you should get together as a group and make a decision.  This can be pretty informal – everyone goes around the room reading their feedback and making an argument one way or the other.  

A note on bias here: everyone needs to submit their feedback independently before the hiring meeting.  If you don’t enforce this, then people end up biasing each other depending on who speaks first.  It’s ok to be swayed if someone addresses your concern in the hiring meeting, but if more than a couple of the interviewers are changing their minds I tend to want to pass.

Typically I require a unanimous “yes” in order to move forward, which effectively gives every interviewer a veto.  This undoubtedly leads to false negatives – people we should have made offers to that we ended up passing on. However, the cost of a false positive (a bad hire) is much higher than a false negative, so I’m ok with this bias.  A bad hire ends up wasting a lot of management time, money, opportunity cost in terms of hiring someone better, and so on. Keep your hiring bar high.

Reference checks

The final step before making an offer is checking references.  Ninety percent of the time the references will reinforce your existing opinion and make you more confident in the offer.  But sometimes they will give you pause.

Even though it takes a bit of extra time, it’s worth it – there have been many times where I was going to make an offer, but the reference call made me change my mind (e.g. I’ve uncovered candidates lying about their previous positions; I’ve had people tell me in not so many words that the candidate was just fired, and so on).

I try to get three references from each candidate, where two of them had a supervisor relationship (not peers, friends or coworkers).

The reference call itself should take about 10 minutes.  I prefer a call to an email because it’s more likely to get honest answers and let you read between the lines.  But if you can’t get a call, then use an email – it’s way better than nothing.

Here’s the call template I use:

---

Intro – “Hi I’m Zach, thanks for taking the call, it shouldn’t take more than 10 minutes…”

• Company overview – “We do X, Y, Z.  The company is X people”
• Role overview – “We are making X an offer for software engineer”
• Questions
  1. What is your relationship to X?
  2. Can you describe X’s role & responsibilities?
  3. Why did X leave his job? (optional)
  4. How would you rate X’s overall job (or school) performance?
  5. What percent is X in for programmers? Top 5%? Top 10%? Top 25%?
  6. For a specialist (e.g. an AI person)   How would you rate X’s expertise in the field?
  7. How would X be at translating research into engineering?
  8. How is X as a programmer vs. a researcher?
  9. Do you think X would do well at an early stage startup?
  10. How were X’s communication skills?
  11. How was X’s work ethic?
  12. How quickly did X work? Is X more of a pragmatist or perfectionist?
  13. If I were to be his manager, do you have any advice on how i could help X succeed?
  14. What could X improve on?
  15. How does X deal with conflicts?
  16. What was it like to work with X?
  17. Would you want to work with X again? If so, why? If not, why not?Is there anything else I should know about X?

---

Question 13 is bold because it’s often most telling.  It’s a way of asking about the candidate’s weaknesses without explicitly asking about them.  You need to keep in mind that the person you are getting the reference from most likely has been chosen by the candidate to say nice things about them – that’s the idea.

Still, people want to be honest and if you give them a way to give honest feedback which might be less positive without feeling like they are betraying the candidate, they will do it.  That’s why this is a good question – it says, tell me how I can help the candidate, but what it’s really asking is what does the candidate need help with.

Everyone has weaknesses, so hearing about areas for improvement about a candidate is not at all a disqualifier – it can help you manage them.  But, if enough of the references give the same negative feedback, you might consider whether you missed something at the interview and whether the candidate really is a good fit.

That said, the things that are most likely to make me rescind an offer at the reference stage have to do with a candidate not being honest.  For instance, if a candidate told me they had a certain role at a company and I find out the role was different, that’s a huge flag; I had that happen with someone pretending they were full time at google, and I only found out at the reference that he was a contractor (very big difference).

Offers & Closing

The offer should come via a phone call from you – it’s more personal and exciting than getting an email.

It should include salary, start date, title, equity, benefits and a ton of enthusiasm about the candidate.

Some notes…

Explain your equity grant.  

• Don’t count on candidates understanding startup equity – in the interest of being transparent you should be clear how the vesting works, what the strike price is, what the exercise period is, and what percentage of the company the equity amounts to.  
• Re: disclosing percentages vs number shares – just tell the candidate the percentage; trying to give a large sounding number of shares without explaining what it’s actually worth makes the offer harder to think about and is disingenuous.
• One thing that can be helpful is showing what the equity would be worth in dollar terms in different exit scenarios.

Have other folks reach out.

• It’s a good idea to have the folks who interviewed the candidate reach out with short notes saying how excited they are and offering to answer questions.
• Off for other people involved in the company (e.g. investors and advisors) to hop on a call and answer questions – having an outside perspective can be helpful for candidates

Give the candidate some time, but don’t keep it open ended.

• There should be a date by which you and the candidate agree they will give you an answer.  I think 2-4 weeks from the day you give the offer is fair.
• You want to give some time so the candidate gets to make an informed choice, doesn’t feel pressured, and, if they say “yes” it’s clear that the job is something they really want.
• On the flip side, if it starts to really drag out what is probably happening is that they don’t want to work there that badly and they are probably using your offer as leverage in the rest of their interview process to ratchet up what they can get.Having a lot of people that you’ve put the time into interviewing who are shopping your offer for something better is bad for morale at your company, and, if the candidate does eventually join, it can feel off for everyone (like they are attending their safety school).

On the other hand, I don’t think it pays to be too aggressive in the deadline.  Don’t give an exploding offer – this is a good way to end up with a bad fit candidate and mess up the reputation of your company.  You might be able to strong-arm some people this way, but it’s really a shitty thing to do from a cultural perspective.  Fundamentally the candidate has the leverage in these situations and using a pressure sales tactic to try and reverse that is bad.