When you’re looking for a job as a software test engineer, you know you’ll run a gauntlet of questions before you get a handshake and a new job. What do you need to know?
When I want to know about software testing, I ask Angie Jones. Angie is the senior developer advocate here at Applitools. And, Angie is a test engineering rock star.
Check out Angie’s online presence at http://angiejones.tech. She has held leadership positions at well-known companies in test engineering. She knows functional test and test automation. And she knows how you can become a test engineer who knows the answers.
While Angie first shared her webinar, Ace Your Next Job Interview, with us in 2017, we have seen lots of open jobs in functional testing and Selenium testing. Demand grows for engineers with SDET (Software Development Engineer in Test) qualifications to help build working and maintainable software. Whether you are test automation engineer looking to interview with someone who advertises for a “Selenium testing” position, or you are looking for an SDET position, coding skills will come up on your interview.
We reviewed this article with Angie. She said the content is just as relevant today as it was two years ago. So, here is a summary of her webinar (which you can also watch below).
Test Automation Coding
If you take away nothing else from this column, take away this: automation engineering for functional tests requires coding skills, knowledge of test approaches, and a way to marry the two effectively. Automation engineers need to know how to build appropriate tests into the code being tested. As a result, you need to expect to be quizzed on both your test knowledge as well as your coding knowledge.
Angie’s point?
“The game has definitely changed. The automation engineer interview might be the most difficult one because not only do they ask us automation questions but they’re also asking us testing questions and developer questions. Each one of those may have its own round of interviews.”
Effectively, people are looking for someone with a tester’s mindset, and a developer’s skill set. This could be an SDET position, or just an automation engineer with code analysis tools.
Let’s review the ideas Angie shared, so you can feel more prepared. She broke the interview into two parts: testing questions, and developer questions.
Ace The Testing Questions
If you’re interviewing for a functional testing role, you need to ace the functional testing questions. Savvy development teams may implement a “shift left” strategy and move a percentage of testing to the developers themselves. You must expect to answer the testing questions from both your developer interviewers as well as your test interviewer.
The classic interview questions for functional testing asks you to develop a test approach for an everyday object. Like, say, a chair.
How Would You Test A Chair?
SDET, Automation Engineer or Manual Tester – stop and consider this question.
If you haven’t come across this question – or one like it – remember two concepts: assumptions and limits. Your assumptions will dictate how you approach the problem. Your test cases will be influenced by both your assumptions and what you think about limits.
Let me be more concrete. When you think about testing anything, do you begin with test strategies? Test cases? What have you missed?
Validate your assumptions by asking questions. If you only assume, you can miss some of the important design considerations that will drive your test strategy.
Here are some questions that might help:
- Who is the user?
- What are the purposes for which the chair was designed?
- Was there a weight/size/height/age assumption?
- Was the chair designed for a person with specific abilities, or specific disabilities?
You might think your chair tests are fine until you consider a set of chairs like this one:
Yes, each chair supports sitting. But one is clearly mobile, one or two may be too awkward or too heavy for an individual to move, and at least one may have a weight limit.
Each has its own set of specifications and use cases. And, you might think that this set is sufficient to broaden your definition until you come across the chair that doubles as a stepladder. In this last example, if you do not consider the stepladder use consideration, you will miss a set of important functional tests.
You want to make sure that you develop use cases from the design and intended use – and not simply what you thought would be important. For instance, “How high can a person stand on the chair?” is not a concept for the one on casters.
Once you know the intended use cases, you can consider tests to run and appropriate limits for those tests. So remember – ask questions.
The Automation Round
The most common mistake for engineers pursuing test automation jobs, Angie says, is people who prepare only on UI tests. Those candidates who think the job is Selenium Testing are stumped by any question falling outside UI testing.
Angie suggests that everyone know and understand the test automation pyramid, first introduced by Mike Cohn:
In this pyramid, the bulk of automation involves unit tests. Services tests, which validate all but the behavior of the UI, are the next largest chunk. UI tests are the smallest volume of automation tests, as they are both complex and costly in terms of engineering time to ensure both behavior validation and test coverage.
If you want to be prepared for discussing automation, you need to know how to handle unit tests as well as service and business logic tests.
If you are unfamiliar with unit testing, you can take the Unit Testing course on Test Automation University.
Let’s start with unit tests. Imagine you had the following method to automate:
public int add (int a, int b);
Language aside, let’s think about what is needed to test this behavior.
You may be thinking, “Oh crap, I don’t ever do unit tests.”
Take a second to understand the inputs and outputs. Here, this method takes two integers and it returns an integer. Given the name of the method, add, you might assume that it’ll add the numbers and return the sum, but it doesn’t hurt to ask a clarifying question such as:
“I assume this method adds the two parameters and returns the sum, right?”
just to make sure is not a trick question. In fact, if there’s anything that’s not perfectly clear or obvious to you, ask about it.
If you find yourself becoming self-conscious and you think you’re asking too many dumb questions then just remind the interviewer (and yourself) that you’re a tester and you’re trained to challenge assumptions. Your interviewer will appreciate that.
Handling Unit Test Questions
So for this question, Angie recommends listing every test you can come up with. You don’t see the body of this method, so don’t assume what’s inside of it. Don’t assume everything works beautifully. In fact, assume this method will break in normal operation – so think of normal operations you might run.
Angie listed the following tests from the top of her head:
- a and b both positive
- a and b both negative
- a positive and b negative
- a negative and b positive
- a being zero
- b being zero
- both a and b being zero
- the sum of a and b exceeding the memory allocated
- for an integer
The more tests you can consider the better you will do here.
While this question is gauging your testing abilities, it’s also testing how much you understand about code. You need to understand the difference between valid UI tests and valid unit tests.
Avoid making common mistake UI testers make in suggesting impossible coded tests – something like:
“Oh, I want to pass a String as one of the arguments.”
That’s a very typical UI test, to see how the UI behaves when a user enters a String in a number field. However, from a unit test perspective, your tests are directly calling into these methods in code. If you were to try to pass a String as an argument, your test wouldn’t even compile.
Know the difference between valid and invalid unit tests. If you suggest an invalid unit test, it sends a signal to your interviewer that maybe you don’t understand code.
Service Layer Tests
Automation testers must understand how web services behave. An SDET is expected to understand service calls, service responses, and hot to test service behavior. Increasingly, Selenium tester interviewing for an automation engineering role needs to know this as well.
If you know UI but don’t know web services, dig in and learn (there are various sites to check out, including the course “Exploring Service APIs through Test Automation” on Test Automation University). Otherwise, spend a little time brushing up. Angie says interviewers consistently ask about web services.
Here is a sample question:
Given a user profile, how would you test CRUD operations of a REST API?
CRUD is an acronym for Create, Read, Update and Delete. These have equivalent commands REST API commands in Create = POST, Read = GET, Update = PUT, and Delete = DELETE.
In the question, you aren’t given any information about parameters. Should that stop you? Nope. You’ll want your interviewer to know that you can think abstractly – and then be more specific if you’re given more detailed parameters. So, you can ask:
“Do you have a spec, or would you like me to solve this in general terms?”
If they want you to solve this in general terms, you can lean on what you already know to answer this question.
Answering the Service Layer Question
Given that you have already created a user profile someplace you can leverage your experience to think about this problem in the abstract. Once you consider that most service calls have both required fields and optional fields, you can think about passing different parameters. Let’s start with the “Create”, or REST POST method.
The four POST tests Angie considered included:
- POST with all required and optional data
- POST with required data only
- POST with required data missing
- POST with invalid data for parameters
You might be able to come up with more than these four basic scenarios, but as a starting point, these are sufficient. It shows you are thinking about normal and abnormal inputs as ways to validate output. If you knew more about the API, you could be more specific.
You can also point out that successful POST calls will result in a status code of 201 for successful creation. You can mention that if you knew more about the API, you could validate more details within the body and header. And this is typically in JSON, XML or plain text format.
Angie put together the tests she considered as basic for all four of the commands:
These include error paths as well as happy paths, multiple calls, etc. You can also mention appropriate response codes and checking the body values if you have them. For instance, 200 is the response code for a successful GET, PUT, and DELETE.
Remember to think broadly and abstractly for tests to consider.
If these don’t seem obvious to you, it’s worth studying service calls some more. There are some great courses on Test Automation University. Check out:
UI Automation
So, it may seem like the most trivial UI question you’ll get is something like: use Selenium API to log into an app. It’s a pretty overused question that most interviewers skip by these days.
Overtime, interviewers have concluded this question exposes little about the prospect’s knowledge of practical UI testing techniques. If you hire someone just on this type of simple question, you may discover your employee can do rote work, but not the analytical thinking needed to write real tests.
Instead, Angie suggests that you expect something like what she uses in interviews – showing a real UI and asking you, the candidate, to use the Page Object Model as a reference for creating an approach to creating UI tests. Something like this:
This is a Twitter profile on mobile. Given this page, and the Page Object Model pattern, how would you build the test framework?
This kind of question can expose whether you understand how to approach testing a real-world application.
To answer this question, look at the screen and determine which elements your tests might need to interact with and/or verify.
So, on this page, there’s a
- banner header
- a profile photo
- name
- handle
- bio
- location
- link
- number following
- number of followers
- an Edit Profile button
- Tweet button, and a
- navigation footer
There are also four tabs, and each will have it’s own content and interactions.
Thinking Through a User Interface
The key is thinking about how to organize this into a Profile Page class. This all seems straightforward to organize, but Angie points out that this example can separate the great candidates from others. For instance, if you realize that the “Edit Profile” button only exists for the logged-in user, and it won’t show up on anyone else’s profile – that’s great. More importantly, how do you design your page object class for this case? Do you include the Edit Profile button in the Profile Page class, knowing that sometimes it’s visible, and sometimes it’s not – or do you create a base profile page class that has common elements and subclass it with a My Profile vs. Other People’s profile? Whichever way you go, you need to justify your answer.
Also, think about the tabs. There are Tweets, Tweets, and Replies, Media and Likes. How would you handle that? That might be a question for you.
In fact, your interviewer might wait for you to see these tabs and come up with an approach. If not, the interviewer would possibly point them out to you as a hint (and ding you a little on the interview) before asking you how you would deal with these.
Just like with the Edit Profile issue, your approach to these could be to make these part of the Profile Page class or to make each their own class accessible from the Profile Page Class. It’s up to you – you just have to be ready to justify your approach.
Finally, there’s the navigation bar at the bottom. What do you know about it? If it turns out it’s on the bottom of every page, would you create separate calls for each page of your app? Or, would you make a base page class that all pages inherit from which includes the navigation bar? Or, would you do something totally different? Again, you need to justify your answer and be able to discuss the tradeoffs and pros and cons of your choice.
Angie’s Approach to the UI Question
Angie says her design would include:
- a base page class that includes all the elements that appear on all pages
- the navigation bar at the bottom with appropriate methods
- any other elements that show up on every page.
This approach allows code re-use and avoids code duplication (which makes the code easier to maintain if functions and layout change between releases).
For each of the tabs, Angie says she would create separate classes and link to them via “click” methods from the Profile page class. The result is a small Profile page class that is easy to maintain if the tabs ever change.
The tabs themselves are components that have their own web elements and corresponding methods. If this were all one page, it would become a mess to manage, as one would have to keep track of which tab were active and which methods were appropriate at that time.
Finally, with the Edit Profile button, Angie recommends considering the profile page of the user vs. the profile page of another user asking, “Is the Edit Profile button the only difference?” This might be a question for the interviewer. As it turns out, the Edit Profile button only appears on my page, but it turns out that there are lots of buttons on another user’s page – like “Follow”, “Unfollow”, “Block”, “Mute”, etc. This makes the case for having a base profile page that contains all the common profile page elements, as well as two more profile pages classes for “My Profile” and “Other Profiles”. Each would contain what is specific to those pages.
Comfort With the DOM
Another set of interview questions determines whether someone is comfortable with the DOM. It’s such an important part of testing, and yet so many people rely on the locators recommended by their browser’s Developer Tools. This results in flaky tests.
Let’s look at this example.
How do you build a test that makes a selection and records a vote? Your test should be generic enough that it can select either choice and record a vote.
As an interviewer, Angie shared, she would start with this UI, and then share the HTML that creates this UI:
From her observations, many test engineers freak out when they see the HTML, because they don’t understand how to automate HTML tests.
Angie finds it disappointing when people say, “Well, I just use Firebug.” Because, when there are no locators – yes, Firebug can build a test, but it will be extremely brittle.
This is what Firebug gives for the second poll option:
/html/body/div/div/div/div/div/div[2]/label/span[2]
Yes, that’s accurate for this build of the app – and it’s brittle. You don’t have any contextual reference. You don’t have a way to link this to the app itself. You don’t link to any reference in the code. You can’t automate this – this is the link to the second option explicitly and directly from the HTML root. If the app changes, this code will break – and you may not know why. You’re going to have to recode this manually all the time. Yuck.
Angie wrote a whole blog post about the perils of relying on recommended locators from Developer Tools.
In going back to the DOM, there’s a span element for “Yes” and a separate span element for “No”. Should you use these? Again, the answer here should depend on how comfortable you are with the idea that “Yes” and “No” are only used in the HTML in these locations on this page. How comfortable are you with that likelihood (HINT: Not very!).
Going up one level for each, there is a call to a radio class for the radio button value. It turns out you can create a CSS selector which allows you to uniquely call one or the other:
.//span[contains(@class, ‘PollXChoice-choice–radio’)]/span[text()=‘%s’]
If you want to know about how to select web element locators like the CSS selector here, there’s a whole course on Test Automation University.
Development Round
Development questions are the scariest ones for test engineers. Angie says that she thinks they’re scary, too. Engineering management teams are requiring more development-level tests because of the frequency of test automation project failures. They want their test engineers to be top-notch coders.
You may get as many as five of these questions during your interview.
Here are three things to brush up on:
- Big O Notation
- Data Structures
- Algorithms
Big O Notation
This is huge in coding interviews. It’s a measurement of the performance of the algorithm (by execution time) based on the algorithm’s design.
The point is to know the relationship between the algorithm design and performance, related to the size of the input set. An algorithm with a O(1) is excellent – performance is the same regardless of the input set. An algorithm with O(2^N) is horrible – something that runs recursively with multiple calls. If this is familiar, awesome. If not, there is a little more detail in this article.
Data Structures
The next thing to worry about is data structures. There are four common data structures you will encounter:
- Hash Table – by far the most common in tech interview questions.
- Stack – shows up in some questions. Stack is a last-in/first-out data structure.
- Queue – shows up in some questions. Queue is a first-in/first-out data structure.
- Linked Lists – doesn’t show up in very many questions. – Each element points to the next element.
Angie says she encountered questions about hashtables frequently, and stacks and queues as well. Linked lists weren’t ones she encountered often, but they did come up.
You can find a more detailed overview of these and other data structures here.
Sample Questions
Angie pointed out that there are two kinds of test approaches she encountered during her interview experiences. Sometimes, she was asked to use a coding site like hackerrank.com. Other times, she would be asked to enter code using an online text editor like Google Docs. In still other cases, she would be asked to code on a whiteboard. Angie suggests you ask your recruiter or the hiring manager what to expect for doing coding examples.
During her interviews, she mentioned being in a room with developers and automation engineers who were evaluating her answers.
Here are some tips to make things go smoothly.
1) Very rarely will you get written questions. Most of the time, the questions are spoken. Listen and write down what you think people are asking so they can see, and then clarify.
2) Write down an example to code against. This will help you structure your answer.
Sample Question 1
Angie was given the following question:
“You’re given two arrays and you’re asked to print out any characters that appear in both arrays”.
Using Tip #1, Angie reminds us to think – what does this question mean? Make sure to ask clarifying questions if you’re not sure. So, she asked:
“The array is an array with each element being a single character, correct?” This validates that the array wasn’t an array of strings of arbitrary length, which might make the coding more challenging.
What other questions might you ask?
Approaching Sample Question 1
Using Tip #2, Angie reminds us to think about coding cases – simple cases to define correct behavior, and exception cases that one would expect to encounter.
For the simplest cases in Angie’s example, she wrote down a simple set of arrays:
a1={‘a’,’b’,’c’}
a2={‘b’,’c’,’d’}
She said she normally uses a more complex example to help push her coding approach. As she said,
“If you code for the simplest examples, you may miss the tricky examples. I try to put the tricky one up front to force me to think it through.”
She then updated her array example to have an array with duplicate elements:
a1={‘c’,‘a’,’b’,’c’}
a2={‘b’,’c’,’d’}
Another suggestion is – once you have your example, write out what your expected result would be for the example before you code – it’s a way to check your work. Here, the answer should be:
Expected Result: {‘c’,’b’}
Now you have a way to validate your work before you get started. What you don’t want to have is code that doesn’t result in your expected output – now you have to work backward to figure out what you did wrong, and it will leave you flustered.
Thinking Through Approaches
Take a minute to think through Big O notation, algorithms, and data structures to begin. Take enough time to pick an approach, but don’t take too long to start – your interviewers will think you don’t know what you are doing, and you will feel flustered. Pick a solution that comes to mind first.
Say something like, “Okay, let me get my thoughts out here and I can refine them later.” That seems to work well.
Angie took a moment to write out code related to her sample question. Her first approach was to write a nested loop, which advanced through array a1 and compared each element to the elements in a2 – printing the character if the element in a1 matched an element in a2. :
When she walked through this nested loop, she realized there was a problem, as the algorithm would match the ‘c’, then the ‘b’, and then the ‘c’ again. This is not the output she wants.
This is also an O(N^2) approach, which is pretty inefficient. There should be a better way.
Catching Any Mistakes First
The important point is for you to catch these issues before the interviewing team does. They want you to know the problems in your approach to guide you to a better outcome.
In this case, the issue of duplicate values – along with knowing that the O(N^2) algorithm is not the most performant, makes it clear that you know something useful. You know BigO complexity, and you know how to look at your solution and find cases to break it – meaning you understand testing as well. Also, it means you are not afraid to admit when you mess up.
Angie then thought through the best way to improve the solution. She decided that the best approach was to create a hashtable from array a1, then compare array a2 with the a1 hashtable. The code looks like this:
This is more code than the original solution and it looks ugly, but it ends up being an O(N) solution, which is really efficient. It also addresses the duplicate issue and the O(N^2) performance problem of the first solution. Therefore, it’s a superior answer.
As you walk through issues, your interviewers may be giving you hints or clues about your code. Pay attention to both their hints and their questions.
Sample Question 2
Angie dealt with the next question:
“Compare a String with a List of String elements and count the number of times the String is found in the list.”
Using Tip #1, she asked the questions that came to her mind to clarify the parameters of the problem. Then, she came up with an example and started thinking about code. She came up with this:
The solution here works, but everyone else will figure this approach out. How do you stand out as creative?
Standing Out From the Crowd
What happens if the list is null, or an entry is null? Won’t lines 3 or 4 generate an error? How do you handle that?
What’s more, if you know your language library, you can take advantage of built in methods. In this instance, there is a built-in Java library method that does this. The modified code looks like this:
Here, the code is much more efficient as it leverages the language library. It also shows that you have thought about the null list failure mode (and the element null issue is addressed in the built-in function).
Additional Resources
Slide Deck
https://slides.com/angiejones/technical-interviews#/
Angie’s Webinar Video on YouTube
Top comments (0)