Specification-based Testing: Understand the requirements

Understanding the requirements is an important part of testing your code. If you already know the business rules you can create tests to validate it.

Also, you can create some that will prevent unwanted behavior too.

When writing tests, consider the inputs, how they affect the code, and if the results meet your business rules. Do you have some doubts about it? What should be the result in some specific situation? Or does the code have some requirements to run? Or something like that?

If some questions don't have clear answers, it might be a good idea to ask stakeholders before testing and releasing the code.

Example:

I was creating a CHIP-8 emulator. To implement an emulator you have to follow the system specifications.

To exemplify let’s take a look at the ADD with carry instruction present in the Chip-8 CPU. Here’s the code that I wrote:

// 8XY4
// ADD
int registerxIndex = ((instruction & 0x0F00) >> 8);
int registeryIndex = ((instruction & 0x00F0) >> 4);

int firstToSum = registers[registerxIndex];
int secondRegisterToSum = registers[registeryIndex];

int registersSum = (firstToSum + secondRegisterToSum);
registers[registerxIndex] = registersSum & 0xFF;
if (registersSum > 255) registers[registers.length - 1] = 1;
else registers[registers.length - 1] = 0;
pc += 2;

This code is a result of following the specifications described below:

• The instruction will be a combination of 2 bytes fetched from RAM.
• The CPU has 16 registers, that can be indexed from 0 to 15.
• From the 16 bits, 8 will be used to identify the instruction (in this case the 8XY4 ADD instruction), the other ones will be 4 to identify the index of the register X in an array of registers and the other 4 to identify the index of register Y.
• After identifying the registers:
  • Get the 8-bit value present in the register X (being X a number between 0 and 14, from the 4-bit value that you extracted from the instruction, the 15th is to store the carry flag)
  • Get the 8-bit value present in the register Y (being X a number between 0 and 14)
• Then add the values, if the result is greater than 8 bits, then we should set the carry flag to 1 otherwise we set it to 0 (register 15 or F in hexadecimal).
• We pick only the resultant 8 bits from the result and store them in the X register.
• Update the program counter by 2.

By looking at this specification, we can start thinking on what are the test cases that will guarantee the proper behavior of the code.

Let’s think of some scenarios for these specs:

• Scenario 1 - Sum without carry:
  • Store a value in a register X (120 for example).
  • Store a value in a register Y (10 for example).
  • The two values sum must not be higher than 255.
  • Set the program counter to 0.
  • Check if the sum result value in the register X is correct (in this scenario the sum will be 130).
  • Check if the carry flag is 0.
  • Check if the register Y was not changed.
  • Check if the program counter was incremented by two.
• Scenario 2 - Sum with carry:
  • Store a value in a register X (235, for example).
  • Store a value in a register Y (30, for example).
  • Set the program counter to 0.
  • The two values sum must be higher than 255.
  • Check if the sum result value in the register X is correct (9 in this example).since we had an overflow of the 255 max value, we store only the 8-bit part, since 265 is a 9-bit number (100001001), we store only the 8-bit part in X which will be 00001001 = 9.
  • The 9th bit that overflows will be set in the carry flag.
  • Check if the carry flag is 1.
  • Check if the register Y was not changed.
  • Check if the program counter was incremented by two.
• Scenario 3 - The other registers cant affect X and Y:
  • Store 0 in X and Y.
  • Store values in registers that are not X and Y.
  • Set the program counter to 0.
  • Check if the X remains 0.
  • Check if Y remains 0.Check if the carry is 0.
  • Check if the program counter was updated by two.

Just by using the specification we already can create some test scenarios to guarantee the proper behavior of our code.

Write down your specification, and the scenarios you can come up with using the specs as your guide. It will clear your head and your tests will be way more focused on the business rules.

Also, if you have some doubts about the specs, you can always question and figure out what is missing before finishing development.

In this blog post, I won’t be writing the code for these test cases. Since this blog post is part of a series, I’ll be writing all test scenarios when we are going to discuss the devise test case topics.

Want to learn more about this topic?

In the following days, I’ll dive into each of those steps in more detail, follow me, and do not miss my next blog posts of this series when it comes out.

In my next blog post, we are going to discuss how you can explore the code, so you get a better understanding of the code, especially the code you did not write.

Stay tuned to learn more! Don't miss out!

Willian Moya (@WillianFMoya) / X (twitter.com)

Willian Ferreira Moya | LinkedIn