Types of tests - Software Testing (II)

Continuing with our Software Testing series, today we will take a short tour of the different types of tests that our code can perform. Here you can see the posts of this series:

Unit Tests

They are functional tests that allow the detection of problems in an individual module. It focuses its activity on exercising the logic of the module following the structure of the code (white-box technique) and the functions that the module must perform attending to the inputs and outputs (black-box technique).

The percentage of code tested by unit testing is called software coverage. In the next post we will be delving more into unit tests.

Integration Tests

They are functional tests that allow to detect problems between related modules and previously individually tested by unit tests. They take into account the assembly mechanisms of fixed modules in the program structure, that is, the interfaces between the components of the software architecture.

There are different types of integration tests depending on the order of integration. The order of integration chosen affects various factors, such as: the way of preparing cases, the necessary tools, the order of coding and testing the modules, the cost of debugging, and the cost of preparing the cases. The fundamental types of integration are as follows:

• Incremental integration: the following module to be tested is combined with the set of modules that have already been tested. There are two fundamental types:
  • Ascending (Bottom-up): it begins with the node modules in hierarchical tree.
  • Descending (Top-down): it begins with the root module.
• Non-incremental integration: Each module is tested separately and then all are integrated at once and the entire program is tested. It is also called Big-Bang because the number of modules grows instantaneously.

Ascending Incremental Integration

The characteristics of bottom-up integration are:

• You can work with the modules individually or combine the low-level modules in groups that perform some specific function in order to reduce the number of integration steps.
• A driver module is written for each group, which is a module written to allow simulating the call to the modules, entering test data through the input parameters, and collecting the results through the output parameters.
• Each group is tested with its driver module.
• Driver modules are removed from each group and replaced by higher level modules in the hierarchy.

For example, the stages of integration would be 6 in the case of having the following modules.

Stage 1, 2 and 3: Unit tests of E, C and F are carried out. The driver modules are represented with a dashed border in the following graph.

Stage 4 and 5: On the one hand, the B unit test and the B-E integration (or interface) test are carried out simultaneously, and on the other hand, the D unit test and the D-F interface, simultaneously. The modules already tested in previous stages are represented with a thick border in the following graphic.

Stage 6: Module A is incorporated and the complete program is tested, which does not require a driver module, since all the input and output management code for the program is present.

Descending Incremental Integration

The top-down incremental integration begins with the main module (higher level or root module) and gradually incorporates subordinate modules. There is no general rule for determining subordinate modules that should be incorporated first. As a recommendation, critical parts and input / output modules should be incorporated as soon as possible. There are two fundamental orders of downward integration:

• First in depth: branches of the modular tree are completed. In the example above, the sequence of modules would be A-B-E-C-D-F
• First in width: horizontal levels of modular hierarchy are being completed. In the example above, the sequence of modules would be A-B-C-D-E-F.

The characteristics of the top-down integration are:

• The root module is the first to be tested and simulators modules (stubs) are written, to simulate the presence of absent subordinates, who will be called by the root module.
• Once the root module has been tested, one of the stub module is replaced by the corresponding module in the chosen order, and new stub module are incorporated to collect the calls of the last incorporated one.
• The detailed process for the root module is repeated, that is to say, the corresponding tests are executed each time a new module is incorporated and at the end of each test, a stub module is replaced by its corresponding real one. It is a good idea to repeat some test cases from previous runs to ensure that no new defects were introduced.

Coding subordinate stubs modules is more complicated than creating drivers modules. The stubs modules must simulate that they take control of the call and that they do something with the parameters passed to them.

For example, using the same module design as in bottom-up integration, and using the first order in depth, 6 stages would be necessary.

Stage 1: Unit test of A is performed. The stubs modules are represented with a dashed border in the following graph:

Stages 2 and 3: The unit test of B and the integration (or interface) test A-B are carried out simultaneously, and then the unit test of E and the interface B-E are carried out simultaneously. The modules tested in previous stages are represented with a thick border in the following graph.

Stage 4: Test of module C and interface A-C is carried out simultaneously.

Steps 5 and 6: The unit test of module D and the A-D interface is performed simultaneously and then the unit test of module F and the D-F interface, leaving all the integration tested.

Non-Incremental Integration

Non-incremental integration is the only case where unit testing and integration are completely separate. The characteristics of non-incremental integration are:

• Each module requires to be tested:

  • From a driver module that transmits the test data to the module and displays the results of the test cases.
  • The stubs modules necessary to simulate the function of each module that are subordinate to the module that we are going to test.

• After testing each module separately, they are all assembled at once to form the complete program.

Comparison between integration types

The selection of an integration strategy depends on the characteristics of the software and sometimes on the project planning.

System or implantation tests

They are non-functional tests that verify that the complete system meets the functional requirements and specified technicians, and that it relates correctly to other systems. Ademais incorporates other tests such as:

• Overload or stress tests: They allow evaluating the behavior of the system when subjecting it to a limit situation such as excessive demand for requests, use of the maximum amount of memory, working with little memory, having many users performing the same operation at the same time, use the maximum volume of data.

• Compatibility tests: They allow you to test the system in different environments, media or operating systems and see if there are any errors in appearance or operation.

• Data access and security tests: They allow to test how the system responds to external attacks such as unauthorized breaches or camouflage of malicious code in a data entry.

• Recovery and fault tolerance tests: They allow to cause external anomalies such as electrical, device, external software or communication failures, and to see that the system recovers without data loss and without integrity failures and the time it takes to do so.

Validation or acceptance tests

They are non-functional tests that are used to check if the final product conforms to the initial requirements of the software and is based on the actions visible to the user and on the outputs of the software that the user can recognize.

There may be alpha tests and / or beta tests:

• Alpha: Associated with contracted software. They are performed by the client in a controlled environment under the supervision of the company that develops the software. The software developer will take note of possible errors and usage problems.
• Beta: Associated with software of general interest. They are performed by trusted users in their real work environment without direct control of the software company. The user will report the results of the tests.

Validation or acceptance tests

They are tests that are performed to avoid side effects. They are applied every time a change is made in the software to verify that unwanted behavior or errors do not appear in other modules or parts of the software.

Strategy for the application of tests in the classic life cycle

The application strategy and test planning aim to integrate test case design into a series of well-coordinated steps, by creating different test levels with different objectives. In general, the test strategy follows the steps shown in the following image during the software life cycle:

Documentation of test design

The documentation of the tests is necessary for a good organization of the tests, as well as to ensure their reuse. Documentation of test execution is essential for effectiveness in detecting and correcting defects, as well as recording results of test execution. The documents generated for each execution are:

• Test history: documents the relevant events that occurred during the execution of the tests.
• Incident Report: documents each incident that occurred in the test and that requires further investigation.

As we have seen, there are many types of tests that will allow us to correct errors and will make our software a better product. Do not miss the next post dedicated exclusively to unit testing. See you soon!