Java - tips for improving your Integration Tests

With the increased usage of CI/CD and agile development methodologies, it's more important than ever to ensure that the development pipeline is as complete as it can be. Most typical pipelines are comprised of a compilation stage, where code is compiled, then follows a stage of integration where BOTH unit tests and integration tests are executed to ensure that new changes don't break previous versions of the codebase. Typically, the pipeline can conclude with generation of reports or artifact releases to staging or production servers or containers, etc. We'll focus on integration tests.

Integration tests

Integration tests are important because they test at a different abstraction layer than unit tests. While unit tests test the correctness of a single piece of code given specific data and expected output, integration tests go a step higher in abstraction and can typical test code flows that deal with things like database connections or inter connected flows of checking a certain expected output after running several Spring CLI apps whose inputs depend on each other.

Java has several interesting features that allow us to check for things like database structure and execution of certain flows within an application. All of these are useful when writing integration tests, let's see.

Using class resources to "store" test data

A resource is data (images, audio, text, and so on) that a program needs to access in a way that is independent of the location of the program code.

This can be especially useful if the classes which integration you want to test depend on external data like input files or specific configuration arguments, which is typically the case for Spring CLI apps that add functionality to a certain codebase like populating databases or leveraging internal logic to get meaningful computations out of input data.

Accessing the class resources can be done using the methods in the classes Class and ClassLoader that provide a location-independent way to locate resources.

Typically, the idea is that when an access to a file is required, be it for configuration purposes to exercise some code flow, or as part of the whole idea of the integration test (i.e. "if this file is used to start this specific Spring CLI application, and it's output is required to another one, then, in the happy path, a certain outcome must be verified") we can leverage the methods from java.lang.Class to our advantage. Below follows the code of a very useful one getResourceAsStream:

public InputStream getResourceAsStream(String name) {
        name = resolveName(name);
        ClassLoader cl = getClassLoader0();
        if (cl==null) {
            // A system class.
            return ClassLoader.getSystemResourceAsStream(name);
        }
        return cl.getResourceAsStream(name);
    }

This method can be used to retrieve a specific file that either has been placed under the resources folder of a certain package or from the system class loader, that basically delegates to the VM built-in class loader.

With this, we can load files to use in our test code to kick off a certain flow, for example:

InputStream f = getClass().getResourceAsStream("/test.yml");
        File config = new File("test.yml");
        try {
            org.apache.commons.io.FileUtils.copyInputStreamToFile(f, config);
        } catch (IOException e) {
            e.printStackTrace();
        }

        SomeSpringApp.main(new String[]{config.toPath().toString(), "outputFile.txt"}

File output = new File("outputFile.txt");
//for a given input file, we always need to ensure we have an output file
assertTrue(output.length()>0); 

Like this, we can use files to emulate what would be a real-world use case of our application, instead of testing behaviour at the unit level. This is the essence of an integration test.

Using JDBC to access a database

The last useful tip I have, is that, if for some reason we have created a DB as part of our integration test, we might need to access it afterwards, namely, if we want to check if a certain table exists, or is created successfully. In order to do it, we need to first establish a connection, use the DB credentials to be able to query it, perform our actions and close a connection afterwards.
JDBC - Java Database Connectivity, is an API that provides a set of methods to access a database from Java code. Without further ado, here's the code to connect to a specific DB and check for the existence of a table:

        final String JDBC_DRIVER = "org.mariadb.jdbc.Driver";
        final String DB_URL = "jdbc:mysql://localhost/<db_name>";

        final String USER = "<some user>";
        final String PASS = "<pass>";

        Connection conn;
        Statement stmt;
        try {
            Class.forName(JDBC_DRIVER);
            LOG.info("Connecting to the selected database...");
            conn = DriverManager.getConnection(DB_URL, USER, PASS);
            LOG.info("Connected to database successfully...");
            stmt = conn.createStatement();

            DatabaseMetaData meta = conn.getMetaData();
            ResultSet res = meta.getTables(null, null, "<some_table_name>",
                new String[] {"TABLE"});
            assertNotNull(res);
            assertTrue(res.next());
            String sql = "DROP DATABASE <db_name>";
            stmt.executeUpdate(sql);
            LOG.info("Database deleted successfully...");
        } catch (SQLException e) {
            e.printStackTrace();
        }

Essentially, using JDBC we can establish a connection via a Driver, which is what will internally map a specific DB vendor specification to the internals implemented on the Java side, and using that connection we can create and statements and perform queries by calling the executeUpdate method and we also have access to a ResultSet that allows to explore information and structure about a certain query to a DB.
Like this, we can enrich our integration test with database integration and validation which can prove very useful to check code correctness (or sometimes simply integrity: the databases can be completely empty, but the simple fact that the test ran successfully and a certain table exists can be very meaningful information for certain code flows).

Conclusion

Using class resources and leveraging JDBC are two useful ways I've recently came across of writing complete and meaningful integration tests that can provide good feedback for certain code flows and help you and your team to develop, extend and modify specific code flows.