Integration tests with CDK and Python - a more real world app, less hello world

In the previous post I took the fastest path to building a CDK app with Python with a passing Integration Test. It showed how to create AWS services using CDK and then use integ-runner to automate the deployment of the services, use the services and assert that the result is as expected.

But, the CDK stack, the test runner, and the assertions were all written in one file.

In a traditional CDK we would have a more sensible application layout. We want to define our Constructs and Stacks in the main application, then import those into a test file. This allows us to have re-usable Constructs and Stack, which is a core benefit of CDK.

We want an application layout more like:

my-cdk-app/
├─ my_cdk_app/
│  ├─ my_stack.py
│  ├─ lambda/
│  │ ├─ index.py
├─ test/
│  ├─ integ_my_stack.py
├─ app.py
├─ cdk.json

Now we have a green, passing test, we will refactor the code into more of a real world CDK app than a "hello world" app. We will also cover some of the rough edges in using integ-runner - a developer preview tool, which very much feels developer preview.

You can see the full code for this post here

Test ... or tests

The cdk init command will bootstrap your CDK app, including a "unit test" inside the /tests directory.

But, out of the box, integ-runner expects your tests to live in the /test (singular) folder.

Let's first fix this small inconsistency, and (more importantly) introduce ourselves to how we can better configure our integration tests.

Create a new file called integ.config.json. This file contains the runtime options you want for your testing. Here's mine:

{
    "maxWorkers": 10,
    "parallelRegions": [
        "eu-west-1",
        "us-east-1",
    ],
    "language": "python",
    "inspect-failures": true,
    "update-on-failed": true,
    "directory": "tests"
}

You can see what each of these options does in the integ-runner docs. These are the command line options, made into JSON.

The important thing we will do here is change the value of the directory config to "tests" and then move all of the contents of the test directory in there. We will create a new folder in tests called integration to keep these tidy and separate from the unit tests.

mkdir tests/integration
mv test/* tests/integration/
rm -r test/

And, now, we can re-run integ-runner to check that we're still green.

While it took 3 or 4 minutes to run initially, when we re-run it here it'll pass in around 10 seconds.

We've moved the files around, but the deployed stack would have been the same as before, so it doesn't need to deploy the services in AWS.

You should see a successful test run output like:

Verifying integration test snapshots...

  UNCHANGED  integ_hello 13.031s

Snapshot Results: 

Tests:    1 passed, 1 total

Running integration tests for failed tests...

Running in parallel across regions: eu-west-1

Test Results: 

Tests:    0 passed, 0 total

Reusable constructs

Our test file also contains the Stack that we would want to use in our App.

We can't import the Stack from that integ_hello.py file into our main App. So, we can only deploy it through the test.

Not much use.

Let's create a new file in our main application directory touch integration_tests_with_cdk_and_python/hello_world_stack.py and add the stack definition to that:

# integration_tests_with_cdk_and_python/hello_world_stack.py

import os

from aws_cdk import (
    Stack,
    aws_lambda,
)
from constructs import Construct


class HelloWorldStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        self.function = aws_lambda.Function(
            self,
            "MyFunction",
            runtime=aws_lambda.Runtime.PYTHON_3_11,
            handler="index.handler",
            code=aws_lambda.Code.from_asset(f"{os.getcwd()}/lambda"),
        )

Note that I’m now defining the path to the lambda relative to the ‘cwd’ instead of relative to the test file; which we were doing previously.

The cwd is the Current Working Directory; which is where you run your scripts from, almost always the root of your app.

So, move the lambda code out of the test directory into the root of the app:

mv tests/integration/lambda/ .

Now, we should be able to import that into our integ_hello.py file:

# tests/integration/integ_hello.py

from integration_tests_with_cdk_and_python.hello_world_stack import HelloWorldStack

...

When we run integ-runner now, we get an error:

ModuleNotFoundError: No module named 'integration_tests_with_cdk_and_python'
  ERROR      integ_hello.py (undefined/eu-west-1) 5.054s
      Error during integration test: Error: Command exited with status 1

It can't find our Stack now that we've defined it in our main app.

I've raised this bug as an issue here, so if it's fixed there, then this section might not be a problem anymore.

When integ-runner runs, it can only see files in the same directory (or below) the directory where our test file lives. So, only files inside tests/integration/ for us. That's no use.

To make this easier to work around, I created a package called cdk-integ-runner-cwd-fix which we can use.

Add cdk-integ-runner-cwd-fix to your requirements-dex.txt file and re-run pip install -r requirements-dev.txt

So, now, our tests/integration/integ_hello.py file can be updated to:

1. Import the Stack from the main directory
2. Use cdk-integ-runner-cwd-fix to fix the bug so that import works
3. Remove the old Stack definition we had in the test file

# tests/integration/integ_hello.py

import aws_cdk as cdk
from aws_cdk import (
    integ_tests_alpha,
)
from cdk_integ_runner_cwd_fix import fix_cwd

# This needs to be before any import from your CDK app to work around a bug in the CDK
fix_cwd()

from integration_tests_with_cdk_and_python.hello_world_stack import HelloWorldStack

app = cdk.App()
stack = HelloWorldStack(app, "HelloTestStack")

integration_test = integ_tests_alpha.IntegTest(app, "Integ", test_cases=[stack])

function_invoke = integration_test.assertions.invoke_function(
    function_name=stack.function.function_name,
)

function_invoke.expect(
    integ_tests_alpha.ExpectedResult.object_like(
        {"StatusCode": 200, "Payload": '"Hello world"'}
    )
)

app.synth()

Now, you need to set CDK_INTEG_RUNNER_CWD to your project's root directory before running integ-runner:

CDK_INTEG_RUNNER_CWD=$(pwd) integ-runner

And, we should be back to a green test.

Conclusion

Now, we have a Stack defined in the main application, as we would usually write a CDK app. We can import that into a test file and run tests against it. We can import that into an app.py file and deploy it for real.

So, from here, you should be able to return to your usual CDK workflow, but use integration tests to check that your lambdas behave, and your S3 buckets can accept files, or IAM roles can perform certain tasks ... whatever AWS service you want to automate the deployment, execution and assertion of.