We're heavy end-to-end monitoring users here at Checkly and always experiment with how to architect our tests the best way. Over the past months, we've settled on a few workflows that make it much easier to spin up new tests, avoid code duplication, and make the entire test setup easier to manage.
One of those strategies is to strictly separate concerns in our tests.
For example, instead of having one check that creates and deletes a resource in the UI, we've split these actions into multiple test runs. One browser check tests the resource creation, and another one the resource deletion. But then, if one test is restricted to resource creation, when do you delete and clean up the resource? And also, if a test only checks the deletion, where do you create the resource in the first place?
This situation is when working on the API level helps. We introduced a custom API client to prepare and tear down all the UI test cases using the underlying API. A few lines of code speeded up the process and made tests more reliable.
We successfully use API calls in our UI tests to execute any prerequisite and postrequisite actions. Below we'll go into more detail on how we split up a large Playwright test codebase and how an API client helped us get there!
Sounds intriguing? Read on!
Plain API calls in end-to-end browser tests
Previously, our Playwright UI tests would create a Checkly dashboard and delete it all in one browser session. This structure led to issues where errors in the "create" phase would immediately cause the test to fail and not execute the test's "delete" portion. We littered our test account with dummy dashboards. To solve this problem, we split our "create" and "delete" tests into separate check runs.
But by decoupling these functionalities, we couldn't rely on a particular setup or clean-up while running the tests. For example, the "delete" test won't garbage collect the "create" test, and the "create" test won't generate entities for the "delete" test to remove.
What if we could work on the API directly to ensure our browser checks are running correctly without leaving traces behind? We introduced a custom API client to stabilize our Playwright tests.
Below is an excerpt of our "Delete Dashboard'' check. The test imports checklyApi from our simpleChecklyApiClient snippet and uses the API client to create a Checkly dashboard so that we can test its deletion in the UI.
This way, the check focuses on one functionality in a nice and clean way!
// DELETE DASHBOARD
const { chromium } = require("playwright")
const { checklyApi } = require("./snippets/simpleChecklyApiClient")
let dashboardId
;(async () => {
try {
// Create a dashboard via API, to be deleted right after
dashboardId = await checklyApi.dashboards.create({
customUrl: "<https://customdashboard.example.com>",
header: "Dashboard Creation E2E Test",
})
const browser = await chromium.launch()
const page = await browser.newPage()
// Try to delete the dashboard using Playwright within a browser session
// ...
// ...
} catch {
console.error("Error deleting dashboard")
} finally {
// If anything fails in that process,
// we explicitly delete the dashboard to ensure
// this Check has no left-over side effects
if (dashboardId) {
checklyApi.dashboards.remove(dashboardId)
}
}
})()
But what happens if the end-to-end test fails to delete the dashboard?
We introduced a finally block to ensure that even if something goes wrong, we'll always clean up the test run by attempting to delete the dashboard with our API client.
A custom Node.js API wrapper
The checklyApi is a JavaScript class wrapper around axios. It includes methods for HTTP verbs that are preconfigured with our baseUrl and an API key. The object also comes with additional convenience methods that target explicit resources in our API (i.e. /checks, /check-groups, etc.). The client is easy to use and tailored to our test scenarios.
// CHECKLY API HELPER
const axios = require("axios")
const API_ROOT = process.env.E2E_API_ROOT_PRODUCTION
const API_TOKEN = process.env.E2E_CHECKLY_API_KEY_PRODUCTION
const ACCOUNT_ID = process.env.E2E_CHECKLY_ACCOUNT_ID_PRODUCTION
const headers = {
Authorization: `Bearer ${API_TOKEN}`,
"X-Checkly-Account": ACCOUNT_ID,
}
class ChecklyApi {
#buildPath(url) {
return `${API_ROOT}${url}`
}
#requestMethodWithNoData(method, url, config) {
return axios({
headers,
...config,
method,
url: this.#buildPath(url),
})
}
#requestMethodWithData(method, url, data, config) {
return axios({
headers,
...config,
method,
url: this.#buildPath(url),
data,
})
}
async $get(url, config) {
return (await this.#requestMethodWithNoData("get", url, config)).data
}
async $post(url, data, config) {
return (await this.#requestMethodWithData("post", url, data, config)).data
}
// More convenience HTTP Methods
dashboards = {
create: (payload) => {
return this.$post(`/v1/dashboards`, payload)
},
remove: (dashboardId) => {
return this.$delete(`/v1/dashboards/${dashboardId}`)
},
}
}
module.exports = {
checklyApi: new ChecklyApi(),
}
Having all the API client logic centralized in this helper class allows us to make adjustments in a single place and reuse the client across all of our checks. Like in the "Delete Dashboard" example check above, the usage is as simple as:
checklyApi.dashboards.remove(dashboardId)
And for completeness, here is the opposite check "Create Dashboard":
// CREATE DASHBOARD
const { chromium } = require("playwright")
const { checklyApi } = require("./snippets/simpleChecklyApiClient")
const dashboardUrl = "https://testdashboard.example.com"
;(async () => {
let dashboardIdToDelete
try {
const browser = await chromium.launch()
const page = await browser.newPage()
// Navigate the UI and try to create a dashboard
// ...
// ...
// Retrieve check ID from the creation response triggered by UI interactions
const { dashboardId } = await (
await page.waitForResponse((response) => {
return (
response.url().endsWith("/dashboards") &&
response.request().method() === "POST" &&
response.status() === 200
)
})
).json()
dashboardIdToDelete = dashboardId
// Dashboard should be visible in the list
await page.locator(`a[href="${dashboardUrl}"]`).waitFor()
} catch (e) {
console.error(e)
} finally {
if (dashboardIdToDelete) {
// Housekeeping, delete the created dashboard via API
checklyApi.dashboards.remove(dashboardId)
}
}
})()
Both Playwright tests leverage plain API calls to set up and tear down, but there's a crucial difference.
How to extract resource ids using Playwright request interception
Combining API calls with end-to-end testing can sometimes be challenging. Setup steps are generally easier to realize than teardowns. But why's that?
When you create resources on the API level, you're in control of the data, requests and responses. But if you want to clean up and delete resources created in a browser session, this data might not be available to you.
This is when Playwright request interceptors shine!
// Retrieve check ID from the creation response
const { dashboardId } = await (
await page.waitForResponse((response) => {
return (
response.url().endsWith("/dashboards") &&
response.request().method() === "POST" &&
response.status() === 200
)
})
).json()
We run our UI tests as usual and listen to our app requests using waitForResponse to evaluate what resources need to be deleted in the teardown phase. By monitoring the network activity, we can snatch all the required resource ids and delete anything created in your Playwright test using the API.
There are no traces, no matter if the UI test fails or succeeds!
Wrapping Up
Writing fast and stable end-to-end tests can be a real challenge, but keeping tests lean and separating them into smaller chunks helps us to keep our test coverage high without slowing us down.
Our tests are now easier to debug, they share more code, and everything runs faster and in parallel. And even though making API calls in our end-to-end tests felt odd initially, we're not creating sequential test waterfalls anymore, thanks to a tiny Node.js API wrapper. Most of our Playwright tests are idempotent, so that we can run them continuously. And the best part: we're sure everything's cleaned up, no matter how many times we run our tests. This is how it should be!
This practice is only the tip of the iceberg. We're continuously improving our monitoring setup and will share more tricks here on the blog. If you want to learn more about end-to-end monitoring, we've plenty of other resources available. Until then, happy testing!
Top comments (0)