Tracetest Monitors: Synthetic Monitoring with OpenTelemetry and Playwright

I think we can both agree 😉 that ensuring the performance and reliability of web apps is crucial. At Tracetest, we understand this need and are committed to providing a robust testing tool to help our users maintain optimal system functionality.

But have you ever wondered if Tracetest itself needs rigorous testing to stay fully operational? 🤔 The answer is “Yes!” In this blog, I’ll guide you through how Tracetest uses its own tool 🤯 to deliver exceptional user experience through a combination of synthetic monitoring and end-to-end trace-based testing with Playwright.

Understanding Synthetic Monitoring

Synthetic monitoring, also known as proactive monitoring, uses automated scripts to simulate user interactions or API calls to an application. This approach helps identify performance issues, downtime, and other anomalies before they affect real users. Now, imagine the power of combining synthetics with trace-based testing. That's why we made trace-based synthetic monitoring a reality 🎉, allowing you to schedule test runs with Monitors without overloading your DevOps or Infrastructure team with excessive automation tasks.

Introducing the Tracetest Playwright Engine

We all know what Tracetest does, right… right!? 👀

In case you’ve been out of the loop, here’s a quick introduction. Tracetest is a synthetic monitoring and testing tool based on OpenTelemetry that allows you to test distributed applications. You can use data from distributed traces generated by OpenTelemetry to validate and assert the functionality of your systems.

On the other hand, Playwright is an open-source automation library developed by Microsoft. It’s known for its robust and reliable automation capabilities, making it an ideal choice for synthetic monitoring and end-to-end testing.

We've combined Tracetest with Playwright to bring a new level of sophistication to end-to-end testing. The Playwright Engine trigger in Tracetest allows for the execution of Playwright scripts, enabling detailed simulation of user interactions and providing full visibility of the entire user flow, from browser to back-end processes.

Testing Tracetest Using Tracetest

How does Tracetest ensure its functionality and identify issues before they impact real customers? The answer is straightforward: We test Tracetest using Tracetest.

There’s a lot of “test” in that sentence, but it’s simpler than it sounds. Let me walk you through one of our testing processes to validate a workflow.

For this demo, I’ll use the Create Environment use case. In Tracetest, users can create Organizations, and within these organizations, they can establish multiple Environments to organize teams and manage development, staging, and production ecosystems.

The first step is to select the trigger type for validating this scenario. I’ll choose the Playwright Engine trigger because it allows validating the entire flow, from the user signing into the application to creating a new environment. This trigger type enables us to use Playwright’s execution and assertions while also leveraging OpenTelemetry data generated by our front-end and back-end instrumentation. This approach is incredibly powerful because it lets you validate the functionality of other system components that may be broken, even when end-to-end tests appear to be passing.

Now, let’s write the JavaScript Playwright script to simulate user interactions.

const { expect } = require("@playwright/test");

async function createEnvironment(page) {
  // Validate page is loaded
  await expect(page.getByText("Continue With Github")).toBeVisible();

  // Authenticate with Github
  await page.getByText("Continue With Github").click();
  await page.getByLabel("Username or email address").fill("my-user-name");
  await page.getByLabel("Password").fill("super-secret-password");
  await page.locator('input[name="commit"]').click();

  // Validate post login page
  await expect(page).toHaveTitle(/Tracetest/);

  // Close onboarding modal
  await expect(page.getByText("Switch between envs here!")).toBeVisible();
  await page.getByText("Switch between envs here!").click();
  await page.getByLabel("close").click();

  // Select the default organization
  await page.getByTestId("organization-selector-button").click();
  await page.getByTestId("selector-item").getByText("default-org").click();

  // Select the create environment option
  await page.getByRole("link", { name: "Environments" }).click();
  await page.getByRole("button", { name: /create a new environment/i }).click();

  // Validate the create environment modal is visible
  await expect(page.getByRole("heading", { name: "Create a New Environment" })).toBeVisible();
  await expect(page.getByText("What are Environments?")).toBeVisible();

  // Create the environment
  const environmentName = "my demo env";
  await page.locator("#environment_name").fill(environmentName);
  await page.getByTestId("environment-modal-button-create").click();
  await expect(page.getByText(`Environment ${environmentName} created successfully`)).toBeVisible();
}

module.exports = { createEnvironment };

As you can see, this is a straightforward Playwright script without any additional third-party libraries or complex code.

The script simulates the following user flow:

1. Loads the app.tracetest.io app and signs in using GitHub.

1. Waits for the default organization and environment to be displayed.

1. Navigates to the organization management section and creates a new environment.

With the script ready, the next step is to create a test using Tracetest. To do this, add the target URL, upload the Playwright script file, select the method you want to execute from the script, and click the Run button to see the magic happen.

Once the run is successful and the output is green, it's time to add some trace-based tests 🥳. Navigate to the Test section to review the results and start creating test specifications.

As a side note, we can see the internal Tracetest traces and spans here because our environment is configured with an agent that has access to the Tracetest internal tracing backend. If you're interested in learning more, please check out our documentation on Agents and Connecting to Tracing Backends.

Without digging too deeply into the internal workings of Tracetest, I want to validate that the environment was successfully created in the database and that a tenant and token records were generated for this specific environment.

Additionally, I’ll add some performance testing to ensure that the processing time of all the database operations is less than 100ms. By navigating to the Automate tab in the Tracetest UI, I can access the test definition that includes the new test specifications.

type: Test
spec:
  id: create-environment
  name: Create a Tracetest Environment
  trigger:
    playwrightEngine:
      target: https://app.tracetest.io/
      script: ./script.js
      method: createEnvironment
  specs:
    - selector: span[tracetest.span.type="database" name="environments.insert"
        db.system="dbsystem" db.operation="insert"]
      name: "Environment created in DB"
      assertions:
        - attr:tracetest.selected_spans.count = 1
    - selector: span[tracetest.span.type="database" name="tenants.insert"
        db.system="dbsystem" db.operation="insert"]
      name: "Tenant created in DB"
      assertions:
        - attr:tracetest.selected_spans.count = 1
    - selector: span[tracetest.span.type="database" name="tokens.insert"
        db.system="dbsystem" db.operation="insert"]
      name: "Token created in DB"
      assertions:
        - attr:tracetest.selected_spans.count = 1
    - selector: span[tracetest.span.type="database"]
      name: "All Database Spans: Processing time is less than 100ms"
      assertions:
        - attr:tracetest.span.duration < 100ms

And, voila! We have achieved true end-to-end testing with full visibility of the entire user flow, from the browser to back-end processes. But wait, there’s still something missing—synthetic monitoring, remember? Let me enhance this test by enabling a synthetic monitor to proactively check and ensure the performance and reliability of this use case.

For this step, I need to create a native synthetic monitor in Tracetest. I’ll configure the monitor to run every hour and set up an alert to notify the team via Slack if anything goes wrong with the execution of the synthetic test.

Benefits of Synthetic Monitoring with Tracetest and Playwright

Integrating Tracetest with Playwright for synthetic monitoring offers numerous advantages. Here are the key benefits:

• Full End-to-End Testing: Combining Tracetest Monitors with our Playwright Engine trigger enables true end-to-end testing, covering the entire user journey from the browser to back-end processes, including asynchronous transactions.
• Proactive Issue Detection: Synthetic monitoring allows for the proactive detection of performance issues and anomalies before they impact actual users. By running regular tests, you can significantly reduce the MTTR for failing components.
• Automation: Setting up synthetic monitors with Tracetest is straightforward and efficient. Automated tests can be scheduled to run at regular intervals, ensuring continuous monitoring without manual intervention.
• Real-Time Alerts: Tracetest allows the configuration of alerts to notify teams in real-time if any issues are detected during the execution of synthetic tests.
• Improved User Experience: The combination of synthetic monitoring with Tracetest and Playwright helps ensure a superior user experience. By proactively validating critical functionalities, applications can meet and exceed user expectations.

Learn more about Synthetic Monitoring with Tracetest and Playwright

Synthetic monitoring is a vital component of modern web applications, providing proactive insights into performance and reliability. The integration of synthetic monitoring with the Playwright engine trigger offers a powerful solution for simulating complex user interactions and detecting issues with precision.

Would you like to learn more about the Playwright Engine trigger and Monitors in Tracetest? We’ve prepared a few step-by-step guides that you can follow to experience the power of Tracetest for yourself:

• True End-To-End Trace-Based Tests with the Tracetest Playwright Engine Trigger
• Synthetic Monitoring with Trace-based Playwright Tests

Last, but not least, do you want to learn more about Tracetest and what it brings to the table? Check the docs and try it out it today!

Also, please feel free to join our Slack community, give Tracetest a star on GitHub, or schedule a time to chat 1:1.