Using Playwright to Monitor Third-Party Resources That Could Impact User Experience

Today’s web consists of lots of 3rd party resources. Let it be your fonts, transformed and optimized media assets, or analytics and ad scripts, many sites out there include resources that they don’t own. Your website probably has a lot of those dependencies, too!

And while implementing third-party resources has downsides for performance and you should self-host your assets when possible, sometimes relying on external files is unavoidable. But be warned — using them can sometimes cause real headaches.

As Chris Coyier pointed out, external resources are often integrated too tightly into your own code, or implemented the wrong way, so that things can really go sideways. Ad blockers could block your external scripts or a service vendor could be down. Both situations can result in frontend downtime, poor UX, or entirely broken functionality on your end.

And while Chris asks if you could detect the cases of broken sites due to ad blockers with end-to-end testing, I think it’s more about all the 3rd party code running on the internet.

But heck yeah, testing and monitoring of failing external resources is possible!

Let’s have a look at an example that could be affected by 3rd party code and ways to monitor your stack with Playwright to guarantee that no external providers or ad blockers can mess with your business. 🫡

Examples of problematic external resource usage

But first, let’s look at patterns that can break your sites.

In my experience, there are two challenging 3rd party resource scenarios.

Slow and render-blocking resources

Years ago, I worked in an ecommerce startup and we were serving ad banners via an external provider. It was a quick solution to receive click statistics while providing marketers with a comfortable way to handle images on the site.

The banners were implemented with synchronous script elements above the fold.

<script type="text/javascript" src="http://some-provider.js"></script>

And we had plenty of these elements embedded in a slideshow. The scripts would load and replace containers with content coming from elsewhere. It worked beautifully… until it didn’t.

A synchronous script element stops the HTML parser and forces it to wait for the script to be downloaded and executed. The browser only continues parsing and rendering once this is done so that this script resulting in an embedded image can literally block anything on your site.

And that’s what exactly happened to me back in the day.

The script was loaded fairly quickly from Berlin, Germany and we didn’t notice any delays in rendering. But one day our third-party vendor experienced an outage. And interestingly, requests wouldn’t fail but just hang and time out eventually.

The result: a browser started rendering our site, stopped at the slideshow and waited for the scripts to time out. Our customers were looking at a white page until this happened and our site was entirely broken. A third-party vendor brought our frontend down because we implemented synchronous scripts. Be aware that every external dependency is another risk for your sites!

That’s why it’s generally better to self-host your assets and avoid critical render-blocking elements. No matter if you’re using ad tech, error monitoring scripts, or external stylesheets, make sure to double-check what happens when these resources fail. I doubt you want your system to be affected because of someone else's downtime.

That’s not the only situation where third-party resources can lead to trouble, though. Let’s come back to Chris’ example.

Tightly coupled JavaScript dependencies

Chris mentioned that he experiences broken sites on a regular basis. Some sites would fail in essential situations like making an order. The culprit: a thrown JavaScript exception.

That’s interesting because it’s unlikely that web developers aren’t testing their most important functionality. So what’s happening?

If you’re an ad blocker user there’s a high chance that this tiny browser extension is the reason. And that’s not the ad blocker’s fault. It’s doing what it’s supposed to do — blocking trackers (most likely third-party scripts). But the issue is often that developers expect these scripts to be loaded. For example, if your website’s JavaScript expects a sendTracking method to be available in the global window object, an ad blocker can easily mess with the site and cause JavaScript to fall on its nose.

Here’s Chris’ example:

// https://tracking-website.com/tracking-script.js
window.trackThing = function() {
  // report data
}

// /script/index.js
function init() {
  // this throws an exception if `tracking-script.js` was blocked
  trackThing("page loaded or something");
}
init();

If tracking-script.js fails or is blocked by a browser extension your code throws an exception and could blow up your entire app. If you’re implementing external JavaScript functionality, you should make sure that things are still functional in error cases because I bet you prefer processing an order over tracking it.

But how can you go around these two problems and guarantee that you won’t be running into frontend downtimes based on synchronous resources or “death by ad blocker” scenarios in the future?

How to test and monitor risky third-party resources with Playwright

Let’s take a simple HTML example running on my localhost:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Frontend downtime</title>
  <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-GLhlTQ8iRABdZLl6O3oVMWSktQOp6b7In1Zl3/Jr59b6EGGoI1aFkw7cmDA6j6gD" crossorigin="anonymous">
</head>
<body>
  <h1>Hello world</h1>
  <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/js/bootstrap.bundle.min.js"
integrity="sha384-w76AqPfDkMBDXo30jS1Sgez6pr3x5MlQ1ZAGC+nuZB+EYdgRZgiwxhTBTkF7CXvN"
crossorigin="anonymous"></script>
  <script>bootstrap.Alert.getOrCreateInstance('#myAlert')</script>
</body>
</html>

It includes Bootstrap’s CSS and JavaScript, renders the headline “Hello world” and tries to access bootstrap.Alert which is available in the global window scope.

Note that in this case, both external resources are render-blocking. The stylesheet has to be loaded before anything appears on the screen and even though the script is placed at the end of the document, it will still block rendering (there’s just nothing to render after it).

A quick Playwright test for this page could look like this:

// @ts-check
const { test, expect } = require("@playwright/test")

test("has the correct title", async ({ page }) => {
  await page.goto("http://localhost:8080")

  // evaluate the largest contentful paint
  const largestContentfulPaint = await page.evaluate(() => {
    return new Promise((resolve) => {
      new PerformanceObserver((l) => {
        const entries = l.getEntries()
        // the last entry is the largest contentful paint
        const largestPaintEntry = entries.at(-1)
        resolve(largestPaintEntry.startTime)
      }).observe({
        type: "largest-contentful-paint",
        buffered: true,
      })
    })
  })

  console.log(parseFloat(largestContentfulPaint))

  await expect(page).toHaveTitle(/Frontend downtime/)
})

Playwright navigates to localhost and evaluates the popular largest contentful paint metric by injecting custom JavaScript into the page. When you run the script with npx playwright test you’ll see the following in your terminal:

Running 1 test using 1 worker
[chromium] › example.spec.js:4:1 › has the correct title
CLP: 116.39999999850988ms

1 passed (837ms)

Wonderful! The largest contentful paint is a hundred milliseconds long and everything works. But what if the Bootstrap resources on jsdeliver.net are slow to respond?

How to emulate slow third-party resources with Playwright

Let’s find out and delay requests by ten seconds that aren’t fetching resources from the same origin with Playwright’s page.route method.

// @ts-check
const { test, expect } = require("@playwright/test")

test("works with slows resources", async ({ page }) => {
  page.route(
    "**",
    (route) =>
      new Promise((resolve) => {
        const requestURL = route.request().url()

        if (requestURL.match(/http:\/\/localhost:8080/)) {
          resolve(route.continue())
        } else {
          setTimeout(() => {
            console.log(`Delaying ${requestURL}`)
            resolve(route.continue())
          }, 10000)
        }
      })
  )
  await page.goto("http://localhost:8080")

  const largestContentfulPaint = await page.evaluate(() => {
    return new Promise((resolve) => {
      new PerformanceObserver((l) => {
        const entries = l.getEntries()
        // the last entry is the largest contentful paint
        const largestPaintEntry = entries.at(-1)
        resolve(largestPaintEntry.startTime)
      }).observe({
        type: "largest-contentful-paint",
        buffered: true,
      })
    })
  })

  console.log(`CLP: ${parseFloat(largestContentfulPaint)}ms`)

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Frontend downtime/)
})

The test still passes but its test duration skyrocketed from 800ms to 10 seconds and similarly, the largest contentful paint also joined at the ten seconds mark.

Running 1 test using 1 worker
[chromium] › example.spec.js:4:1 › has the correct title
Delaying https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/css/bootstrap.min.css
Delaying https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/js/bootstrap.bundle.min.js

CLP: 10186.10000000149ms

1 passed (11.0s)

What is going on? Let’s create a Playwright trace and debug this test.

There are two things to note in this Playwright trace: first, the test duration increased to ten seconds because Playwright’s page.goto waits for the page onload event which is heavily delayed by the slowed-down stylesheet.

And second, and probably worse than a slow test case, the user experience suffered tremendously because the page only started rendering after ten seconds.

After running a quick Playwright test, you now know that there are 3rd party resources that could really harm your overall performance and you can check if they’re necessary or if you should implement them in another way.

But what if these requests fail or are blocked by browser extensions?

How to block requests with Playwright

So far we’ve only been delaying third-party resources, but what happens when we block them entirely? Luckily, blocking resources is quickly done with page.route, too.

test("works with blocked 3rd party resources", async ({ page }) => {
  page.route("**", (route) => {
    const requestURL = route.request().url()

    if (requestURL.match(/http:\/\/localhost:8080/)) {
      route.continue()
    } else {
      route.abort()
    }
  })

  // monitor emitted page errors
  const errors = []
  page.on("pageerror", (error) => {
    errors.push(error)
  })

  await page.goto("http://localhost:8080")

  console.log(errors)
  expect(errors.length).toBe(0)
})

If you run the script the test case fails because the page emits pageerrors. The inline JavaScript tried to call a Bootstrap method that wasn’t available because of resource blocking similar to how an ad blocker would do it.

Running 1 tests using 1 workers
[chromium] › example.spec.js:43:1 › works with blocked 3rd party resources
[
  Error [ReferenceError]: bootstrap is not defined at http://localhost:8080/:18:11
]
1) [chromium] › example.spec.js:43:1 › works with blocked 3rd party resources ====================

Error: expect(received).toBe(expected) // Object.is equality
    Expected: 0
    Received: 1

    60 |
    61 |   console.log(errors)
  > 62 |   expect(errors.length).toBe(0)
       |                         ^
    63 | })
    64 |

Tracking page errors is a handy way to monitor your JavaScript in a synthetic lab setup. In the case of ad blockers and canceled requests, you could still perform your end-to-end tests and evaluate if a blocked script affects your site at all.

Either way, page.route is an easy-to-use way to emulate network conditions with Playwright.

So what’s next?

Here’s the takeaway: whenever you implement and rely on resources you don’t control, make sure that they’re not affecting or blowing up your application! People use browser extensions, networks are flaky, services go down… many things that can go wrong.

Know how your app handles slow responses or requests that might become victims of ad blockers.

• Note: This post’s code snippets implement page request handling on a test-case basis for simplicity. I recommend looking into Playwright’s fixtures to not clutter your tests and focus on your end-to-end logic. Work with a nicely abstracted pageWithSlowThirdParties object instead!

But how should you go about these two scenarios then? Should you run all your deployment end-to-end tests with changed third-party network settings?

Frontend testing vs. monitoring

End-to-end testing with resource constraints is a great step to gain confidence and guarantee that your deployed code works no matter if your dependencies are slow or unavailable. But honestly, testing end-to-end is only the first step to real confidence because between today’s deployment and tomorrow plenty of other things can go wrong.

While end-to-end testing guarantees that your site works at one moment in time and helps to evaluate regressions in your source code, if you’re relying on third-party resources, your third-party provider’s downtime can still affect your site. And that’s sometimes unavoidable but at least, you should be aware of your dependencies’ effects on your site and bet on end-to-end monitoring. Be the first one to know when your vendors are struggling!

Conclusion

First, if you’re relying on resources that you don’t own, check how their unavailability could affect your frontend. Opt for self-hosting your assets, and if that’s impossible, rely on asynchronous and fail-safe implementations.

Second, test your site during deployments and stop, if new and risky dependencies were added to your frontend. A script element pointing to another CDN is quickly implemented. Be aware of your external resources and how they affect your application.

And third, if you have to rely on external resources, start monitoring your frontend, run your end-to-end tests on a schedule, and ensure that your site—and also those external resources!—are up and running. Don’t be me, coming to work only to learn that your customers have been looking at a blank screen 30 seconds long for the last 8 hours.

And maybe you want to give Checkly a try, we’ll run your Playwright tests for you. 😉