Angular Testability: Dealing with Selenium or Protractor timeouts

Running automated integration tests is a crucial element for any serious CI/CD pipeline and within our company, our QA uses Selenium for that purpose. Angular offers Protractor for that purpose, but to each artisan his tool, our Angular apps are just part of a suite that include non Angular apps and it's easier for the team to test them all using Selenium.

Anybody that has messed with Protractor knows that one of the early frustrations you will run into is knowing from the Selenium's perspective when the Angular page is done doing what it does and is ready to be tested. For novices that would mean any API calls you would be making to the backend to get data, any animation running on the page that as a user you typically wait for before knowing that yes, this page looks like it's ready for me to interact with. Early on our team ran into just that issue and time and resources being an issue, decided to use timers to wait for a reasonable time for the test to assume that the page is ready to interact with. That works most of the time but of course it's not the way you want to run a long term stable QA solution.

The Testability API

For the purpose of testability, Angular exposes the Testability API:

The Testability service provides testing hooks that can be accessed from the browser and by services such as Protractor. Each bootstrapped Angular application on the page will have an instance of Testability.

Plugging into that API is the way for your testing framework to know when your Angular app is ready to be tested. It uses NgZone to keep track of all outstanding operations currently running in your application and once they are all completed, marks your application as stable. The way that is done with most selenium scripts is to run a Javascript script from within Selenium that does one of the following:

Synchronously you can execute a Wait condition that waits for your script to return true like:

Java code
...

 wait.until(new ExpectedCondition<Boolean>() {
            public Boolean apply(WebDriver driver) {
                String angularScriptPath = "src/angular-script.js";
                String angularScript = null;
                try {
                    angularScript = SeleniumWebDriverTest.readFile(angularScriptPath);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                Boolean angularPageLoaded = ((JavascriptExecutor) driver).executeScript(angularScript);

                return angularPageLoaded;
            }
        });

...

and your angular-script.js would look like

return window.getAllAngularTestabilities().findIndex(x=>!x.isStable()) === -1

Check out this well written article that explains how to wait for Angular page readiness using Selenium and gives a JSWaiter Java class that implements the correct approach for Angular > v5.

You can also go the async way and run an async script in Selenium that will provide a seleniumCallback callback function that your script will invoke when Angular is ready

let rootElement = window.getAllAngularRootElements()[0];
let testability = window.getAngularTestability(rootElement);
testability.whenStable(seleniumCallback);

The catch is that this is only half the story, because any async process that you have running in your Angular app needs to indeed be done before Angular thinks that all testabilities are done and ready.

If you have tried this above and it is still not working, it's more than likely because you have an async process running that prevents your app from ever attaining stability for test purposes. This is a particular true if you're using setTimeout or setInterval, let's say for example that as soon as the user logs in you launch a setInterval to determine when their authentication or session tokens expires. If you trigger that a setInterval as soon as your app boots up, you're toast in terms of testing if you don't take the proper measures as your app testability will never reach the stable point. That will more than likely mean a timeout if you're using Protractor or Selenium.

How to get stable

To attain stability for testing if you're using setTimeout or setInterval there are two things you can do:

Use ngZone.runOutsideAngular()

Use ngZone.runOutsideAngular if you're doing any work that does not require UI updates. If your work does require UI updates (where change detection is required), you can easily re-enter the Angular Zone by using ngZone.run. Check out this article for an in -depth explanation of ngZone and how it works with testability.

Delay starting any timers until you really need to

In my own case I was starting a timer as soon as the user logged in. I just delayed setting that timer until the Angular app is indeed ready for testing. This was a simple as delaying starting the timer in my authentication logic until Angular told testing was ready using a Testability on the Angular zone:

new Testability(ngZone).whenStable(()=>　{ 
        console.log("Now I am stable");
    　});

Check out this Stackblitz for an idea on how to do that:

With this simple change in the flow, the app reaches testing stability before the other processes can kick in.

One last thing that might help in debugging this issue for you, how would you know that you have incomplete async tasks still running? Lucky for you, this blitz will help you setup your app to log all running microtasks in your Angular zone. This will allow you to check on any process you might have overlooked that is keeping your app from stabilizing initially:

Also check out this Stackoverflow discussion for some more context on this issue and how to solve it.