I believe this is an issue that should be treated as a Cypress bug, but it has an easy-enough work-around, so I'm documenting my findings here as a blog post in case my learnings can be helpful to someone else.
Most Cypress tests use a beforeEach() hook to do things like login to a site, a series of it() blocks to define the tests, and then perhaps an after() hook to perform some data cleanup or reset state. While running tests in the desktop runner, it can be helpful to mark a test with .only to selectively run it as you make changes. However, .only will cause the entire spec to fail in headless mode, and therefore you must remove all usages of it before pushing your changes up to your CI environment.
Sometimes a test needs to be removed from your suite, or perhaps you want to commit some code for a test that is incomplete and not ready to run. You might want to use .skip to do that, but unlike .only - this will work both for local headed test runs, and headless runs.
So, let's imagine that you have some logic in an after() hook, and it doesn't need to reauthorize the user, so you just use some commands that assume the browser still has the page open from the last run test in the suite. If that last test is marked with .skip, your after() hook will run just fine, because the local headed runner doesn't tear down the browser automatically after the last test, and any tests marked with .skip will never launch any action at all.
In headless runs, however, the page is torn down after each test, and skipped tests still do launch a browser page instance, though they never launch the baseUrl or perform any actions. For this reason, in headless runs, an after() block that is immediately preceded by a test marked with .skip will only see the blank Cypress page, with no content loaded, and thus they cannot be expected to execute the way they would in a local headed run.
The workaround for this is to treat your after() hook exactly like your test cases, and replicate the login and page load commands from your beforeEach() hook in your after() hook. This will guarantee that you get back to the necessary app page and state before you perform your cleanup steps. This issue tripped me up when I first encountered it, I hope you don't run into it yourself after reading this.
Top comments (0)