E2E testing: challenges & lessons learned

End-to-end testing. The holy grail of all testing types.

Here's everything we learned regarding the ins and outs of end-to-end testing - lessons, takeaways, mistakes, you name it.

So, let's start out with why

Swimm started growing, we added new features, and we found that we simply couldn't keep up with changes.

We realized that our goal for a testing solution had two parts:

1. Find a solution that allowed us to catch regressions bugs DURING development
2. Save manual regression QA time

E2E testing seemed to be the right answer to our testing paradigm.

We ended up choosing Cypress

Pros

• Awesome community

• Great documentation

• Great features: automatic waiting, automatic sreenshots & videos, time travel

• Cypress is intuitive & easy to set up

• Writing tests is fun

• Using Cypress' API applies for user behavior

Cons

• Lack of multi-tab support

• Difficulty with iFrames

• Limited browser support (just Chrome-based browsers & firefox)

Challenges of E2E testing

Wins:

• Identified & fixed a lot of bugs

• Saved so much time by not having to repeat manual flows weekly

• Dramatically increased stability & coverage
  Challenges

• Human vs machine testing

• We had zero experience with e2e tests

• Monitoring tests that are less reliable

• It takes more time than you think

Key takeaways

1. Appoint an owner for E2E testing
2. Patience, patience, patience
3. Write stable tests
4. Trust. Your. Tests.
5. Stable tests = stable product

Here's a detailed blog on how we implemented E2E testing.