This is my test automation journey from Selenium (well, technically from HP QTP but nobody I know seems to be using that nowadays) to Katalon (which is Selenium + UI) to Cypress (not the country “Cyprus”, but the automation tool — although, I still get the latest happenings in the country when I google “Cypress latest news”). Hey Google - change your algorithm!
Like all "Gen-X" QA’s (we were called “testers” back in the days — the debate is another story to tell), I started with manual testing. However, as my career progressed (and having a developer background), I leaned towards doing more test automation. As enterprise software moved into browser-based application, I felt I needed to up-skill myself in that area in order to survive the ever-evolving tech world.
Nine years ago, I was given an opportunity by my company (a global bank) to explore automating the pain points our internal clients were facing on the company website. My managers only had two requirements: I needed to come up and implement a solution within 3 months and there is a minimal chance the company will pay for a licensed tool. Without doing much research, it was obvious that Selenium (Webdriver) was the perfect choice (as if there were other options back then). I started building a basic framework using the eclipse IDE and my Java skills (back from my university days). Without much background and just some good ol’ self-learning (thanks mostly to stackoverflow -Yes, I'm not embarassed to say I am a "stackoverflow" developer), to say "it was a pain" was an understatement. Over time, I enhanced the framework to include logs, reporting, data-driven csv files, etc. — I hated trying find ways how to outsmart the explicit waits — sorry I had to use implicit waits sometimes (how anti-pattern!).
Fast-forward to last year when I moved to Melbourne and worked as a QA lead in a local bank. The company had a framework based on Selenium with very complex logics and would require multiple manual interventions in the process. It was very challenging to maintain and took forever (well, maybe months) to spin-up a similar framework as it was not completely re-usable for new projects.
I interviewed multiple QA’s (they had a massive team doing manual tests) and conducted sessions to understand their issues and how it can be improved. Basically, 90% of the QA’s (mostly non-technical) were not motivated to learn the framework due to its complexity (For a semi-technical person like myself, it took me weeks to finally understand it). As such, I looked for a tool that was easy to learn and use, and found Katalon studios. It covered both Mobile and Internet banking, the company's main customer channels. I was able to integrate it with the test management tool, source control tool, and the CI/CD pipelines seamlessly. Even the auto-creation of JIRA bugs was automated thanks to my limited groovy scripting skills, which was similar to Java, anyway. It took less than a day to spin up a new framework for any project. Heck, I even spent time to create a detailed Confluence documentation on it (step-by-step-by-step that even a 5-year older can replicate — maybe a tenth grader).
This takes me to my current role as a Senior QA (Yes, there are still QA roles available and not all are SDET’s but slowly moving towards that direction so it scares me a bit). Taking a leaf from my previous playbook on "understanding the current landscape", I did my initial observation on the situation. All the other QA’s have technical background. Our team are testing both API’s and UI for their e2e flows, and there is already an established Selenium (and Flurl) structured framework in place.
I looked up what other teams in the company (yes, a bit siloed but most companies are anyway especially for scale-ups) are using and found that most of them are converting their Selenium scripts into Cypress (which I’ve heard a couple of years back but it was too immature of a tool back then, that I didn’t want to waste time researching it). I was initially considering Katalon (with bias since I recently used it) but for some coincidence (destiny I would say), their parent company decided to license some of the free tools like CI/CD integrations. This discouraged me massively and I suppose, I wasn’t the only one when I saw the hundred likes on my rant on their community forum.
I tried doing a bit of PoC (Proof of Concept) on Cypress with one of our functionalities (great thing was our developers were open for people trying out new innovations) and boy, it was impressive! Open source and easy to install, fast execution, no need to build "smart waits", great documentation, “time travel” screenshot and video capability for debugging and finding the exact root cause, easy to maintain with readable scripts, and many more… I was able to create scripts within hours (even without much knowledge on JavaScript, which unfortunately, is it’s only language). Thus, I found more time increasing the test coverage. I believe a tool like this would resolve our issues with flaky environments, limited e2e test coverages, and slow execution times. Cypress, imho, builds more confidence on our tests, that when they fail, we know it’s not because of our scripts, but because something is really wrong with the code or environment.
It might take a bit of time to convert the other QA’s into this new-found religion and adoption rate at this stage might be low due to the fact that it needs significant amount of time to re-factor our Selenium scripts into Cypress. Furthermore, there are still limitation of Cypress like multiple domains handling and cross-browser testing (spoke too soon, Cypress 4.0 or higher now supports Edge and Firefox). These are fair points to consider at this stage but it’s just so hard not love Cypress when I compare the big gap I spent coding with Selenium as opposed to Cypress.
"Unscalable maintenance is the death of progress" — PlanGrid (Autodesk)
In my next article (hopefully comes sooner depending on the level of my laziness), I would do a comparative analysis of the three tools (well, Selenium is not really a tool but I’ll do it anyways) based on my personal experience. Just a warning, Cypress takes the cake even if I used it only for a couple of months as compared to Katalon for a year, and Selenium for multiple years.
Top comments (0)