Test Automation as a process has existed since the early 1970s-1980s when diskettes and manuals were sold separately by companies. A key issue of testing software faced by users during that time was something that might ring familiar with Automation and QA teams today: Maintenance. It's incredible how an age old problem that existed with primitive automation software still haunts QA teams today.
Test Automation has come a long way since then and the efficiency of a well structured and defined framework can drastically reduce the effort required in QA. But to build such frameworks and systems around your code, that’s not just stable but also scalable with every build update is no easy task. It usually takes upto a year to start seeing real value in test automation projects -- when they are done right. But should we really wait that long?
Why test automation becomes a high effort parallel development project
Let's get to the 3 primary barriers teams have to overcome to build a fully automated testing framework:
1. Complexity:
“Test Automation is a development process, end of story.” - Anonymous
Before you start piecing together code that forms the foundation of your framework there are several important questions that need answering:
i. What language should I choose? (Python, javascript etc. usually dependent on what your product is built with)
ii. What combination of frameworks do I need? (for. Eg; Selenium + Junit OR BDD framework like Cucumber +TestNG)
iii. What exactly should we automate? Unit tests, Sanity or Smoke, Regressions?
These are just a few to begin with. If we keep going you’ll start having to look at Test data, Page Object Models, Build management etc.
As you can see, bringing all these elements together is no mean feat. It requires months of planning, development, not to mention you need the right team to help you build the entire thing.
2. Maintainability
You’re 6-8 months into your Test Automation journey. You’ve been able to automate 40-50% of test suites which is above average and a good speed of development. But then the dev team announces that there is going to be an extensive UI overhaul in your product.
To testing teams this means one thing, a boring, tedious maintenance effort that's going to slow the speed of your test coverage. Now this is obviously a worst case scenario.
But maintenance even since the early 1980s, still persists as the number one pain point for testers and automation teams alike. There are several reasons why maintenance is so hard:
i. Conducting impact analysis after code changes is arduous
ii. No easy solution to handling dynamic elements and selectors
iii. Test documentation is not upto speed for new software releases
iv. Traceability of affected tests is time consuming and difficult to get 100% right
3. Scalability
Once you have set everything up and sorted out your maintenance issues and reached a good level of testing cadence for your release cycles it’s time to scale. As your application continues to grow, your company is also launching a couple of new products
So just re-deploy your Test Automation .bat file for the new testing teams right? Well it's not that simple. It turns out, the new application is being built using Python instead of Java and your framework has been set up to support automated testing for Java based applications.
Did you know: Testsigma is completely codebase and Techstack agnostic? Its built-in APIs and drivers make adding integrations and extending automation a lot easier!
Scaling in test automation has always been difficult because it is hard to establish design patterns and standardize frameworks across teams in an organization.
With the variance in techstacks, skill level and experience, the Testing CoE (Center of Excellence) is often faced with dilemmas on how to replicate the success of one automation project across different teams.
Which leads us nicely into our next section:
Build vs buy? Why this question is harder than it looks
Traditionally there are two main approaches to implementing Test Automation. You either use a set of open-source frameworks and build your setup from scratch, OR you purchase a ready-made solution. But there are so many factors to consider and even after going through them all it's a difficult and sometimes irreversible decision.
Let's look at why this is so and what problems each approach creates.
- Building a framework
i. Pressure on realizing ROI is high, therefore becomes a results-driven process
ii. Strategic planning of automation becomes short-sighted and has long-term stability effects
iii. Not enough skilled automation engineers to really carry out the implementation quick enough
iv. Testing deadlines are too short to wait for automation to develop and it results in more manual testing
v. Product quality is compromised in search of managing deadlines
- Buying an automation tool
i. Upfront cost is inevitable but paid tools are a ready made solution
ii. The variance in testing scenarios means no tool can match your needs perfectly
iii. Paid tools are closed systems so there is no scope of extending or customizing the platform to your needs without the intervention of the vendor
iv. Eventually scaling becomes highly dependent on the roadmap of the product
Sounds familiar?
Bringing stability and standardization to Test Automation
The objective of testing has always been ensuring the delivery of bug-free, high performance products, services & applications. But testing teams are usually handcuffed when it comes to resources. The developer-tester ratio is a huge debate in modern engineering teams and budgets also differ based on company and team size.
So we have a problem.
i. On one hand, we have several tools and frameworks to choose from but a decision taken always leads to a compromise on either time or flexibility.
ii. On the other hand, we have teams with varying budgets and skill sets and a pressure to keep up with the speed of development.
So, what’s the solution to this long standing problem in Test Automation? Well, first there has to be a change in approach which involves
i. Setting realistic timelines and goals for your automation project
ii. Building a clear testing & automation strategy
iii. Involving not just QA teams but everyone else in the quality process
But this is just half the battle. It still does not account for the varying degree of capabilities current tools and frameworks in the market provide.
To bring real standardization to Test Automation, a framework needs to be both powerful and malleable. Here’s what a Test Automation system should ideally provide:
i. A framework should be simple to set up,, easy to use and reliable to scale
ii. A framework should be easy to adopt for both Automation and QA teams but should also enable them to keep up with the speed of development.
iii. A Framework should help teams shift left without compromising on technical & budgetary constraints.
iv. A Framework should enable end-to-end test automation and simplify how teams can extend it, to match the variety of test scenarios encountered.
We’re on a mission
At Testsigma, we’ve taken a completely new approach to Test Automation and built a platform from the ground up that brings together the best of both open-source and paid tools. Testsigma is a first of its kind Open-Source platform with an out-of-the-box framework.
i. A simpler set-up process that takes only a few minutes
ii. Integrated modules to manage tests, elements and data
iii. An Intuitive UI to navigate and organize everything
iv. Built-in drivers, SDKs and Test Runner for powerful, quick results
More importantly, Testsigma is a completely customizable platform. You can build utilities or install add-ons from our marketplace as your testing requirements evolve.
Our mission is to transform the way Test Automation is achieved by teams and companies worldwide. By eliminating the complexity in setting up, testing and scaling Test Automation but also adopting a team approach to testing, testing
For too long, QA has lagged behind development in terms of speed, focus and capability. It’s time we levelled the playing field and finally make significant strides to fixing Test Automation
Top comments (0)