In a world where CI/CD is finally being taken seriously, everyone is trying to have stable and useful Automated Tests.
Most teams have started using cloud platforms which provide out-of-the-box features, such as pixel-by-pixel screenshot comparison, video recording and advanced assertions.
Without further ado, let’s start:
1. Endtest
A platform that allows you to create, manage and run Automated Tests for Web Applications and Native Mobile Apps (Android & iOS), without having to write any code.
It’s mostly free to use and they’ve got some really nice features, such as:
• Cross-browser grid, running on Windows and macOS machines
• Codeless Editor for Automated Tests
• Support for Web Applications
• Support for both native and hybrid Android and iOS apps
• Unlimited Video Recordings for your test runs
• Screenshot Comparison
• Geolocation
• If Statements
• Upload files in your tests
• An Endtest API, for easy integration with your CI/CD system
• Advanced Assertions
• Mobile Tests on real mobile devices
• Self-Healing tests, powered by Machine Learning
You can even run Automated Tests on Real Mobile Devices:
According to their Twitter account, they seem to be adding new features every week:
Endtest is a truly powerful and awesome platform and clearly the winner.
2. Ghost Inspector
Second place goes to Ghost Inspector, an interesting tool developed by a small team from Seattle.
It does have a Codeless Editor and a basic cross-browser grid (only headless Chrome and Firefox running on Ubuntu).
This tool can easily be integrated with different CI/CD systems by using their API.
You should really check out their Documentation section, it covers a lot.
The main inconvenience related to this tool is that you cannot run a test for longer than 10 minutes.
3. BrowserStack
A few years ago, BrowserStack was the best tool for cross-browser testing.
But the times have changed, new technologies have emerged and they failed to adapt.
Nowadays, offering a huge grid of VMs with browsers and mobile devices is no longer such a big deal.
Users are asking for smart solutions, built-in features and codeless approaches.
4. Testim
Developed by a small team from Israel, this tool offers some interesting features and a very limited cross-browser grid for users to run their tests on.
They’re offering a really cute feature: self-healing tests.
This is actually a very basic technology, which is also known as element locator hierarchy. Here’s how it works:
- The test runs and finds locators for each element. A locator can be an ID, Class Name, XPath, CSS Selector, Text Inside, etc.
- It remembers all those locators for each element and puts them into an order.
- The next time the test will run, it will use one of those locators. If the element is not found, it will just use the other locator.
Sadly, they are falsely advertising this feature as Machine Learning.
Testim trying to sell their solution:
Top comments (3)
I'm curious: why do you list "codeless technology" as the foremost plus of most of these systems? I ask because, in my experience, GUI-based testing tools often end up being limited and awkward compared to code-based tools. They also tend to discourage teams from making tests first-class citizens of the project, or at least make it impractical to track them by diffing. Am I missing your point?
Hi Paolo, I think you're referring to the previous generation of testing tools.
Things have changed a lot.
I believe that nowadays saying that you prefer to code your own tests is like saying that you want to host your application on your own bare metal server instead of hosting it on AWS.
It's quite possible that I'm not current. The latest testing libraries I've used are Nightwatch and Cypress.io. The second worked well for me. The first... not so much.
Ideally, a flexible tool would allow both models: the GUI when you can, and code when you need to. (By the principle: "make simple things easy and complex things possible"). It would also allow me to move easily between the two levels. In particular, it's essential that I can save tests in a text-based format that can be committed to Git, hand-edited and diffed. I'll look through the tools you listed for these features.
Thanks for the wrap-up, Klaus!