Demystifying Smart Selectors

In recent years, the term smart selector has become established in UI testing. But what is it actually? This article explain what’s behind this term and whether they deliver what has been promised – namely, no more flaky tests.

But before we get to smart selectors, we first need to understand what selectors actually are and what problems are associated with the classic approaches.

Classical Approaches

UI elements (button, text field, etc.) are checked for compliance with requirements in a UI test. Typically, so-called happy paths are tested for functionality. Happy paths represent critical workflows in a system, such as a login or an ordering process.

A selector selects the UI elements that are used in this process and executes the described action in it.

For a human tester, this works via the eye and visual properties, i.e. we as humans have learned what a button or a text field looks like and can select them via the eye.

On the other hand, a computer needs code-based selectors in classical test automation to be able to address these elements. Typically, XPath or CSS selectors are used, which uniquely assigns elements in the Document Object Model (DOM), for example. The DOM describes the structure of the UI.

However, using such unique selectors also has disadvantages for automation:

• If the selectors in the program code change, the selectors must also be adapted in the automation script. Otherwise, they will fail. Such tests are called flaky tests.
• The selectors were defined in the developer’s program code and can have very long and complicated names depending on the interface. This can lead to confusing tests.
• UI tests are actually supposed to test the system from the outside like a human tester. However, by using code selectors, such a black-box test is not possible.

The so-called smart selectors promise to solve these problems to a certain degree.

What Smart Selectors Are

Let us now come to this concept of the smart selector. Not every smart selector works in the same way. But based on my experience, the following three approaches are basically followed, which are then combined with each other and with classic selectors in practice. Sometimes you also read about so-called dynamic selectors, which are so-called smart. They are actually a mere combination of classic selectors which secures one another.

But now to the more interesting approaches!

Optical Character Recognition (OCR)

OCR is the recognition of text from images. The technology has been used for a long time for various use cases and has now also established itself in UI testing tools. OCR, in combination with classical selectors, is a powerful tool to make UI tests more robust.

For example, the presence of text on a page can be tested as an acceptance criterion. If a test step includes clicking on a text, OCR can extract the text’s coordinates and use them to address the text uniquely.

Picture-in-Picture Search

Picture-in-Picture Search refers to the search of cut-out elements on the UI by pixel matching. The element is usually cut out and stored as an image. If you want to select this element in a test, the picture runs over the entire UI and tests for a match between the cut-out elements and the actual UI. If a match is found, the position of the element can be derived from this. This procedure can be used to automate tests selector-independent. Although this approach is closer to the human tester than a classical selector, it is often only used in combination with them. This is because pixel-precise matching is very difficult due to different resolutions on test devices interfering with it.

Relational Descriptions

A new approach is relative locators, which were also introduced in Selenium 4. The functionality is simple: Instead of selecting an element directly, it is addressed via the relation to other elements. For example, an email text field can be found by its relation to the text description “Email”.

The great advantage of this addressing is that elements positioned relative to each other are also tested for the correctness of their position relative to each other and checking for functionality. On the other hand, if there are no clear relations available, it is hard to locate elements solely using this approach.

Takeaway

The use of these smart selectors helps to make test cases more robust and indeed serves its purpose. Especially the abstraction to simulate human behaviour tremendously helps in the quality assurance of software. However, one should also understand the technology behind the terms to use them correctly and not expect miracles.

In this article, I have given a brief introduction to various technologies behind smart selectors. I would be interested to know: Have you already used smart selectors, and what was your experience with them?

Join our Discord community to discuss all things testautomation!