If you have ever written tests on Espresso, the open-source testing framework from Google, you know that they are not always stable and easy to read. My name is Ksenia Nikitina, and I am an Android developer at Kaspersky. In this article, I will provide you with a way to make your autotests meet all the key qualities, including high readability, stability, logability, possibility of taking screenshots, compatibility with AndroidOS, and, finally, a well-thought-out and intuitive architecture.
So, when there is a need to write autotests on Android, we most often turn to the Espresso framework by Google (if you are not familiar with Espresso, you can find out more about it in the official documentation). Espresso allows you to work with application elements natively and using the white-box method. You can first find the necessary elements on the screen using matchers, and then perform various actions or tests on them.
Let's look at how the autotests work using a specific example. You can launch the application and reproduce all the steps described below by yourself by downloading the source code on our GitHub and launching the Tutorial project.
When we open the application, we see a screen with the "Internet Availability" button, and when we click on it, we navigate to the Wifi status check screen.
The "Check Wifi status" button is clickable. Once clicked, the current Wifi status is displayed.
It is also important to us that when we rotate the screen, the text indicating the Wifi status does not change.
- To sum it up, to verify that the application works correctly, we need to go through the following steps:
- Before starting the test, set the device to landscape mode, turn on Wifi, and launch the main screen of the application.
- Check that the Internet Availability button is visible and clickable.
- Check that the header indicating the Wifi status does not contain any text.
- Click the "Check Wifi status" button.
- Check that the header text changed to "enabled".
- Disable Wifi.
- Click the "Check Wifi status" button.
- Check that the header text changed to "disabled".
- Flip the device.
- Check that the header text is still "disabled".
Let's try to automate this text test case and use it as an example to discuss the existing shortcomings of Espresso.
Espresso test example
Our test class using the Espresso framework:
package com.kaspersky.kaspresso.tutorial
import androidx.test.espresso.Espresso.onView
import androidx.test.espresso.action.ViewActions
import androidx.test.espresso.assertion.ViewAssertions.matches
import androidx.test.espresso.matcher.ViewMatchers.isClickable
import androidx.test.espresso.matcher.ViewMatchers.isDisplayed
import androidx.test.espresso.matcher.ViewMatchers.withId
import androidx.test.espresso.matcher.ViewMatchers.withText
import androidx.test.ext.junit.rules.activityScenarioRule
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.platform.app.InstrumentationRegistry
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
@RunWith(AndroidJUnit4::class)
class WifiSampleEspressoTest {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
// launch target screen
onView(withId(R.id.wifi_activity_btn)).check(matches(isDisplayed()))
onView(withId(R.id.wifi_activity_btn)).check(matches(isClickable()))
onView(withId(R.id.wifi_activity_btn)).perform(ViewActions.click())
// set portrait orientation
val uiAutomation = InstrumentationRegistry.getInstrumentation().uiAutomation
uiAutomation.executeShellCommand(SHELL_COMMAND_AUTO_ROTATE_DISABLE)
uiAutomation.executeShellCommand(SHELL_COMMAND_PORTRAIT_ORIENTATION)
// test
onView(withId(R.id.wifi_status)).check(matches(withText("")))
onView(withId(R.id.check_wifi_btn)).check(matches(isDisplayed()))
onView(withId(R.id.check_wifi_btn)).check(matches(isClickable()))
onView(withId(R.id.check_wifi_btn)).perform(ViewActions.click())
onView(withId(R.id.wifi_status)).check(matches(withText(R.string.enabled_status)))
// Turning off wifi
uiAutomation.executeShellCommand(SHELL_COMMAND_TURN_OFF_WIFI)
// wait for switching wifi
Thread.sleep(3000)
// test
onView(withId(R.id.check_wifi_btn)).perform(ViewActions.click())
onView(withId(R.id.wifi_status)).check(matches(withText(R.string.disabled_status)))
//rotate
uiAutomation.executeShellCommand(SHELL_COMMAND_AUTO_ROTATE_DISABLE)
uiAutomation.executeShellCommand(SHELL_COMMAND_LANDSCAPE_ORIENTATION)
// wait for rotation
Thread.sleep(3000)
// test
onView(withId(R.id.wifi_status)).check(matches(withText(R.string.disabled_status)))
}
private companion object {
const val SHELL_COMMAND_AUTO_ROTATE_DISABLE = "content insert --uri content://settings/system --bind name:s:accelerometer_rotation --bind value:i:0"
const val SHELL_COMMAND_PORTRAIT_ORIENTATION = "content insert --uri content://settings/system --bind name:s:user_rotation --bind value:i:0"
const val SHELL_COMMAND_LANDSCAPE_ORIENTATION = "content insert --uri content://settings/system --bind name:s:user_rotation --bind value:i:1"
const val SHELL_COMMAND_TURN_OFF_WIFI = "svc wifi disable"
}
}
Kotlin
In this example, we realized that it was impossible to implement the test using Espresso alone, so we had to bring in the UIAutomator library. The test runs and allows you to check if the application is working correctly, but, unfortunately, the code has several drawbacks at the moment.
What we would like to improve in the code: separate the logic of element descriptions and the sequence of actions and checks; use a declarative style of writing tests; eliminate multiple repetitive onView calls and hierarchical nesting. Unfortunately, it is impossible to cover all the needs for Android autotesting with Espresso alone due to the lack of certain features.
Code improvement
Let's try rewriting the code of our test by resorting to the open-source framework Kaspresso, which takes a declarative approach to writing tests. Let's look at a specific section of the code.
It looked like this:
onView(withId(R.id.wifi_activity_btn)).check(matches(isDisplayed()))
onView(withId(R.id.wifi_activity_btn)).check(matches(isClickable()))
onView(withId(R.id.wifi_activity_btn)).perform(ViewActions.click())
And now it turns out like this:
wifiActivityButton {
isVisible()
isClickable()
click()
}
Kaspresso uses Kakao, a Kotlin-DSL wrapper for Espresso. The Kaspresso open-source framework inherited two main concepts from Kakao. The first is KView - a special representation of the UI elements that the test interacts with. Using KView saves us from constantly calling the onView method; now we just need to add matchers in the KView constructor once.
The second concept is the Screen class (implementation of the Page Object pattern), which describes all the elements that the test interacts with. The concept comes from web development and is about creating a description of the screen visible to the user. This object does not contain any logic. This allows you to describe Screens and their elements in a separate file and interact with them from the test class code in a declarative style. Thus, Page Objects are completely independent, which makes them reusable to the max.
Let's rewrite our test class using the Kaspresso framework. If you run into problems with any of the following steps, you can find a ready-made version of the autotest in the tutorial_results branch.
The first step is to create the MainScreen Page Object:
package com.kaspersky.kaspresso.tutorial.screen
import com.kaspersky.kaspresso.screens.KScreen
import com.kaspersky.kaspresso.tutorial.R
import io.github.kakaocup.kakao.text.KButton
object MainScreen : KScreen<MainScreen>() {
override val layoutId: Int? = null
override val viewClass: Class<*>? = null
val wifiActivityButton = KButton { withId(R.id.wifi_activity_btn) }
}
Here we need to determine the ID of the layout installed on the screen (layoutId) and the name of the class (viewClass). This is required to link the test to a specific layout file and the activity or fragment class. Linking them makes the tasks of further support and development of the test more convenient, but for now, we are faced with the task of considering a demo test class, so let's leave the value at "null".
Next, let's create a WifiScreen:
package com.kaspersky.kaspresso.tutorial.screen
import com.kaspersky.kaspresso.screens.KScreen
import com.kaspersky.kaspresso.tutorial.R
import io.github.kakaocup.kakao.text.KButton
import io.github.kakaocup.kakao.text.KTextView
object WifiScreen : KScreen<WifiScreen>() {
override val layoutId: Int? = null
override val viewClass: Class<*>? = null
val checkWifiButton = KButton { withId(R.id.check_wifi_btn) }
val wifiStatus = KTextView { withId(R.id.wifi_status) }
}
Now let's get down to the test cases. The test class should be inherited from the TestCase class. Let's create such a WifiSampleTest class and add a method annotated with a @test (import org.junit.Test) to check how the application works.
package com.kaspersky.kaspresso.tutorial
import androidx.test.ext.junit.rules.activityScenarioRule
import com.kaspersky.kaspresso.device.exploit.Exploit
import com.kaspersky.kaspresso.testcases.api.testcase.TestCase
import com.kaspersky.kaspresso.tutorial.screen.MainScreen
import com.kaspersky.kaspresso.tutorial.screen.WifiScreen
import org.junit.Rule
import org.junit.Test
class WifiSampleTest : TestCase() {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
MainScreen {
wifiActivityButton {
isVisible()
isClickable()
click()
}
}
WifiScreen {
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
wifiStatus.hasEmptyText()
checkWifiButton {
isVisible()
isClickable()
click()
}
wifiStatus.hasText(R.string.enabled_status)
device.network.toggleWiFi(false)
checkWifiButton.click()
wifiStatus.hasText(R.string.disabled_status)
device.exploit.rotate()
wifiStatus.hasText(R.string.disabled_status)
}
}
}
We mentioned earlier that Espresso doesn't work with Android OS, so users have to resort to the UIAutomator library. Note that the test code uses an instance of the Device class, which is part of the Kaspresso framework. This object has many useful methods suitable for interacting with Android OS. You can read more about them here.
What did we achieve?
- Screen descriptions and testing the logic of their work are now separated. Test cases can now be described independently of Screens, and Screen classes can be reused in different tests.
- The code has become much more readable, and it is now possible to write tests in a declarative style. We simply indicate, what actions and checks we want to perform and on which Screen.
Code separation according to the steps of the test cases
The test class code is already looking better, but it still has some issues. Usually, any tests (including manual ones) are performed according to test cases. That is, the tester has a sequence of steps that they follow to check the Screen performance Now imagine that our test class contains way more steps and, accordingly, lines of code. Because it is unclear where one step ends and another one begins, it gets extremely difficult to understand and expand. We can mitigate this problem by using comments. Let's add comments to each step in our test class:
package com.kaspersky.kaspresso.tutorial
import androidx.test.ext.junit.rules.activityScenarioRule
import com.kaspersky.kaspresso.device.exploit.Exploit
import com.kaspersky.kaspresso.testcases.api.testcase.TestCase
import com.kaspersky.kaspresso.tutorial.screen.MainScreen
import com.kaspersky.kaspresso.tutorial.screen.WifiScreen
import org.junit.Rule
import org.junit.Test
class WifiSampleWithStepsTest : TestCase() {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
// Step 1. Open target screen
MainScreen {
wifiActivityButton {
isVisible()
isClickable()
click()
}
}
WifiScreen {
// Step 2. Check correct wifi status
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
wifiStatus.hasEmptyText()
checkWifiButton {
isVisible()
isClickable()
click()
}
wifiStatus.hasText(R.string.enabled_status)
device.network.toggleWiFi(false)
checkWifiButton.click()
wifiStatus.hasText(R.string.disabled_status)
// Step 3. Rotate device and check wifi status
device.exploit.rotate()
wifiStatus.hasText(R.string.disabled_status)
}
}
}
This slightly improves the readability of the code, but it won't solve all the issues. For example, if one of the tests fails, how do we know at which step it happened? Espresso in this case just dumps it into the log, so you have to examine the data, trying to figure out what went wrong. It would be much better if the logs displayed information about the start and end of each step.
We can also wrap some parts of the code in "try/catch" blocks to catch test failures. Now our test looks like this:
package com.kaspersky.kaspresso.tutorial
import android.util.Log
import androidx.test.core.app.takeScreenshot
import androidx.test.ext.junit.rules.activityScenarioRule
import com.kaspersky.kaspresso.device.exploit.Exploit
import com.kaspersky.kaspresso.testcases.api.testcase.TestCase
import com.kaspersky.kaspresso.tutorial.screen.MainScreen
import com.kaspersky.kaspresso.tutorial.screen.WifiScreen
import org.junit.Rule
import org.junit.Test
class WifiSampleWithStepsTest : TestCase() {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
try {
Log.i("KASPRESSO", "Step 1. Open target screen -> started")
MainScreen {
wifiActivityButton {
isVisible()
isClickable()
click()
}
}
Log.i("KASPRESSO", "Step 1. Open target screen -> succeed")
} catch (e: Throwable) {
Log.i("KASPRESSO", "Step 1. Open target screen -> failed")
takeScreenshot()
}
WifiScreen {
try {
Log.i("KASPRESSO", "Step 2. Check correct wifi status -> started")
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
wifiStatus.hasEmptyText()
checkWifiButton {
isVisible()
isClickable()
click()
}
wifiStatus.hasText(R.string.enabled_status)
device.network.toggleWiFi(false)
checkWifiButton.click()
wifiStatus.hasText(R.string.disabled_status)
Log.i("KASPRESSO", "Step 2. Check correct wifi status -> succeed")
} catch (e: Throwable) {
Log.i("KASPRESSO", "Step 2. Check correct wifi status -> failed")
}
try {
Log.i("KASPRESSO", "Step 3. Rotate device and check wifi status -> started")
device.exploit.rotate()
wifiStatus.hasText(R.string.disabled_status)
Log.i("KASPRESSO", "Step 3. Rotate device and check wifi status -> succeed")
} catch (e: Throwable) {
Log.i("KASPRESSO", "Step 3. Rotate device and check wifi status -> failed")
takeScreenshot()
}
}
}
}
In some catch blocks, we take screenshots that can help us analyze fails in the future. One way to take a screenshot is to call the takeScreenshot() method, but it is not recommended to use it directly. You can get acquainted with a more convenient and flexible tool for taking screenshots in Kaspresso in this lesson.
The open-source Kaspresso framework offers Step, a useful and convenient abstraction. Everything that we have just written manually is implemented inside of it.
To use Steps, you need to call the run {} method and list in curly brackets all the steps that are to be run during the test. Each step should be called inside a Step block. The test code then looks like this:
package com.kaspersky.kaspresso.tutorial
import androidx.test.ext.junit.rules.activityScenarioRule
import com.kaspersky.kaspresso.device.exploit.Exploit
import com.kaspersky.kaspresso.testcases.api.testcase.TestCase
import com.kaspersky.kaspresso.tutorial.screen.MainScreen
import com.kaspersky.kaspresso.tutorial.screen.WifiScreen
import org.junit.Rule
import org.junit.Test
class WifiSampleWithStepsTest : TestCase() {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
run {
step("Open target screen") {
MainScreen {
wifiActivityButton {
isVisible()
isClickable()
click()
}
}
}
step("Check correct wifi status") {
WifiScreen {
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
wifiStatus.hasEmptyText()
checkWifiButton {
isVisible()
isClickable()
click()
}
wifiStatus.hasText(R.string.enabled_status)
device.network.toggleWiFi(false)
checkWifiButton.click()
wifiStatus.hasText(R.string.disabled_status)
}
}
step("Rotate device and check wifi status") {
WifiScreen {
device.exploit.rotate()
wifiStatus.hasText(R.string.disabled_status)
}
}
}
}
}
Let's take a look at the logs. There we see both logs of specific actions and checks performed during the test, as well as meta-information about the steps - name, calling class, completion time, success message, or error log if the test failed.
Any user of the Kaspresso framework can add the necessary improvements to the functionality of the Steps: recording steps in the allure report, screenshots taken at the moment of step failure, view hierarchy output, and much more. To do this, the user needs to write their own interceptor or use a ready-made one.
Let's run the test again with the internet turned off:
Sections - preparing states before and after the test
Great, but there's something else to take into account. Often, when we compile tests, we have to consider the necessity of checking the application performance under certain preconditions. So in our test, we need the device to be in the default state before each launch, and to return to the same state after the test is completed.
To do this, there are before and after blocks in Kaspresso. The code inside the before block is run before the test - we can set the default settings here. During the test, the phone state may change: we can turn off the internet or change the orientation mode, but after the test, we need it to return to its original state. And that's where the after block comes in.
Let's improve our test class by using sections. Before starting the test in the before section, let's set the landscape mode and turn the Wifi on:
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
device.network.toggleWiFi(true)
The before and after blocks help not only to visually separate the setup of the desired state into separate blocks but also to guarantee the completion of these blocks. If we look at the test that we wrote at the beginning of this article using Espresso, we can see that the same actions were recorded as steps, and if the test fails abruptly, the default settings may not be restored. Using before and after sections ensures that these blocks are run at the proper time.
Also note that now, after flipping the device, we check if the text remains the same, but not if the orientation mode has actually changed. It appears that if the device.exploit.rotate() method did not work for some reason, the orientation mode does not change and there is no point checking the text. Let's add a check to verify whether the orientation mode of the device has changed to landscape.
Assert.assertTrue(device.context.resources.configuration.orientation == Configuration.ORIENTATION_LANDSCAPE)
Now the full test code looks like this:
package com.kaspersky.kaspresso.tutorial
import android.content.res.Configuration
import androidx.test.ext.junit.rules.activityScenarioRule
import com.kaspersky.kaspresso.device.exploit.Exploit
import com.kaspersky.kaspresso.testcases.api.testcase.TestCase
import com.kaspersky.kaspresso.tutorial.screen.MainScreen
import com.kaspersky.kaspresso.tutorial.screen.WifiScreen
import org.junit.Assert
import org.junit.Rule
import org.junit.Test
class WifiSampleWithStepsTest : TestCase() {
@get:Rule
val activityRule = activityScenarioRule<MainActivity>()
@Test
fun test() {
before {
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
device.network.toggleWiFi(true)
}.after {
device.exploit.setOrientation(Exploit.DeviceOrientation.Portrait)
device.network.toggleWiFi(true)
}.run {
step("Open target screen") {
MainScreen {
wifiActivityButton {
isVisible()
isClickable()
click()
}
}
}
step("Check correct wifi status") {
WifiScreen {
wifiStatus.hasEmptyText()
checkWifiButton {
isVisible()
isClickable()
click()
}
wifiStatus.hasText(R.string.enabled_status)
device.network.toggleWiFi(false)
checkWifiButton.click()
wifiStatus.hasText(R.string.disabled_status)
}
}
step("Rotate device and check wifi status") {
WifiScreen {
device.exploit.rotate()
Assert.assertTrue(device.context.resources.configuration.orientation == Configuration.ORIENTATION_LANDSCAPE)
wifiStatus.hasText(R.string.disabled_status)
}
}
}
}
}
Conclusion and useful references
So, let's summarize. The open-source Kaspresso framework allows you to make test code readable due to a declarative approach using Kotlin DSL and an "out-of-the-box" implemented Page Object pattern, and it also allows you to make it stable due to internal improvements and interceptors. Here are some advantages of Kaspresso, highlighted by us using a specific example:
- Good readability. You no longer need to use long constructs with matchers to search for elements on the screen for interaction from the test, and all the UI elements that your test interacts with can be described in one place (in a specific Screen object).
- Logging. Kaspresso provides detailed and intuitive logs indicating the current step. You don't need to add them manually, because everything is implemented internally. If necessary, you can change and supplement the logs as you wish.
- Code architecture. Using the Page Object pattern implementation described above, you can make your code in test files more readable, maintainable, reusable, and intuitive. Kaspresso also provides various methods and abstractions to improve the architecture (such as Steps, before and after sections, and more).
- Low entry threshold for mastering the framework. No programming language knowledge is required to write autotests; all you need to do is use the declarative approach.
If this article was useful to you and you plan to use Kaspresso in your projects, join the Kaspresso community on Discord. We will try to provide you with all the necessary support and announce the following articles from the current series of materials about Kaspresso.
You can also take the Tutorial prepared by the Kaspresso team to familiarize yourself with other features of the framework and visit the Wiki section. We hope to see you among our contributors soon!
Furthermore, we welcome you to read our previous publications about autotests on Android:
A step-by-step tutorial in codelab format for Android UI testing
And don't forget to give us a star on Github ;)
Top comments (0)