UI testing in Android has always been controversial for many reasons. Tests are slow because they must run in emulators or real devices. They can be flaky for many different reasons. Test code is usually hard to read and maintain because of Espresso and Android APIs. UI testing is a pain in the ass, and it will continue to be, I’m sorry.
But still, in my experience, these kinds of tests are the most useful ones for Android development. While unit tests help us during development, UI tests give us peace of mind during releases, knowing that the most important features of the app won't just stop working for everybody.
In this post, I want to share some of the work we’ve been doing at InfoJobs the past few years to make UI testing less painful.
Our main efforts went to
- Reduce flakiness
- Make tests more semantic
- Write simpler and maintainable tests
Mandatory disclaimer: There are many approaches you can take to UI testing, and most of them are right. I’m not going to tell you how you should do it. I’m going to show what has worked for us and how we do it, so you can hopefully take something from it.
Beginning with UI tests and Espresso
Espresso is Google's official UI testing framework. It can perform actions and check conditions on views. So a really simple UI test could look like this:
@Test
fun mySimpleTest() {
onView(withId(R.id.greet_button)).perform(click())
onView(withText("Hello Steve!")).check(matches(isDisplayed()))
}
But in our day-to-day, we want to cover more complex features and flows involving multiple screens. So a real test could become something like:
@Test
fun shouldShowRecommendedOffers_whenComingToHomeBackAfterLogin() {
// Perform search
onView(withId(R.id.toolbar)).perform(click())
onView(withId(R.id.keyword_field)).perform(replaceText("Java"))
onView(withId(R.id.search_button)).perform(click())
// Log in
onView(withId(R.id.searchResultLoginButton)).perform(click())
onView(withId(R.id.loginEmailField)).perform(replaceText("email@example.com"))
onView(withId(R.id.loginPasswordField)).perform(replaceText("password"))
onView(withId(R.id.loginButton)).perform(click())
// Go back
onView(isRoot()).perform(pressBack());
// Assert header text
onView(withText("10 ofertas recomendadas")).check(matches(isDisplayed()))
}
It’s not the prettiest code, but it works. This is our starting point. Let me tell you what we did to improve it from here.
Barista
Espresso is ugly
A bit of history. When we started using Espresso to write our UI tests years ago we had to learn its API. And it wasn’t pretty. Don’t get me wrong, it’s very powerful and flexible. We can do almost anything to views. But it’s not pretty. The code above is barely readable.
Tip: Check out this awesome cheat sheet in the official docs to understand how the Espresso API works.
At the time, a couple of backend engineers were learning Android development and working on some features for the app. They wanted to cover the features with UI tests, but having to learn and remember the syntax was a problem. So they suggested adding static methods for common actions that they used over and over again. Like clickOn(R.id.button)
and assertDisplayed(“Some text”)
.
I must confess that I didn’t agree with this idea at the time. I believed that we should get used to the Espresso API so we could understand how it works and do more complex things when needed. Luckily my friend Roc was also on the Android Team and convinced me to play along. So we tried it for a while. And oh boy, was I wrong. It was a great decision, it improved a lot the readability of our tests.
Espresso is dumb
Ok, maybe “dumb” is unfair. Let’s say it’s “strict”. For example, we would often need to interact with elements inside scrolls. Espresso fails if you perform an action on a View below a scroll. So we need to tell it to scroll first.
fun click(@IdRes id: Int){
onView(withId(id)).perform(scrollTo(), click())
}
But there is a catch! Espresso will fail if the view is not inside a ScrollView. Bummer!
From the point of view of the test, I don’t care if the view is inside a scroll or not, I just want the test to click it as if it was an user using the app. So we came up with a trick:
fun click(@IdRes id: Int){
try {
onView(withId(id)).perform(click())
} catch (e: PerformException) {
onView(withId(id)).perform(scrollTo(), click())
}
}
Disclaimer
This is a naïve approach, but it's a quick-win. It can be improved in many ways, but it was our first iteration and it worked for us.
Another common example is having a ViewPager with the same layout on multiple pages.
A simple click()
action will fail with
androidx.test.espresso.AmbiguousViewMatcherException: 'with id: id/button' matches multiple views in the hierarchy.
As you see, Espresso is strict. If it gets confused and doesn't know what to do, it will fail. It forces you to be more explicit about your actions. There's nothing wrong with that, it's a valid approach. But we were comfortable assuming that, if there are multiple matching views, we only want to click the one that is displayed:
fun click(@IdRes id: Int){
// I omitted the try/catch from the previous example for brevity
onView(allOf(isDisplayed(), withId(id))).perform(click())
}
This kind of code could be considered a bad practice in production code. But in testing code we found it makes our tests simpler and more robust since they would not depend on variable things like the screen size or future layout changes. It was totally worth it!
With this same philosophy we created a class named EspressoUtils
, with many more similar static methods. It kept growing as we added new utilities for new scenarios we wanted to test. Over time we noticed that even long tests were much more readable using our custom methods than the Espresso API, but more importantly, easier and faster to write.
We added click(id)
, click(stringRes)
, openDrawer()
, acceptDialog()
, assertDisplayed("String")
, and many more. Writing new UI tests with EspressoUtils
was a delight. We would copy and paste this file internally from project to project until we realized we should share it with everyone as a library. So we moved it to an independent project, renamed it to Barista (great name Sergi!), designed a beautiful logo (thanks Diego!), added tests for all the interactions and assertions (thanks Roc for so many hours!) and published it. Barista was born!
Introducing Barista
Using Barista we removed a lot of the boilerplate from Espresso, making the code easier to read and write. Time has passed and Barista has grown a lot. Today it contains many more interactions and assertions. By making acceptable assumptions it is smarter, it can decide to scroll the screen when needed, it can deal with ViewPagers that contain similar layouts, it interacts with ListView and RecyclerView indistinctively, it allows making custom assertions with a couple of lines thanks to assertAny
, it contains some useful Test Rules to fight flaky tests like cleaning Shared Preferences and Databases between tests, and more. Take a look at the Readme, you will be surprised by how much stuff it can do with little code.
AdevintaSpain / Barista
☕ The one who serves a great Espresso
Barista
The one who serves a great Espresso
Barista makes developing UI tests faster, easier, and more predictable. Built on top of Espresso, it provides a simple and discoverable API, removing most of the boilerplate and verbosity of common Espresso tasks. You and your Android team will write tests with no effort.
Download
Import Barista as a testing dependency:
androidTestImplementation('com.adevinta.android:barista:4.2.0') {
exclude group: 'org.jetbrains.kotlin' // Only if you already use Kotlin in your project
}
You might need to include the Google Maven repository, required by Espresso 3:
repositories {
google()
}
Barista already includes espresso-core
and espresso-contrib
. If you need any other Espresso package you can add them yourself.
We are super proud to have reached 1.2K stars in GitHub and 1 million downloads in Bintray. And it makes us especially happy seeing a lot of people in the community using, contributing and enjoying it.
But the story doesn't end here. Keep reading to see what else we did to improve our tests.
Page Objects
Barista was just the tip of the iceberg. It’s not just a useful library. It changed our approach to UI testing, it gave us the mindset of writing simpler and easier to read tests. Page Objects are a step in a similar direction, they make test code more semantic and easier to write.
The idea of using Page Objects started many years ago. QA engineers and web developers were used to a concept named “Fragment”. Basically, they were Java abstractions over the actions and assertions that could be done on each page from the perspective of the user (yea, not those fragments). When some backend engineers started working on the Android app they imported those ideas to our shiny Espresso tests. We later learned (thanks to Pedro) that this pattern was already a thing called Page Object. Great! It wasn’t a bad idea after all! 😃
(image from https://martinfowler.com/bliki/PageObject.html)
Our implementation of these Page Objects has evolved over time. We now call them Screen Objects, but that's not relevant. After many iterations and agreements among our Android Team, the result looks like this:
@Test
fun shouldShowRecommendedOffers_whenComingBackAfterLogin() {
HomeListScreenObject()
.clickSearchToolbar()
.search("Java")
.clickFloatingLoginButton()
.fillCredentialsAndLogIn()
.assertThat { headerContains("10 ofertas recomendadas") }
}
Example implementation (simplified)
class HomeListScreenObject : NavigationScreenObject() {
fun clickSearchToolbar(): SearchScreenObject {
clickOn(R.id.toolbar)
return SearchScreenObject()
}
fun assertThat(assertionBlock: HomeListScreenAssertions.() -> Unit): HomeListScreenObject {
HomeListScreenAssertions().assertionBlock()
return this
}
}
class HomeListScreenAssertions : BaseScreenAssertions(R.id.homeListRoot) {
fun headerContains(text: String) {
assertDisplayed(text)
}
}
class SearchResultScreenObject : BaseScreenObject {
fun clickFloatingLoginButton(): LoginScreenObject {
clickOn(R.id.searchResultLoginButton)
return LoginScreenObject()
}
fun clickFilters(): SearchFiltersScreenObject {
clickOn(R.id.searchResultFilterButton)
return SearchFiltersScreenObject()
}
fun assertThat(assertionBlock: SearchResultScreenAssertions.() -> Unit): SearchResultScreenObject {
SearchResultScreenAssertions().assertionBlock()
return this
}
}
class SearchResultScreenAssertions : BaseScreenAssertions(R.id.searchResultRoot) {
fun createAlertButtonIsDisplayed() {
assertDisplayed(R.id.searchResultCreateAlertButton)
}
fun searchBarContains(text: String) {
assertAnyView(
viewMatcher = withParent(withId(R.id.toolbar)),
condition = withText(text)
)
}
}
The test code can be read out loud to any person following those steps in front of the app and they would understand what to do. You don’t see any Espresso code or Android resources ID. You only see concepts from the user perspective.
Here are some general rules that we follow:
- One PageObject per screen: There is a Page Object class for each screen in our app. The concept of Screen is subjective, it can be an Activity, a Fragment inside the BottomBar, a complex dialog, etc. When in doubt we ask ourselves: “what does the user perceive as a screen in this case?”.
-
Actions: A PageObject contains functions for the actions that the user can perform on the current screen. They can be very concrete like
clickSaveButton()
, or wider likefillFormFields()
. They can also contain parameters, likesearch("Java")
. - Navigation: Each function returns the PageObject of the next screen after the action is performed. If the user stays on the same page, it returns itself. If the user navigates to a different screen, it returns the PageObject of that screen. This way we can fluently write actions across multiple screens, with the help of the IDE’s autocomplete.
-
Assertions: We have an
assertThat
builder function in every PageObject that returns the Assertions of this PageObject. This Assertions object contains functions to assert different things on the screen. - Be pragmatic: We don’t write every possible action or assertion for every screen. We add them as we need them. Remember that this is still code that must be maintained.
No fancy framework here, we try to keep it as simple as we can. These abstractions let us write UI tests really fast and keep them very semantic and readable. Modifying a feature or testing a new one is a breeze.
Barista is hidden in the implementation of the PageObject functions, so we still benefit from its magic. Sometimes we'll implement more complex actions or assertions that cannot be written with Barista and we'll write some custom Espresso code. Barista and Espresso are not exclusive!
Utilities
The last piece of our UI testing story is a series of smaller utilities that we've been adding over time. We grouped them in a InstrumentationTest
interface, and I'm going to show some of them because I consider they help a lot with test readability.
Launch
This function takes an Intent
and a lambda block, launches the Activity with the Intent, runs the code from the lambda, and then closes the Activity.
If it sounds awfully familiar to an ActivityScenario
, it is because our launch
function is just a wrapper around ActivityScenario
to make the code a little bit easier to read and write.
@Test
fun openHome() {
givenCountryIsSelected()
givenUserIsNotLogged()
launch(intentFactory.home.create(appContext)) {
assertDisplayed(R.id.homeOffersList)
}
}
Same code without the alias
@Test
fun openHome() {
givenCountryIsSelected()
givenUserIsNotLogged()
ActivityScenario
.launch<Activity>(intentFactory.home.create(appContext))
.use { scenario: ActivityScenario<Activity> ->
assertDisplayed(R.id.homeOffersList)
}
}
fun launch(intent: Intent, testBlock: ActivityScenario<Activity>.() -> Unit) {
ActivityScenario.launch<Activity>(intent).use { scenario ->
scenario.testBlock()
}
}
Before this, we were using the old ActivityTestRule
(now deprecated). The test code didn’t look too bad, but it had some design issues that have been solved thanks to ActivityScenario
. To mention some of its advantages, it provides fine-grained control of when the Activity gets launched and destroyed, we can easily launch different Activities on the same test, it lets you interact with the Lifecycle and even run code on the Activity’s Thread without sketchy tricks. There is an ActivityScenarioRule
, but I would personally recommend avoiding it in favor of ActivityScenario
, in order to have better control over the Activity or Activities in your test.
DeepLinks
Small and silly trick, but we added a launchDeepLink
function to simplify testing deeplinks, hiding boilerplate code and workarounds.
@Test
fun offerSearchOpensWithDeepLink() {
launchDeepLink("https://www.infojobs.net/ofertas-trabajo/barcelona") {
SearchResultScreenObject()
.assertThat {
screenIsDisplayed()
searchBarContains("Barcelona")
}
}
}
fun launchDeepLink(uri: String, testBlock: ActivityScenario<Activity>.() -> Unit) {
val deepLinkIntent = Intent(Intent.ACTION_VIEW, Uri.parse(uri))
// Component required because of a bug in ActivityScenario: https://github.com/android/android-test/issues/496
deepLinkIntent.component = ComponentName(appContext.packageName, DeepLinkActivity::class.java.name)
return launch(deepLinkIntent, testBlock)
}
IntentFactory
Another common task when dealing with Activities is creating Intents. In our app, we have an IntentFactory
class what we use to create Intents to any screen. This class is injected by Koin, because it might depend on other dependencies like Feature Flags or A/B tests.
By having a commong InstrumentationTest
interface we provide an easy shortcut instead of manually injecting it on every test eveery time. Again, a very small improvement that adds up!
interface InstrumentationTest {
val intentFactory
get() = KoinContextHandler.get().get<IntentFactory>()
// ...
}
Rules
I'm sure you usually apply some JUnit rules to your instrumentation tests, don't you? We can apply a bunch of common test rules to all our tests. And we have many of them!
@get:Rule
val instrumentationRules: TestRule
get() = RuleChain.outerRule(
FlakyTestRule().allowFlakyAttemptsByDefault(FLAKY_ATTEMPTS))
// ↓ All rules below flakyTestRule will be repeated if the test fails
.around(FinishAllActivitiesRule())
.around(DisableAnimationsRule())
.around(IdlingUseCaseRule())
.around(ReloadKoinRule())
.around(ClearFilesRule().includeFilesMatching("/(.*).json"))
.around(ClearCacheRule())
.around(ClearPreferencesRule())
.around(ClearDatabaseRule())
.around(ResetLocaleRule())
.around(RESTMockRule())
.around(ScreenshotOnFailureRule())
private val FLAKY_ATTEMPTS: Int
get() = if (BuildConfig.IS_CI) 7 else 1
And more!
I can't list every single little helpful feature we have for our tests, but I'd say these are the most impactful ones. Mocking the API, testing custom views, testing event tagging or screenshot testing to name a few. Maybe we'll share some more of these in the future!
The takeaway from this last part is that there is a lot of verbosity associated with UI tests that can be hidden away to make them easier to read, write and modify. These small things only save a few lines of code in each test. But they work together to make you forget about the testing framework, focus on what you really care about in the test, and let the existing infrastructure handle the rest.
Conclusion
UI tests can be a pain sometimes. But investing some time into them can make your life much easier, you'll soon find yourself writing new tests without effort.
I showed many different things here. You might like some, you might dislike others. I told our story because context is important. Find what works best for you and your team, and don't be afraid to iterate your solutions.
Writing a test for a button: 4 minutes.
Finding why the button hasn't worked in production for a week because you didn't write that test: 4 hours.
Write more UI tests! Don't trust Android!
Last but not least, I wrote this post but the contents and learnings are a product of many people that have worked on our Android Team. Thank you Roc, Jose, Rubén, Sergi, Bernat and many more contributors!
If you have any questions about the post or want me to expand on some part, please feel free to leave a comment below or ping me on Twitter @sloydev.
Thanks for reading!
Top comments (11)
Nice article.
You should also try Endtest for building and executing automated tests for Mobile Apps.
It uses an Appium engine and it even has a test recorder.
These chapters are a good starting point:
• How to create Mobile Tests
• Finding elements in Mobile Applications
• How to execute Mobile Tests
I actually work there, so feel free to ask me anything about it.
If Endtest is using Appium, I believe that's another topic. According to the official doc
Espresso is much more fast, and have lots of other advantages like very good at test RecyclerView.
Do you have any studies or data to support your claim that Espresso is faster than UiAutomator2?
I'm curious to learn more about that.
From google official training course
From 3rd introduction
Espresso was built by Google.
Your first source is literally from Google.
And your second source is an article published on Medium by someone.
None of those articles contain any comparison data.
I was hoping for something more reliable, that would include data, measurements, comparisons, etc.
I also see Kakao and Kaspresso as a good library to support write UI test. How does it compare to Barista?
I haven't personally used Kakao or Kaspresso, so take my response with a grain of salt. But from what I've seen about them, they seem more opinionated than Barista on how to write your tests. They are very linked to the concept of page object with a custom DSL, which can be helpful for some projects for sure.
Barista is more "low level", closer to Espresso but with a simpler API and some hidden "magic" as described above. You decide how to structure your own tests. In this post, I showed how we do our PageObjects, but you're free to use any other approach.
In my personal and debatable opinion, our PageObjects are more semantic, because we write them from the perspective of what the user would be doing. However, the DSL from Kakao exposes some information about the structure of the screen (button this, recyclerview that). I'm not saying that's wrong, it's just different.
I hope I could shed some light on your question :)
Great job!
I want use Barista, but I have an issue:
Library Version:
3.7.0
Describe the Bug:
I followed the instructions to integrate Barista. I put this lines in my build.gradle
androidTestImplementation('com.schibsted.spain:barista:3.7.0') {
exclude group: 'org.jetbrains.kotlin'
}
But when I try to use it, Android Studio don't recognize the library.
What is my error?
I'm using Android Studio 4.1 and com.android.tools.build:gradle 4.0.1
I see in Github you already solved it :)
Hi guys!
Barista is an interesting solution to make Espresso tests easier. But, there is a more modern and perfect solution - Kaspresso. Check it =)
Hi, I was wondering could you elaborate on why you think Kaspresso is more modern and perfect?