First impressions on Maestro - The simplest mobile testing framework

Photo by Mark Williams on Unsplash

Last week I participated in Droidcon NYC and outside the main conference room was a stand of mobile.dev. I usually don't stop at stands (unless I see some great t-shirts or socks 😄) but something caught my eyes.

It was this demo

What caught my eye was the simplicity of this framework. I have some experience with Espresso and I know how hard it is to write the same test in it.

So I promised myself to go and check it out when I return, and so I did.

In this post I'll share my first impressions and maybe this article will actually become a series 😄

Installation

Installation was actually a breeze, I head over to the guide and everything is explained there.

I decided to first install it locally, and since I want to test both Android and iOS I didn't install it into gradle but via homebrew.

Android

Running my first test on Android was easy.

I created a new file simple_login_test.yml that looks like this

appId: "ai.viz.notifier"
---
- launchApp:
    clearState: true

Then, I headed over to Android Studio to open an emulator.

And then I run this line from my terminal
maestro test simple_login_flow.yml

And that was it!
The app has launched!

iOS

For iOS though things have been a little bit harder.

I first needed to add another installation from homebrew

brew tap facebook/fb
brew install idb-companion

This is a library from Facebook that will allow us to connect to an iOS simulator.

After installing run this command in your terminal

idb_companion --list 1

it will list all of the available simluators, find one that you like (for me it was iPhone 13 mini) and copy its udid.

Then you will launch it with this command

idb_companion --boot 85921D77-E655-4FA4-B202-D705E66E0D8A

After it's launch you'll need to connect to it with this command

idb_companion --udid 85921D77-E655-4FA4-B202-D705E66E0D8A

Now you should be able to start your test.

So I created another file simple_login_flow-ios.yml with the same code just replacing the android's appId with a iOS' bundle id.

appId: "viz.ai.VizAINotifier"
---
- launchApp:
    clearState: true

But it didn't launch 😥

Apparently I had to shut down the Android emulator first, and only then it launched.

I guess they first try to find if there's an Android emulator and if there is they will try to launch on it. Then, it didn't find any app with that appId, so it just quite the test.

Running a simple flow

So the first flow we're going to run is a simple login flow. I want to launch the app, enter wrong credentials, see an error, fix the credentials and login successfully.

Simple, right?

Well, with other frameworks (Espresso ☕️ I'm looking at you 👀), this would take one class to set all the views to their IDs, another to launch the app and dozen functions or so to interact and assert it.

If you have to wait between actions, you'd have to also implement a timer to keep checking if your view is already displayed.

Not with maestro!

In here, to start the app all I need is this

appId: "ai.viz.notifier"
---
- launchApp:
    clearState: true

To tap a button to continue to the login screen, all I need is

# Welcome screen (yes, this is a comment)
- tapOn: GET STARTED

To do some assertions on views, to check that the views that suppose to be there are actually there, I needed

- assertVisible: "Email"
- assertNotVisible: "Password"
- assertNotVisible: "Reset Password"

The way that this works is that instead of looking for a view id, it looks for the text, the same way as your QA engineer will do it.

But what if you don't have a text? I wanted to assert if some icon was showing.

Fortunately, you can use IDs as well.

- assertVisible:
    id: "ai.viz.notifier:id/fingerPrintButton"

Maestro hierarchy

How did I get this id? Do I need yo open Android Studio and try to find it there? Hell no!

We want simplicity and simplicity will shall get 😄

To get all the views from a current screen, just run maestro hierarchy in your command line. The output will be all your view tree, with texts and IDs.

This is how I knew what ID to use here.

Other assertions

I did have one more thing I wanted to test.

In our app we disable the next button if you don't have text, this is part of the QA process and I wanted Maestro to do the same, but I didn't find any option to assert whether or not a view is clickable.

So I asked in their Slack channel whether they plan on having more assertions types.

And this is how they answered

Make sense!

So what I did was to tap on the button and check that the screen hasn't changed.

- assertVisible: "Email"
# Test next button is disable when no text entered
- tapOn: "Next"
- assertVisible: "Email"

iOS

I have other assertions in my file but I won't bore you with details 😄.

I now wanted to have the same test with iOS.

I started with the same test

appId: "viz.ai.VizAINotifier"
---
- launchApp:
    clearState: true
- assertVisible: "Enter your email"
- assertNotVisible: "Password"

But my first assertion failed. Although the text was there as a hint in the edit text, Maestro didn't find it.

So I posted another question in the slack channel.

Just 3 days after reporting, the issue got fixed!

This is an amazing pace for a library, and I think that this shows how hard they work to improve it and this is indeed a good sign.

But to test the icon that was there, I didn't test by its ID, in iOS it actually use the image id as the text, so I was able to test it like this,

- assertVisible: "sfIcon60"

Problems?

But then I got stuck, after the initial login flow, we have a view pager for a walkthrough. But no matter what I tried I wasn't able to test it.

Running maestro hierarchy on this screen gave me absolutely nothing.

orelzion@N9QFJYD6GW maestro % maestro hierarchy
{
  "attributes" : { },
  "children" : [ {
    "attributes" : {
      "text" : "VizDebug",
      "bounds" : "[0,0][375,812]"
    },
    "children" : [ ],
    "clickable" : null
  }, {
    "attributes" : {
      "resource-id" : "screen.onboarding.container",
      "bounds" : "[0,0][375,812]"
    },
    "children" : [ ],
    "clickable" : null
  } ],
  "clickable" : null
}

I was also trying to ask in the Slack channel, but this time, although they did try to help me, it just didn't work.

I'm guessing I'm gonna need to investigate that some more to understand what is so special about this screen 😄.

Wrap up

This post will probably have a follow up, the maestro framework is great! Super simple and intuitive, and I liked the philosophy of it.

This is defiantly a library that I will continue to investigate, and I look forward to integrate that into our projects.

Stay tuned!