DEV Community

Cover image for Best tips & tricks for E2E Maestro with React Native
Davyd NRB
Davyd NRB

Posted on • Edited on

Best tips & tricks for E2E Maestro with React Native

#reactnative #maestro #testing

A collection of practical tips and techniques that simplify your developer experience of Maestro E2E framework!

1. Installing Maestro

Let's add a new script to the package.json file:

{
  "scripts": {
    "install-maestro": "MAESTRO_VERSION=1.37.0 curl -Ls 'https://get.maestro.mobile.dev' | bash"
  }
}
Enter fullscreen mode Exit fullscreen mode

I define the version we now support using the environment variable MAESTRO_VERSION. The use of the same version by our team and CI is guaranteed.

docs: https://maestro.mobile.dev/getting-started/installing-maestro

2. Identify the app as being run by Maestro.

When an app is run by e2e, we occasionally need to configure the behavior of the app:

  1. Disable animation / pause video;
  2. Stop sending analytics;
  3. Turn off React Native LogBox LogBox.ignoreAllLogs().

For this case you need to add arguments to a launchApp command

- launchApp:
    appId: "com.example.app"
    arguments: 
       isE2E: "true"
Enter fullscreen mode Exit fullscreen mode

And then on JS side using react-native-launch-arguments package you can access that arguments

import { LaunchArguments } from "react-native-launch-arguments";

LaunchArguments.value().isE2E // true
Enter fullscreen mode Exit fullscreen mode

3. Dot env file support .env

You need to create .env file with next text:

-e UNIVERSAL_USERNAME=user-e2e@test.com
-e UNIVERSAL_PASSWORD=123qwe
Enter fullscreen mode Exit fullscreen mode

After that you can use that file as argument for test or record commands:

{
  "scripts": {
    "record": "$HOME/.maestro/bin/maestro record @.env"    
    "test": "$HOME/.maestro/bin/maestro test @.env"
  }
}
Enter fullscreen mode Exit fullscreen mode

Additionally, you can combine various env files (maestro test @.env @.env.prod), in which case all variables from the most recent file will replace, extend, or reset env variables from earlier files.

"test": "$HOME/.maestro/bin/maestro test @.env"
"test-prod": "yarn test @.env.prod"
"test-staging": "yarn test @.env.staging"
Enter fullscreen mode Exit fullscreen mode

Example:

# .env file:
-e BACKEND=staging
-e USERNAME=test@test.com
-e PASSWORD=123qwe
#-----------------
# .env.prod
-e BACKEND=                 # unset variable
-e USERNAME=myprod@test.com # edit existing 
-e SKIP_UNBOARDING=true     # add a new variable
Enter fullscreen mode Exit fullscreen mode

The following values will be available in the Maestro environment:

${BACKEND}         // null
${USERNAME}        // "myprod@test.com"
${PASSWORD}        // "123qwe"
${SKIP_UNBOARDING} // "true"
Enter fullscreen mode Exit fullscreen mode

4. Different appId between platforms or environments

It's common practice to differentiate production apps from non-production (staging/beta) apps using a Bundle ID (for example: com.example.myapp & com.example.myapp.staging)

# .maestro/my-test.yaml
appId: ${APP_ID}
name: My test name
---
- launchApp # used an `appId` specified above
Enter fullscreen mode Exit fullscreen mode

Then using -e MY_VAR=myVal you can pass APP_ID in each test:

{
  "scripts": {
    "test": "$HOME/.maestro/bin/maestro test"
    "test-android-staging": "yarn test -e APP_ID=com.example.myapp.staging"
    "test-android-prod": "yarn test -e APP_ID=com.example.myapp"
  }
}
Enter fullscreen mode Exit fullscreen mode

and finally, an example of a terminal command:

yarn test-android-staging .maestro/my-test.yaml
# ^^^ Run test using staging app

yarn test-android-prod .maestro/my-test.yaml
# ^^^ Run test using prod app
Enter fullscreen mode Exit fullscreen mode

5. Filtering tests to run using tags

You have the ability to run a particular set of tests using tags in Maestro.

Your nightly builds, for instance, frequently verify the following crucial features:

appId: com.example.myapp
name: Sign-in email+password
tags:
  - "on:pre_release"
  - "on:pull_request"
  - "on:nightly_build"
  - "feature:auth"
  - "backend:prod"
  - "backend:pre_prod"
  - "backend:staging"
---
- launchApp
# ...
Enter fullscreen mode Exit fullscreen mode

You can use --include-tags="on:nightly_build" and path to your directory with yaml files to run all test that include an on:nightly_build tag:

{
  "scripts": {  
    "test": "$HOME/.maestro/bin/maestro test"
    "nightly-test": "yarn test --include-tags='on:nightly_build' ./path_to_flows"
    "pr-test": "yarn test --include-tags='on:pull_request' ./path_to_flows"
  }
}
Enter fullscreen mode Exit fullscreen mode

6. Complicated waitFor logic

I had the issue that Maestro couldn't wait for the appearance of the "A" or "B" elements.

But using a repeat & runFlow we can simulate this logic:

- evalScript: ${output.myStatus = 'unknown'}
- evalScript: ${output.attemptsCount = 0}
# ^^^ specify initial parameters

- repeat:
    while:
      true: ${output.myStatus === 'unknown'} 
    # run until the status changes
    commands:
      - runFlow:
          when:
            visible: "Booking failed"
          commands:
            - evalScript: ${output.myStatus = 'error'}
      # ^^^ first `runFlow` wait for an error case
      - runFlow:
          when:
            true: ${output.myStatus === 'unknown'}
          commands:
            - runFlow:
                when:
                  visible: "Booking success"
                commands:
                  - evalScript: ${output.myStatus = 'success'}
      # ^^^ second nested `runFlow` will wait for a success case

      # The remaining logic deals with a timeout case
      - runFlow:
          when:
            true: ${output.attemptsCount > 10} # Check attempt limit 
          commands:
            - evalScript: ${output.myStatus = 'timeout'}
      - evalScript: ${output.attemptsCount = output.attemptsCount + 1} # Incerunmen an attempt counter


# ^^^ After that you can run any logic based on `output.myStatus`
# for example throw an error if status isn't `success`
- assertTrue: ${output.myStatus === 'success'}
Enter fullscreen mode Exit fullscreen mode

7. Useful bash commands

Android Debug (required to run metro in background)

./android/gradlew assembleDebug -p ./android # build debug apk
find ./android -type f -name "*.apk"         # find apk file
yarn start                                   # run metro bundler
adb reverse tcp:8081 tcp:8081                # open port
adb install "<path_to_apk_file>"             # `path_to_apk_file` - result of `find` command above
Enter fullscreen mode Exit fullscreen mode

Android Release (bundle JS with apk)

./android/gradlew assembleRelease -p ./android # build debug apk
find ./android -type f -name "*.apk"           # find apk file
adb install "<path_to_apk_file>"               # `path_to_apk_file` - result of `find` command above
Enter fullscreen mode Exit fullscreen mode

8. Run on CI using GitHub Actions

⚠️ If you want to run Maestro test using GitHub Actions on Android you can't use default ubuntu runners as nested virtualization is disabled (that required by Android Emulator).

See run-on: * table:

Runners Android iOS Price (min) Spec
large ubuntu ✅ ⛔ from $0.016 to $0.256 4-64CPU 15-256GB RAM
ubuntu-* ⛔ ⛔ $0.008 2CPU 7GB RAM
custom buildjet ✅ ⛔ from $0.004 to $0.048 2-34CPU 8-64GB RAM
macos-*-xl ✅ ✅ 0.32$ 12CPU ??GB RAM
macos-* ✅ ✅ 0.08$ 3CPU 14GB RAM

Simple Github Actions workflow .github/workflows/mobile-e2e.yml to run Android & iOS e2e test using Maestro:

  • no caching
  • no versionlock (xcode,java,nodejs,cocoapods) 
name: Maestro E2E

on: [push]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true # auto cancel prev. run

env:
  MAESTRO_VERSION: 1.37.0
  ANDROID_ARCH: x86_64

jobs:
  ios_e2e:
    name: iOS
    runs-on: macos-12 # or macos-12-xl
    steps:
      - uses: actions/checkout@v3

      - name: Installing Maestro
        run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env

      - name: Installing Maestro dependencies
        run: |
          brew tap facebook/fb
          brew install facebook/fb/idb-companion

      - name: Install node_modules
        run: yarn install --frozen-lockfile

      - name: Install Pods
        working-directory: ios
        run: pod install

      - name: Build app for simulator
        working-directory: ios
        env:
          DERIVED_DATA_PATH: my_build
        run: |
          xcrun xcodebuild \
            -scheme "Myapp" \
            -workspace "Myapp.xcworkspace" \
            -configuration "Release" \
            -sdk "iphonesimulator" \
            -destination "generic/platform=iOS Simulator" \
            -derivedDataPath "${{ env.DERIVED_DATA_PATH }}"

          echo "Print path to *.app file"
          find "${{ env.DERIVED_DATA_PATH }}" -type d -name "*.app"
      # ^^^ Path to *.app file (based on derivedDataPath + working-directory):
      # ./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app

      - name: Run e2e tests
        env:
          APP_PATH: "./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app"
                    # ^^^ change this path to your *.app file
        run: |
          echo "Launching iOS Simulator"
          xcrun simctl boot "iPhone 14 Pro"

          echo "Installing app on Simulator"
          xcrun simctl install booted "${{ env.APP_PATH }}"

          echo "Start video record"
          xcrun simctl io booted recordVideo video_record.mov & echo $! > video_record.pid

          echo "Running tests with Maestro"
          $HOME/.maestro/bin/maestro test .maestro/ --format junit

      - name: Stop video record
        if: always()
        run: kill -SIGINT $(cat video_record.pid)

      - name: Store video record
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: e2e_ios_report
          path: |
            video_record.mov
            report.xml

  android_e2e:
    name: Android
    runs-on: macos-12 # or buildjet-4vcpu-ubuntu-2204, ubuntu-22.04-4core, macos-12-xl
    steps:
      - uses: actions/checkout@v3

      - name: Installing Maestro
        run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env

      - name: Install node_modules
        run: yarn install --frozen-lockfile

      - name: Build apk for emulator
        working-directory: android
        run: |
          ./gradlew assembleRelease --no-daemon -PreactNativeArchitectures=${{ env.ANDROID_ARCH }}

          echo "Print path to *.apk file"
          find . -type f -name "*.apk"

      - name: Install Maestro and run e2e tests
        uses: reactivecircus/android-emulator-runner@v2
        env:
          APK_PATH: ./android/app/build/outputs/apk/release/app-release.apk
                    # ^^^ change this path to your *.apk file
        with:
          api-level: 33 # Android 13
          arch: ${{ env.ANDROID_ARCH }}
          script: |
            adb install "${{ env.APK_PATH }}"
            $HOME/.maestro/bin/maestro test .maestro/ --format junit

      - name: Store tests result
        uses: actions/upload-artifact@v3
        with:
          name: e2e_android_report
          path: |
            report.xml
Enter fullscreen mode Exit fullscreen mode

Top comments (2)

Collapse
 
sregg profile image
Simon Reggiani •

Very useful. Thanks a lot for the article 🙏

Collapse
 
timisthinking profile image
Tim •

Thanks for the tip on passing environment variables from a .env, super helpful!

Read next

dsychin profile image

Lessons Learned Migrating from React CRA & Jest to Vite & Vitest

Desmond Chin -

danrot90 profile image

Writing high quality tests

Daniel Rotter -

signadot profile image

Why Quick Fixes Fail: Rethinking Microservices Testing

Signadot -

pauldmooney profile image

Enhancing the Developer Experience of Testing Part 2

Paul Mooney -