Make Dart/Flutter test report readable

#dart #flutter #testing

Hello there! I will try to be short.

Intro

Short

Flutter test report is not human-friendly.

Long

On my personal Flutter project, I’ve started the setup of Continuous Integration & Continuous Deployment. And in Flutter test report is difficult to use. The report is hard to read & understand in case of failure. Sure, in case of success, everything is good, because there is just one line.

Flutter has big error stack traces. When it has some exception not in your codebase, but in Flutter itself. I mean errors with positioning or sizing, and for them, you will see a few hundreds of lines in logs.

In the front-end development I like the dot notation usually used for unit tests. Dot notation is cool because it’s clear and you can identify skipped and failed tests easily. Such feature useful when you have over 500 tests in your project. But in Dart, there is no such thing, only default and machine reporter.

What to do?

Short

Here is a tool, I’ve created: dart_dot_reporter which makes report readable.

Long

dart_dot_reporter tool transforms machine results to the more readable and clear output. And here is an example:

If you are not familiar with Dot notation

. Dot for passed test
! Warning that test was skipped
X for Failed test

The summary line of icons will be displayed at the start of the report. And in case of skip or failure, it will display the name of the test after the icon on the new line.

I implemented a few features out of the box. If you want - you can:

hide skipped tests
show successful tests
append few error details after test name in case of error
display id number of the test (which is like the order number)
disable color

Also, later I plan to display top of tests which execution took too much time.

Alternative

In some cases, you can find a JUnit report also useful. junit
But in my case, I wanted to have a possibility to understand which test failed from logs or email/slack message with a fast look on the report.

Finally

And you can see the example of that tool usage here in “Run tests” section. Because I use dart_dot_reporter on CI for itself with GitHub actions, which I found useful for personal projects.

If you find some issues - write me a note or create a bug on GitHub.

Top comments (4)

SimoneB • Aug 9 '21

Hey Aleksey,

I was trying today to add this package to the project and I would like to be able to run just integration tests with it.
Unfortunately there is little documentation how to do so. If you follow the guide, it will run all the tests (we have unit, screenshot and integration).
Any idea how to run only integration?

this is how we run just integration tests:

flutter drive \
--driver=test_driver/integration_test.dart \
--target=integration_test/regression_test_runner_test.dart

cuberob • Apr 30 '20

Just wanted to chime in and say thanks! Makes my test look much cleaner and makes it way less of hassle to figure out which test is causing my build to fail. Cheers!