Lock your Android dependencies 🔐

At Swile we had several crashes and regressions caused by hidden dependency changes. We ended up setting a simple dependency lock mechanism to prevent future bugs.

Dependencies locking is a well known mechanism. Web developers have done this for years (package-lock.json or yarn.lock). iOS (Podfile.lock) and Ruby (Gemfile.lock) devs too just to name a few.

In this article we will see why this tool is important and how we can use it in our Android projects.

TLTR

Here are the article key points :

Hidden dependency changes can happen even if you're meticulous.
Export the complete dependency graph to a dependencies.lock file and add it to your VCS. That way you know what exact versions are used and the relations between libraries.
Make your CICD fail the build if the file content is different than your actual dependencies.

🎁 You can find the code here!

So what is this all about?

Any project relies on lower-level or helper libraries to work. These are your project direct dependencies.
Recursively, such direct dependencies also have dependencies. From your project perspective, these are called transitive dependencies.

All of this forms a graph, called the dependency graph.

Declaring a direct dependency also means specifying its version. To do so, you can use fixed, ranged, prefixed or even no versioning. Here is an example of some common notations you can find in a build.gradle file

dependencies {
    implementation "com.google.firebase:firebase-core:20.0.0" // Fixed
    implementation "com.google.firebase:firebase-core:[20.0.0, 20.99.99]" // Ranged
    implementation "com.google.firebase:firebase-core:20.+" // Prefixed
    implementation "com.google.firebase:firebase-core" // Not constrained
...
}

Remember that your direct dependencies also have dependencies, and so on. Some may be referenced with fixed versions, some without. Some may be referenced once, other multiple times and in different versions!
See the problems coming? 😉

For Gradle to choose what version of each dependency to pick in the graph it has to go through a fairly complex process called dependency resolution before building your project. Out of the box Gradle will manage it for you so you won't be asked what to do when there is a conflict.

To see this list simply type ./gradlew app:dependencies. It will display the whole resolved dependency tree of your app module. You can easily see if Gradle fixed a conflict and chose another version.

 +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.30 -> 1.5.21

Here some library needed kotlin-stdlib in version 1.4.30. Another one needed version 1.5.21. So Gradle decided to pick the biggest version.

🎓 Locking your dependencies means writing down the exhaustive list of all direct and transitive dependencies and their exact resolved versions at a specific point in time.
That way we ensure that our project will be compiled, tested and released against these exact versions. When bumping a library, any hidden change will be explicit, giving you control and understanding.

But I already use fixed version numbers everywhere, are you sure I need to lock my dependencies? 🤷

A first problem comes from your transitive dependencies: you don't have control over their release process, what libraries and versions they are using.

The dependency graph may then contain the same library multiple times but in different versions.

Here is an imaginary build.gradle file:

dependencies {
    implementation "com.squareup.okhttp3:okhttp:4.9.2"
    implementation "com.datadoghq:dd-sdk-android:1.10.0"
...
}

If you look closely at Datadog SDK 1.10.0 dependencies, you will see that it requires okhttp:3.12.13

But when running ./gradlew app:dependencies you will find that Gradle has resolved the conflict by picking and compiling against the highest declared version of okhtpp.

+--- com.datadoghq:dd-sdk-android:1.10.0
     +--- com.squareup.okhttp3:okhttp:3.12.13 -> 4.9.2

🎓 Bumping one direct dependency version in your project, may force one of your transitive dependencies to use this version despite the fact it has not been tested against.

Here is another imaginary build.gradle file :

dependencies {
    implementation "com.onesignal:OneSignal:4.3.0"
...
}

If you look closely at OneSignal 4.3.0 dependencies, you will see that it requires work-runtime in a from version 2.0.0 up to 2.99.99.

When running ./gradlew :app:dependencies you will find that Gradle has resolved the version range by picking and compiling against the highest published version (at the time of your build) of work-runtime.

+--- com.onesignal:OneSignal:4.3.0
     +--- androidx.work:work-runtime:[2.0.0, 2.9.99] -> 2.5.0

But what will happen if Google releases a new version of work-runtime the next day? Gradle may pick it and change your dependency graph.
And what will happen if this version contains bugs or if OneSignal does not handle it well? Your app won't compile, work correctly or worse, will crash.

Sometimes a vulnerability is found in a library version. It is quick and easy to spot if your project has a direct dependency on this specific version. This is why tools such a Dependabot exist. However you won't be warned if this version appears in your transitive dependencies.

🎓 Because of version ranges, a new release of a dependency (direct or transitive) may cause Gradle to pick it without you noticing it

Okay I'm convinced, what can I do?

Gradle offers a dependency locking mechanism out-of-the-box. A Nebula Gradle plugin also exists.

As an alternative, in the next section of this post I will show you how to implement a basic locking mechanism. It is using the output of the ./gradlew app:dependencies command. This work was inspired by this Jake Wharton post.

Code has some rough edges but can be a great starting point for an integration in your project CICD pipeline!

Story begins with the generateCurrentDependencies Gradle task. It stores the output of the ./gradlew app:dependencies command to a temporary file.

This task is used by both generateDependenciesLockFile and compareDependencies tasks.

./gradlew generateDependenciesLockFile copies the content of the temporary file to your root project directory under a file called dependencies.lock

./gradlew compareDependencies fails if the content of the dependencies.lock file is different from the actual project resolved dependencies. A call to the diff command allows to quickly spot the differences. See Jake Wharton dependency-tree-plugin if you want to display better diffing results.

How can I use in my projects ?

First step would be to copy these tasks in a file and load it from your main build.gradle

apply from: file('gradle/dependenciesLock.gradle')

Next you have to generate the lock file and commit it to your VCS.
🎉 Congratulations 🎉 this is your baseline, you're now in control !

What happens next will depend on your workflow. If you have a CICD system, you can plug it in. By calling ./gradlew compareDependencies for each build, you will ensure to be warned each time a change has been made on one of your dependencies.
If you don't have such a system, you still can run those tasks manually after each dependency bump to check the impacts.

A good advice would be to bump one dependency after another, update the lock file then commit those changes. By doing atomic changes, impacts will be explicit and a regression easier to revert.

This is how we decided to proceed at Swile and so far we are fairly happy with it !

Useful links

Gradle dependency resolution
Gradle dependency version notation
Surfacing hidden changes - Jake Wharton
Dependency Tree Plugin - Jake Wharton