Postbuild gradle commands in Buddybuild for Android

We’re currently using Buddybuild as our CI system for Firefox Focus for Android. It’s been a great solution for getting CI running with minimal hassle, and it also provides the infrastructure to quickly deploy builds for internal testing.

Although Buddybuild is usually simple to use, enabling static analysis and code coverage was a little bit tricky. Here are some notes which should be helpful if you want to set up similar tools (or even just arbitrary gradle commands) for your own buddybuilt Android projects.

Custom build commands

Buddybuild doesn’t appear to run a normal gradle build (“gradlew build”), instead it looks like it overrides your gradle setup and call the relevant apk building steps directly. Which is nice (you can select exactly which flavours should be built in the web UI), but it also limits you to running whatever Buddybuild thinks should be run. The web UI doesn’t offer custom gradle commands, it only allows disabling/enabling UI tests. (Apparently there is a code coverage option, but that’s not currently operational.)

This is where the “postbuild” hook, which lets you run arbitrary commands, comes in. All you need to do is insert your desired commands into buddybuild_postbuild.sh, and Buddybuild will run it automatically.

I tried adding ./gradlew findbugs there:

# buddybuild doesn't seem to offer any direct way of running findbugs.
# findbugs is run as part of |gradle build| and |gradle| check, but those
# aren't run directly in buddybuild.
./gradlew findbugs

By itself, that didn’t work, as detailed below.

(You’ll also want to insert any code coverage related or other static analysis commands here.)

Disabled Flavours

We have a sizeable matrix of build flavours, with multiple dimensions – in addition to multiple buildTypes. Some of those aren’t needed for now, so we disabled them using a variantFilter. It turns out Buddybuild ignores that variantFilter: the Buddybuild UI lets you select which flavours should be built (which is probably what necessitates removing variantFilters). That’s fine for Buddybuild, since they call the appropriate gradle commands to build only the desired and configured variants – but it means that calling gradlew findbugs will fail if there are unbuildable variants.

Of course: we had some unbuildable variants. I decided it was best just to make those buildable (those variants might be needed in future, hence fixing this issue wasn’t really a waste of time), but until I did that we got build failures associated with those supposedly disabled build variants.

Unbuildable variants probably aren’t a particularly common situation – our project is a bit special in that we wanted different sources for one module depending on one specific flavour dimension. Gradle doesn’t make that particularly easy, so we hadn’t bothered to make sure this worked for all flavours in the matrix. We just needed to sprinkle some gradle magic to ensure the full matrix could be built – we simply hadn’t seen any need to do this yet given that we weren’t shipping those variants.

As it turns out, I could’ve avoided that because of the solution I found for our next issue:

Unbuildable Buddybuild SDK

We enable the Buddybuild SDK for automatic updates and crash-reporting for our test builds – but we only enable that for our master builds (so we only saw this issue after landing on master). It looks like Buddybuild modifies your apps sources to add the SDK. The relevant dependencies aren’t accessible when running buddybuild_postbuild.sh, meaning you’ll see something like following error when running gradle commands from there:

/tmp/sandbox/workspace/app/src/main/java/org/mozilla/focus/FocusApplication.java:7: error: package com.buddybuild.sdk does not exist
import com.buddybuild.sdk.BuddyBuild; // This line isn't in our sources, huh?
^
/tmp/sandbox/workspace/app/src/main/java/org/mozilla/focus/FocusApplication.java:22: error: cannot find symbol
BuddyBuild.setup(this);  // Huh, again
^
symbol: variable BuddyBuild
location: class FocusApplication
2 errors
:app:compileFocusGeckoDebugJavaWithJavac FAILED

At this point I realised it would probably be easier to just revert all of Buddybuild’s changes. In other words, return to whatever state we build with locally. Since we’re using git, we added the following to the top of our buddybuild_postbuild.sh (replace with your choice VCS’s revert/reset commands as appropriate):

# buddybuild modifies our buildscripts and sources (this is partly to enable
# their SDK, and partly to allow selecting flavours in the BuddyBuild UI).
# We don't know where the Buddybuild SDK lives, which causes gradle builds
# to be broken (due to the SDK dependency injected into FocusApplication),
# it's easiest just to revert to a clean source state here:
git reset --hard

This should also fix the gradle variantFilter issues described previously, so I could’ve saved myself the effort of fixing our variants. Disclaimer: I haven’t actually tested with our flavour fixes reverted, don’t trust me on this…

Final postbuild script

#!/usr/bin/env bash
set -e # Exit (and fail) immediately if any command in this scriptfails

# buddybuild modifies our buildscripts and sources (this is partly to enable
# their SDK, and partly to allow selecting flavours in the BuddyBuild UI).
# We don't know where the Buddybuild SDK lives, which causes gradle builds
# to be broken (due to the SDK dependency injected into FocusApplication),
# it's easiest just to revert to a clean source state here:
git reset --hard

# buddybuild doesn't seem to offer any direct way of running findbugs.
# findbugs is run as part of |gradle build| and |gradle| check, but those
# aren't run directly in buddybuild.
./gradlew findbugs

./gradlew jacocoTestReport
bash <(curl -s https://codecov.io/bash) -t $CODECOV_TOKEN

# More here once we enable further tools

(The most up to date version can hopefully be found here.)

Further CI differences

Another difference that might exist is what (if any) tests are built and run: we only chose to run UI tests on master, and not on our development branches. Findbugs, at least in our configuration, runs over all compiled classes (including tests). Hence we only discovered some findbugs issues after enabling findbugs on master. That one is harder to debug, since we only saw the following in our logs:

:app:findbugs FAILED
FAILURE: Build failed with an exception.
 * What went wrong:
Execution failed for task ':app:findbugs'.
> FindBugs rule violations were found. See the report at: file:///tmp/sandbox/workspace/app/build/reports/findbugs/findbugs.html
 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED

Summary

1. Buddybuild modifies your gradle config and sources to enable flavours and to install the SDK (optional).
   Solution: run git reset --hard in buddybuild_postbuild.sh to get your tree back to an extended state.
2. master builds might be different due to the Buddybuild SDK (see 1), and also due to tests that are only run on master.
   Solution: enable the SDK and UI tests on a given branch before merging to master if you’re adding commands to buddybuild_postbuild.sh and/or adding static analysis tools. And make sure you’ve run the same tests locally (before running static analysis tools locally) in order to replicate the inputs that findbugs/etc will encounter in Buddybuild if you see failures on your branch.