Use the Gradle Plugin tasks
The Testify Android Gradle Plugin offers a suite of tasks to conveniently integrate Testify into the normal Android development workflow. It integrates seamlessly with existing test suites, providing tools for running, recording, and managing screenshot tests. The plugin enhances testing efficiency by simplifying device and emulator configurations and supports Android Studio integration for a streamlined workflow. For more details and specific instructions, please refer to the Recipe Book.
Configuration
Applying the Testify Gradle Plugin to your projects automatically configures and includes the Testify Screenshot Library as an androidTest
dependency on your project. The Testify Library and the Gradle Plugin are both configurable via the testify {}
closure in your build.gradle
file.
Gradle extension
Testify supports the following configuration options:
autoImplementLibrary
- Automatically add the
dev.testify:testify
dependency to yourandroidTest
configuration. Defaults totrue
baselineSourceDir
- The directory on the client host machine that contain the original baseline images under version control. This value defaults to the
src/androidTest/assets
directory. installAndroidTestTask
- The Gradle task used to install the Android Test APK. In most cases, this value is automatically inferred from the project configuration and is typically
installDebugAndroidTest
. installTask
- The Gradle task used to install the Application Under Test APK. In most cases, this value is automatically inferred from the project configuration and is typically
installDebug
. isRecordMode
- Indicates that screenshotTest/screenshotRecord should never fail and always record new baseline images. Default is
false
. moduleName
- The name of the module under test. This value is inferred automatically and should not normally be modified by the user.
pullWaitTime
- The length of time to sleep in milliseconds after pulling files from the device. Used to allow time for the local file system to complete write operations. Defaults to 0.
rootDestinationDirectory
- The root directory containing the screenshots on the test device. Used when pulling files from the device. This value is automatically set.
screenshotAnnotation
- The annotation used by ScreenshotTestTask as an argument to
adb shell am instrument
to filter tests being run. See https://developer.android.com/studio/test/command-line#run-tests-with-adb. Defaults todev.testify.annotation.ScreenshotInstrumentation
targetPackageId
- The package ID for the APK under test. For a typical application, testify requires two APKs: the target apk under test, and a test apk containing your tests. e.g.
com.testify.example
testPackageId
- The package ID for the test APK. For a typical application, testify requires two APKs: the target apk under test, and a test apk containing your tests. e.g.
com.testify.example.test
testRunner
- The AndroidJUnitRunner to use when running your instrumented tests. In most cases, this is inferred from your project configuration automatically and is typically
androidx.test.runner.AndroidJUnitRunner
. See https://developer.android.com/training/testing/instrumented-tests#set-testing useSdCard
- Instructs Testify to write screenshots to the SDCARD directory. See https://ndtp.github.io/android-testify/docs/recipes/sdcard#configuring-the-gradle-plugin-to-write-to-the-sdcard. Defaults to false.
useTestStorage
- Instructs Testify to save screenshots to the Test Storage. See https://developer.android.com/reference/androidx/test/services/storage/TestStorage. Defaults to false.
Command-line properties
Gradle project properties can be used to set values on the Testify Gradle Plugin from the command-line.
They can be set from the command line using the -P
/ --project-prop
environment option.
See https://docs.gradle.org/current/userguide/project_properties.html
device
- Index of the Testify Device to target for the command. Use
./gradlew testifyDevices
to see a list of eligible devices. reportFileName
- Override the default file name used locally when copying the file.
reportPath
- Override the default path to copy the report file to.
shardCount, shardIndex
- If you need to parallelize the execution of your tests, sharing them across multiple devices to make them run faster, you can split them into groups, or shards. The test runner supports splitting a single test suite into multiple shards, so you can easily run tests belonging to the same shard together as a group. Each shard is identified by an index number. When running tests, use the
-PshardCount
option to specify the number of separate shards to create and the-PshardIndex
option to specify which shard to run.shardCount
- Specifies the total number of shards into which the test suite is divided
shardIndex
- Identifies the specific shard to be executed, with an index ranging from 0 to `shardCount` - 1.
testClass
- Run all tests in the specified class. Class name should be fully qualified.
./gradlew FlixSample:screenshotTest -PtestClass=dev.testify.samples.flix.ui.common.composables.CreditStripScreenshotTest testName
- Run the specific test case. Must be used in conjunction with
testClass
.
./gradlew FlixSample:screenshotTest -PtestName=longCreditStrip -PtestClass=dev.testify.samples.flix.ui.common.composables.CreditStripScreenshotTest user
- Specify the user ID for multi-user testing. See https://source.android.com/docs/devices/admin/multi-user-testing
verbose
- Print verbose console output. Useful for debugging purposes.
Core Tasks
screenshotTest
Run all the screenshot tests in your app and fail if any differences from the baseline are detected.
$ ./gradlew FlixSample:screenshotTest
> Task :FlixSample:deviceLocale
------------------------------------------------------------
Displays the device locale.
------------------------------------------------------------
Current Locale = en_US
> Task :FlixSample:deviceTimeZone
------------------------------------------------------------
Displays the time zone currently set on the device
------------------------------------------------------------
Time zone = Atlantic/Reykjavik
> Task :FlixSample:disableSoftKeyboard
------------------------------------------------------------
Disables the soft keyboard on the device
------------------------------------------------------------
Success
> Task :FlixSample:hidePasswords
------------------------------------------------------------
Hides passwords fully on the device
------------------------------------------------------------
Success
> Task :FlixSample:installDebug
Installing APK 'FlixSample-debug.apk' on 'Testify_Open_Source_Emulator(AVD) - 10' for :FlixSample:debug
Installed on 1 device.
> Task :FlixSample:installDebugAndroidTest
Installing APK 'FlixSample-debug-androidTest.apk' on 'Testify_Open_Source_Emulator(AVD) - 10' for :FlixSample:debug-androidTest
Installed on 1 device.
> Task :FlixSample:screenshotTest
------------------------------------------------------------
Run the Testify screenshot tests
------------------------------------------------------------
dev.testify.samples.flix.ComposableScreenshotTest:.
dev.testify.samples.flix.ui.common.composables.CastMemberScreenshotTest:.
dev.testify.samples.flix.ui.common.composables.CreditStripScreenshotTest:...
dev.testify.samples.flix.ui.common.composables.GenreStripScreenshotTest:..
dev.testify.samples.flix.ui.common.composables.MetaDataScreenshotTest:....
dev.testify.samples.flix.ui.common.composables.MoviePosterScreenshotTest:..
dev.testify.samples.flix.ui.common.composables.OverviewTextScreenshotTest:..
dev.testify.samples.flix.ui.common.composables.PrimaryTitleScreenshotTest:..
dev.testify.samples.flix.ui.common.composables.SecondaryTitleScreenshotTest:..
dev.testify.samples.flix.ui.moviedetails.MovieDetailsScreenshotTest:.
Time: 8.978
OK (20 tests)
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 13s
177 actionable tasks: 7 executed, 170 up-to-date
screenshotRecord
Run all the screenshot tests in your app and update the local baseline.
$ ./gradlew FlixSample:screenshotRecord
> Task :FlixSample:deviceLocale
------------------------------------------------------------
Displays the device locale.
------------------------------------------------------------
Current Locale = en_US
> Task :FlixSample:deviceTimeZone
------------------------------------------------------------
Displays the time zone currently set on the device
------------------------------------------------------------
Time zone = Atlantic/Reykjavik
> Task :FlixSample:disableSoftKeyboard
------------------------------------------------------------
Disables the soft keyboard on the device
------------------------------------------------------------
Success
> Task :FlixSample:hidePasswords
------------------------------------------------------------
Hides passwords fully on the device
------------------------------------------------------------
Success
> Task :FlixSample:installDebug
Installing APK 'FlixSample-debug.apk' on 'Testify_Open_Source_Emulator(AVD) - 10' for :FlixSample:debug
Installed on 1 device.
> Task :FlixSample:installDebugAndroidTest
Installing APK 'FlixSample-debug-androidTest.apk' on 'Testify_Open_Source_Emulator(AVD) - 10' for :FlixSample:debug-androidTest
Installed on 1 device.
> Task :FlixSample:screenshotClear
------------------------------------------------------------
Remove any existing screenshot test images from the device
------------------------------------------------------------
No failed screenshots found
> Task :FlixSample:screenshotTestRecord
------------------------------------------------------------
------------------------------------------------------------
dev.testify.samples.flix.ComposableScreenshotTest:
✓ Recording baseline for ComposableScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.CastMemberScreenshotTest:
✓ Recording baseline for CastMemberScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.CreditStripScreenshotTest:
✓ Recording baseline for CreditStripScreenshotTest_longCreditStrip.
✓ Recording baseline for CreditStripScreenshotTest_emptyCreditStrip.
✓ Recording baseline for CreditStripScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.GenreStripScreenshotTest:
✓ Recording baseline for GenreStripScreenshotTest_lotsOfGenres.
✓ Recording baseline for GenreStripScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.MetaDataScreenshotTest:
✓ Recording baseline for MetaDataScreenshotTest_onlyCertification.
✓ Recording baseline for MetaDataScreenshotTest_onlyReleaseDate.
✓ Recording baseline for MetaDataScreenshotTest_onlyRuntime.
✓ Recording baseline for MetaDataScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.MoviePosterScreenshotTest:
✓ Recording baseline for MoviePosterScreenshotTest_default.
✓ Recording baseline for MoviePosterScreenshotTest_nullPosterUrlImpliesLoading.
dev.testify.samples.flix.ui.common.composables.OverviewTextScreenshotTest:
✓ Recording baseline for OverviewTextScreenshotTest_longText.
✓ Recording baseline for OverviewTextScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.PrimaryTitleScreenshotTest:
✓ Recording baseline for PrimaryTitleScreenshotTest_longText.
✓ Recording baseline for PrimaryTitleScreenshotTest_default.
dev.testify.samples.flix.ui.common.composables.SecondaryTitleScreenshotTest:
✓ Recording baseline for SecondaryTitleScreenshotTest_longText.
✓ Recording baseline for SecondaryTitleScreenshotTest_default.
dev.testify.samples.flix.ui.moviedetails.MovieDetailsScreenshotTest:
✓ Recording baseline for MovieDetailsScreenshotTest_default.
Time: 7.65
OK (20 tests)
> Task :FlixSample:screenshotPull
------------------------------------------------------------
Pull screenshots from the device and wait for all files to be committed to disk
------------------------------------------------------------
Pulling screenshots:
Source = ./app_images/screenshots
Destination = /Users/admin/android-testify/Samples/Flix/src/androidTest/assets/
20 images to be pulled
Copying CastMemberScreenshotTest_default...
Copying ComposableScreenshotTest_default...
Copying CreditStripScreenshotTest_default...
Copying CreditStripScreenshotTest_emptyCreditStrip...
Copying CreditStripScreenshotTest_longCreditStrip...
Copying GenreStripScreenshotTest_default...
Copying GenreStripScreenshotTest_lotsOfGenres...
Copying MetaDataScreenshotTest_default...
Copying MetaDataScreenshotTest_onlyCertification...
Copying MetaDataScreenshotTest_onlyReleaseDate...
Copying MetaDataScreenshotTest_onlyRuntime...
Copying MovieDetailsScreenshotTest_default...
Copying MoviePosterScreenshotTest_default...
Copying MoviePosterScreenshotTest_nullPosterUrlImpliesLoading...
Copying OverviewTextScreenshotTest_default...
Copying OverviewTextScreenshotTest_longText...
Copying PrimaryTitleScreenshotTest_default...
Copying PrimaryTitleScreenshotTest_longText...
Copying SecondaryTitleScreenshotTest_default...
Copying SecondaryTitleScreenshotTest_longText...
Ready
> Task :FlixSample:screenshotRecord
------------------------------------------------------------
Run the screenshot tests and record a new baseline
------------------------------------------------------------
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 20s
180 actionable tasks: 13 executed, 167 up-to-date
screenshotPull
Copy images from the remote directory on your emulator to your local androidTest/assets
directory.
$ ./gradlew FlixSample:screenshotPull
> Task :FlixSample:screenshotPull
------------------------------------------------------------
Pull screenshots from the device and wait for all files to be committed to disk
------------------------------------------------------------
Pulling screenshots:
Source = ./app_images/screenshots
Destination = /Users/admin/android-testify/Samples/Flix/src/androidTest/assets/
20 images to be pulled
Copying CastMemberScreenshotTest_default...
Copying ComposableScreenshotTest_default...
Copying CreditStripScreenshotTest_default...
Copying CreditStripScreenshotTest_emptyCreditStrip...
Copying CreditStripScreenshotTest_longCreditStrip...
Copying GenreStripScreenshotTest_default...
Copying GenreStripScreenshotTest_lotsOfGenres...
Copying MetaDataScreenshotTest_default...
Copying MetaDataScreenshotTest_onlyCertification...
Copying MetaDataScreenshotTest_onlyReleaseDate...
Copying MetaDataScreenshotTest_onlyRuntime...
Copying MovieDetailsScreenshotTest_default...
Copying MoviePosterScreenshotTest_default...
Copying MoviePosterScreenshotTest_nullPosterUrlImpliesLoading...
Copying OverviewTextScreenshotTest_default...
Copying OverviewTextScreenshotTest_longText...
Copying PrimaryTitleScreenshotTest_default...
Copying PrimaryTitleScreenshotTest_longText...
Copying SecondaryTitleScreenshotTest_default...
Copying SecondaryTitleScreenshotTest_longText...
Ready
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 4s
1 actionable task: 1 executed
screenshotClear
Clear any baseline images that may be remaining on your emulator.
$ ./gradlew FlixSample:screenshotClear
> Task :FlixSample:screenshotClear
------------------------------------------------------------
Remove any existing screenshot test images from the device
------------------------------------------------------------
20 images to be deleted:
x CastMemberScreenshotTest_default
x ComposableScreenshotTest_default
x CreditStripScreenshotTest_default
x CreditStripScreenshotTest_emptyCreditStrip
x CreditStripScreenshotTest_longCreditStrip
x GenreStripScreenshotTest_default
x GenreStripScreenshotTest_lotsOfGenres
x MetaDataScreenshotTest_default
x MetaDataScreenshotTest_onlyCertification
x MetaDataScreenshotTest_onlyReleaseDate
x MetaDataScreenshotTest_onlyRuntime
x MovieDetailsScreenshotTest_default
x MoviePosterScreenshotTest_default
x MoviePosterScreenshotTest_nullPosterUrlImpliesLoading
x OverviewTextScreenshotTest_default
x OverviewTextScreenshotTest_longText
x PrimaryTitleScreenshotTest_default
x PrimaryTitleScreenshotTest_longText
x SecondaryTitleScreenshotTest_default
x SecondaryTitleScreenshotTest_longText
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed
Report Tasks
reportPull
Pull the report file from the device and wait for it to be committed to disk.
You can customize the destination path and file name for the report file by providing the reportFileName
or reportPath
paramaters to the gradle command.
reportFileName
: Override the default file name used locally when copying the file.reportPath
: Override the default path to copy the report file to.
Example:
./gradlew FlixSample:reportPull -PreportPath="/user/testify/" -PreportFileName="my-report.yml"
reportShow
Print the test result report to the console.
$ ./gradlew FlixSample:reportShow
> Task :FlixSample:reportShow
------------------------------------------------------------
Print the test result report to the console
------------------------------------------------------------
---
- session: 06583e71-665
- date: 2021-03-19@20:37:32
- failed: 0
- passed: 1
- total: 1
- tests:
- test:
name: withFocusOnBackground
class: ClientListActivityScreenshotTest
package: dev.testify.sample.clients.index
baseline_image: assets/screenshots/29-1080x2220@440dp-en_US/withFocusOnBackground.png
test_image: /data/user/0/dev.testify.sample/app_images/screenshots/29-1080x2220@440dp-en_US/ClientListActivityScreenshotTest_withFocusOnBackground.png
status: PASS
Utility Tasks
deviceLocale
Displays the locale currently set on the device
$ ./gradlew FlixSample:deviceLocale
> Task :FlixSample:deviceLocale
------------------------------------------------------------
Displays the locale currently set on the device
------------------------------------------------------------
Locale = en_US
deviceTimeZone
Displays the time zone currently set on the device.
$ ./gradlew FlixSample:deviceTimeZone
> Task :FlixSample:deviceTimeZone
------------------------------------------------------------
Displays the time zone currently set on the device
------------------------------------------------------------
Time zone = America/Toronto
disableSoftKeyboard
Disables the soft keyboard on the device. The soft keyboard can interfere with your screenshots, so it's good to turn it off.
$ ./gradlew FlixSample:disableSoftKeyboard
> Task :FlixSample:disableSoftKeyboard
------------------------------------------------------------
Disables the soft keyboard on the device
------------------------------------------------------------
Success
hidePasswords
Hides passwords fully on the device.
$ ./gradlew FlixSample:hidePasswords
> Task :FlixSample:hidePasswords
------------------------------------------------------------
Hides passwords fully on the device
------------------------------------------------------------
Success
testifyDevices
Displays the connected devices.
$ ./gradlew FlixSample:testifyDevices
> Task :FlixSample:testifyDevices
------------------------------------------------------------
Displays Testify devices
------------------------------------------------------------
Connected devices = 1
------------------------------------------------------------
-Pdevice=0 = emulator-5554
Add -Pdevice=N to any command to target a specific device
testifyKey
Displays the Testify output key for the current device. Testify uses the key as the baseline for the given device.
$ ./gradlew FlixSample:testifyKey
> Task :FlixSample:testifyKey
------------------------------------------------------------
Displays the Testify output key for the current device
------------------------------------------------------------
Format: {api_version}-{width_in_pixels}x{height_in_pixels}@{dpi}-{language}
key = 21-768x1280@320dp-en_US
testifySettings
Testify infers several project properties. You can view these properties with the testifySettings
command.
$ ./gradlew FlixSample:testifySettings
> Task :FlixSample:testifySettings
------------------------------------------------------------
Displays the Testify gradle extension settings
------------------------------------------------------------
baselineSourceDir = /Users/admin/android-testify/Samples/Flix/src/androidTest/assets
installAndroidTestTask = installDebugAndroidTest
installTask = installDebug
moduleName = FlixSample
outputFileNameFormat = null
pullWaitTime = 0
reportFilePath = ./app_testify
screenshotAnnotation = null
screenshotDirectory = ./app_images/screenshots
targetPackageId = dev.testify.samples.flix
testPackageId = dev.testify.samples.flix.test
testRunner = androidx.test.runner.AndroidJUnitRunner
useSdCard = false
useTestStorage = false
isRecordMode = false
user = 0
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 845ms
1 actionable task: 1 executed
testifyVersion
Displays the Testify plugin version
$ ./gradlew FlixSample:testifyVersion
> Task :FlixSample:testifyVersion
------------------------------------------------------------
Displays the Testify plugin version
------------------------------------------------------------
Vendor = ndtp
Title = Testify
Version = 1.0.0-beta3