Skip to main content
Version: 3.2.0

Verify the tests

You can use Android Studio's built-in test runner to run your tests. Or, you can invoke the gradle task screenshotTest to run all the screenshot tests in your app. The test will fail if any differences from the baseline are detected.

$ ./gradlew app:screenshotTest

Test failures​

By default, Testify will use a strict binary comparison. This means that any difference in the binary value used for any of the pixels will be considered a failure. You may wish to adjust the matching tolerance through the use of the exactness tolerance. A value of less than 1.0f will result in a more leniant comparison which will exclude visually similar pixels. For more information on Testify's tolerance implementation, please read the blog post Accounting for platform differences.

Tolerance​

To adjust the tolerance, configure the exactness value on the rule or you can use the @BitmapComparisonExactness.

@ScreenshotInstrumentation
@Test
fun setExactness() {
    rule
        .configure { exactness = 0.95f }
        .assertSame()
}

Exclusions​

In addition to adjusting the tolerance, you can also exclude certain parts of the screen from comparion. You can define exclusion rects which Testify will ignore when comparing images.

@ScreenshotInstrumentation
@Test
fun exclusions() {
    rule
        .configure {
            defineExclusionRects { rootView, exclusionRects ->
                val card = rootView.findViewById<View>(R.id.info_card)
                exclusionRects.add(card.boundingBox)
            }
        }
        .assertSame()
}

Diagnosing Differences​

When a test fails, it can sometimes be difficult to determine the cause. You can enable the GenerateDiffs feature which will write a companion image for your screenshot test which can help you more easily identify which areas of your test have triggered the screenshot failure.

The generated file will be created in the same directory as your baseline images. Diff files can be pulled from the device using ./gradlew app:screenshotPull.

  •       Black pixels are identical between the baseline and test image
  •       Grey pixels have been excluded from the comparison
  •       Yellow pixels are different, but within the Exactness threshold
  •       Red pixels are different

This feature can be enabled by adding the testify-generate-diffs tag to the AndroidManifest.xml file in your androidTest target:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <application>
        <meta-data
            android:name="testify-generate-diffs"
            android:value="true" />
    </application>
</manifest>

Alternatively, you can enable/disable diffs programmatically:

@ScreenshotInstrumentation
@Test
fun generateDiffs() {
    TestifyFeatures.GenerateDiffs.setEnabled(true)
    rule.assertSame()
}

Additional Testing Scenarios​

For additional examples and advanced testing scenarios, please check out Testify Recipes.

Testify is built on top of Android Instrumented Tests and so you can also you any of Android's built-in instrumentation test running mechanisms to verify your tests.

You can: