Thursday 18 May 2023

Android Snapshot test GithUb Action

I have the following yaml file:

name: Test and check build

on:
  push:
    branches: [ "main", "dev" ]
  pull_request:
    branches: [ "main", "dev", "hotfix" ]

jobs:
  build:

    runs-on: macos-latest
    if: github.event.pull_request.draft == false

    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: "0"
      - name: set up JDK 11
        uses: actions/setup-java@v3
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: gradle

      - name: Grant execute permission for gradlew
        run: chmod +x gradlew
      - name: Run Unit test
        run: ./gradlew testDebugUnitTest
      - name: Run connected tests
        uses: ReactiveCircus/android-emulator-runner@v2
        with:
          api-level: 29
          target: default
          arch: x86_64
          profile: Nexus 6
          script: ./gradlew connectedCheck
      - name: Build
        run: ./gradlew :app:assembleDebug

I'm using the code from the the google codelabs:

/**
 * Simple on-device screenshot comparator that uses golden images present in
 * `androidTest/assets`.
 *
 * Minimum SDK is O. Densities between devices must match.
 *
 * Screenshots are saved on device in `/data/data/{package}/files/${BuildConfig.SCREENSHOTS_FOLDER}.
 */
@RequiresApi(Build.VERSION_CODES.O)
fun assertScreenshotMatchesGolden(
    goldenName: String,
    node: SemanticsNodeInteraction
) {
    val bitmap = node.captureToImage().asAndroidBitmap()

    // Save screenshot to file for debugging
    saveScreenshot(goldenName, bitmap)
    try {
        InstrumentationRegistry.getInstrumentation()
            .context.resources.assets.open("$goldenName.png")
    } catch (ex: Exception) {
        null
    }?.use { BitmapFactory.decodeStream(it) }?.compare(bitmap)
}

private fun saveScreenshot(filename: String, bmp: Bitmap) {
    val canonicalPath =
        InstrumentationRegistry.getInstrumentation().targetContext.filesDir.canonicalPath
    val path = "$canonicalPath/${BuildConfig.SCREENSHOTS_FOLDER}"
    val folder = File(path)
    if (!folder.exists()) {
        folder.mkdirs()
    }
    FileOutputStream("$path/$filename.png").use { out ->
        bmp.compress(Bitmap.CompressFormat.PNG, 100, out)
    }
    println("Saved screenshot to $path/$filename.png")
}

private fun Bitmap.compare(other: Bitmap) {
    if (this.width != other.width || this.height != other.height) {
        throw AssertionError("Size of screenshot does not match golden file (check device density)")
    }
    // Compare row by row to save memory on device
    val row1 = IntArray(width)
    val row2 = IntArray(width)
    for (column in 0 until height) {
        // Read one row per bitmap and compare
        this.getRow(row1, column)
        other.getRow(row2, column)
        if (!row1.contentEquals(row2)) {
            throw AssertionError("Sizes match but bitmap content has differences")
        }
    }
}

private fun Bitmap.getRow(pixels: IntArray, column: Int) {
    this.getPixels(pixels, 0, width, 0, column, width, 1)
}

When I run the test locally everything works. I use an emulator for Nexus 6 with API 29. The way I did it was to run it once and then copy the generated screenshots in the assets folder. The second time I run the test they succeed.

When I run them on the CI I get the error

Sizes match but bitmap content has differences

It looks like there are differencies between the emulator I used localy and the one selected by:

uses: ReactiveCircus/android-emulator-runner@v2
with:
    api-level: 29
    target: default
    arch: x86_64
    profile: Nexus 6

How can I select the same Emulator on the CI and locally?



from Android Snapshot test GithUb Action

No comments:

Post a Comment