-
Notifications
You must be signed in to change notification settings - Fork 232
Description
I'm not sure if this should be classified as a bug of a feature request:
Description
Golden images are stored in /build/paparazzi/failures
when there is a regression (next to the delta-*.png
files) but, they are not when:
-
The reference image (in
/test/snapshots/images
) is not present:java.lang.AssertionError: File module/ScreenshotTesting/src/test/snapshots/images/...ScreenshotTest_test.png does not exist at app.cash.paparazzi.SnapshotVerifier$newFrameHandler$1.handle(SnapshotVerifier.kt:60) at app.cash.paparazzi.Paparazzi$apply$1.evaluate$lambda$0(Paparazzi.kt:73) at app.cash.paparazzi.PaparazziSdk.takeSnapshots(PaparazziSdk.kt:333) at app.cash.paparazzi.PaparazziSdk.snapshot(PaparazziSdk.kt:178) at app.cash.paparazzi.PaparazziSdk.snapshot$default(PaparazziSdk.kt:176) at app.cash.paparazzi.PaparazziSdk.snapshot(PaparazziSdk.kt:173) at app.cash.paparazzi.Paparazzi.snapshot(Paparazzi.kt:103)
-
The reference image has a different size:
java.lang.IllegalStateException: Images are different sizes at app.cash.paparazzi.internal.OffByTwo.compare(OffByTwo.kt:11) at app.cash.paparazzi.internal.ImageUtils.compareImages(ImageUtils.kt:144) at app.cash.paparazzi.internal.ImageUtils.assertImageSimilar(ImageUtils.kt:55) at app.cash.paparazzi.SnapshotVerifier$newFrameHandler$1.handle(SnapshotVerifier.kt:73) at app.cash.paparazzi.Paparazzi$apply$1.evaluate$lambda$0(Paparazzi.kt:73) at app.cash.paparazzi.PaparazziSdk.takeSnapshots(PaparazziSdk.kt:333) at app.cash.paparazzi.PaparazziSdk.snapshot(PaparazziSdk.kt:178) at app.cash.paparazzi.PaparazziSdk.snapshot$default(PaparazziSdk.kt:176) at app.cash.paparazzi.PaparazziSdk.snapshot(PaparazziSdk.kt:173) at app.cash.paparazzi.Paparazzi.snapshot(Paparazzi.kt:103)
Steps to Reproduce
I can create a simple project for that if it is really necessary, but I think the description is fairly simple, and easy to reproduce, without special setup. Please let me know.
Expected behavior
Having golden screenshots generated even when the test fail because it does not match the reference (or if the reference is missing) would be very valuable in CI scenarios, where the newly created golden images could be immediately extracted without having to run an entire pipeline with a new execution of recordPaparazzi
Gradle task.
Also, running the recordPaparazzi
might lead to unrelated golden images being updated (with invisible variations) that would flood the review diff.
Additional information:
- Paparazzi Version: 1.3.5
Note:
In previous versions (pre-OffByTwo
), when there was a regression with different image sizes, the delta and golden images were stored to disk on /out/failures
.