Add option to add non-existing snapshots instead of raising an error

[claytonkreisel]

Is your feature request related to a problem? Please describe.

The issue is that I have to run --snapshot-update in order to initiate a new assertion in a test. This is mostly an annoyance when testing with PyCharm using the IDE testing tools.

Describe the solution you'd like

I would prefer that there is an option that I can set by default that will not update any existing snapshots, but will add any newly created snapshots.

Describe alternatives you've considered

A flag called create-non-existing-snapshots as an option that can be set. This could then be configured in CLI or an IDE.

Additional context

None

[tducret]

Hi there!
I have the same request 😄

On top of that, I think it would be a great way to onboard former snapshottest users, as creating non-existing snapshots without passing the --snapshot-update option is the default behavior in snapshottest.

Is there any chance we could implement that?

[noahnu]

I'm open to this idea as opt-in, perhaps a flag --snapshot-update-new-only or something like that.

I will say this was an intentional design decision when initially creating syrupy. This is personal opinion however I believe snapshottest's approach of auto-creating the snapshots when missing is problematic. Imagine if you've written a number of new test cases and forget to create the relevant snapshots. If you push up your changes to your version control system and your CI system runs the test suite, with auto-create functionality, the tests will pass. This gives a false sense of confidence in the test suite. For all you know, the generated snapshot could be different every CI run.

Syrupy's stance is that a missing snapshot is no different from a non-matching snapshot. Syrupy has no way of knowing that you've just written a new test case and so we expect a snapshot not to exist, vs. a test case has been renamed disconnecting it from the snapshot file, or the dev forgot to commit the snapshot artifacts to git when pushing up their branch for CI to run.

[rhysawilliams2010]

why not use a CI option to throw on missing tests like jest does?

[noahnu]

I'm open to that idea. I'd still want it to be opt-in though.

[bn-andrew]

I would also like to add a vote for this, a team member just added syrupy to one of our repos but previously we have just done this manually.
In those implementations, we always set it up so if the comparison file is missing, it generates it and then fails the test with an Error message stating it has created the file.
The next run would then pass.

This means that if it was missing due to not being committed or some other reason, it should still fail in CI. But yes makes it a lot easier to update the expectations from within an ide 👍

On that note, it would also be great if it is opt-in, if it was possible through a config file or env var. I think it would be possible to set the flag in the IDE but it would be nicer to not have to mess with pycharm settings to turn that behaviour on.

Thanks