Skip to content

Save database entries for skip_exceptions_to_reraise #4484

@Liam-DeVoe

Description

@Liam-DeVoe

Testing frameworks like pytest can skip tests dynamically using e.g. pytest.skip("skipped dynamically"). Hypothesis re-raises these exceptions to the testing framework, if they match skip_exceptions_to_reraise.

We want to continue doing this. But we should also start to save database entries for these failures, so that Hypothesis retries them immediately next time, instead of having to rediscover the particular inputs which trigger the dynamic exception.

We'll want to make sure this follows standard eviction logic: the entry is not evicted if it reproduces the skip exception, and is evicted if it doesn't reproduce. I'm noting this because we will re-raise the skip exception if it reproduces, which means we skip .freeze()-ing the ConjectureData and associated checks. I expect the direct implementation to work, but we should include tests to future-proof this new code path.

(prompted by Zac-HD/hypofuzz#202 (comment)).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementit's not broken, but we want it to be betterinternalsStuff that only Hypothesis devs should ever see

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions