Skip to content

feat(forge): Remove proptest runner from fuzzed tests #11061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 58 commits into
base: master
Choose a base branch
from

Conversation

grandizzy
Copy link
Collaborator

@grandizzy grandizzy commented Jul 22, 2025

Motivation

Solution

  • consolidate corpus config in FuzzCorpusConfig and use for both fuzz and invariants
  • decouple CorpusManager from invariant config and reuse in fuzz tests (fuzz corpus is same as invariant but with a single tx; BasicTxDetails used in for both strategies and same corpus management). move logic for abi mutate in abi_mutate and use in both fuzz / invariants
  • abi mutate strategies for uint, int, address and fixed bytes inspired from ityfuzz: BitFlipMutator / ByteInterestingMutator / WordInterestingMutator / DwordInterestingMutator / IncDecValue (applicable for uint, int only). Mutation of custom structs, tuples and dynamic arrays (increase / decrease size, mutate random element) https://github.com/fuzzland/ityfuzz/blob/master/src/mutation_utils.rs#L449-L452
  • remove fuzz proptest, introduce time based campaigns
  • remove test data refcells for invariants and fuzz, no longer needed without proptest runner (should also improve perf)
  • unit tests

Maze comparison

Maze PR coverage guided & abi mutations v1.2.3
0 15 (invariants 16, 17, 18, 19, 27, 28, 29, 30, 34, 35, 38, 41, 42, 5, 9) 13 (invariants 16, 17, 18, 19, 27, 29, 30, 34, 35, 38, 42, 5, 9)
1 15 (invariants 12, 16, 17, 25, 26, 31, 32, 33, 37, 38, 39, 46, 47, 48, 5) 14 (invariants 12, 16, 17, 25, 26, 31, 33, 38, 39, 46, 47, 48, 5, 7)
2 TBD 16 (invariants 13, 14, 2, 26, 27, 29, 31, 33, 38, 39, 4, 40, 42, 43, 44, 46)
3 13 (invariants 1, 10, 12, 16, 25, 3, 33, 36, 40, 41, 6, 8, 9) 12 (invariants 10, 12, 16, 25, 3, 33, 36, 40, 41, 6, 8, 9)
4 18 (invariants 10, 11, 17, 22, 24, 27, 28, 32, 33, 36, 37, 38, 39, 41, 43, 45, 5, 6) 15 (invariants 10, 11, 17, 22, 27, 28, 32, 33, 36, 37, 41, 43, 5, 6)

PR Checklist

  • Added Tests
  • Added Documentation
  • Breaking changes

@grandizzy grandizzy changed the title WIP: Remove proptest from fuzzed tests WIP: Remove proptest runner from fuzzed tests Jul 22, 2025
@grandizzy grandizzy self-assigned this Jul 22, 2025
@grandizzy grandizzy moved this to In Progress in Foundry Jul 22, 2025
@grandizzy grandizzy added this to the v1.4.0 milestone Jul 29, 2025
grandizzy and others added 2 commits August 12, 2025 15:22
Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
grandizzy and others added 6 commits August 12, 2025 16:24
Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
@grandizzy grandizzy requested a review from DaniPopes August 13, 2025 07:25
DaniPopes
DaniPopes previously approved these changes Aug 14, 2025
Copy link
Member

@DaniPopes DaniPopes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, last nits

Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
grandizzy and others added 2 commits August 14, 2025 16:05
Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

feat(fuzz): add coverage guided fuzzing for stateless tests (currently only invariant mode works)
2 participants