Skip to content

feat(forge): Remove proptest runner from fuzzed tests #11061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 58 commits into from
Aug 15, 2025
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
4eddd93
Remove proptest from fuzzed tests
grandizzy Jul 22, 2025
9f034be
Persist and replay fuzz failure
grandizzy Jul 22, 2025
2c999b2
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 22, 2025
c70c6d7
Nits
grandizzy Jul 23, 2025
37b1114
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 23, 2025
36fa94c
add fuzz coverage metrics config, cleanup
grandizzy Jul 23, 2025
4e5334a
rm unec RefCell, refactor
grandizzy Jul 23, 2025
764ed64
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 24, 2025
f5d52a7
Add inline fuzz coverage metrics
grandizzy Jul 25, 2025
ed7fc23
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 25, 2025
5a226c8
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 28, 2025
045188c
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 29, 2025
1b5fcd8
Same corpus manager for fuzz and invariant tests
grandizzy Jul 30, 2025
1102723
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 30, 2025
35e8b37
Fix fmt
grandizzy Jul 30, 2025
43c53db
Unify configs
grandizzy Jul 30, 2025
a29f390
Use corpus manager / abi mutation for fuzz tests - Move hit count in …
grandizzy Jul 31, 2025
5df5393
Merge branch 'master' into no-fuzz-proptest
grandizzy Jul 31, 2025
6254863
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 1, 2025
ea0684f
Replay corpus for fuzz tests as well
grandizzy Aug 1, 2025
38a04d7
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 1, 2025
e9630c8
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 4, 2025
5c9a135
Add corpus replay failures for fuzz tests too
grandizzy Aug 4, 2025
3f0a9cb
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 5, 2025
06f713c
cleanup fuzz corpus on forge clean, display errors
grandizzy Aug 5, 2025
ea771e9
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 6, 2025
81a8b79
more abi mutations
grandizzy Aug 8, 2025
c6bcd98
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 8, 2025
7f23b2c
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 9, 2025
2006b1b
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 11, 2025
5c8554b
rand bit flip mutation + add support for more types
grandizzy Aug 11, 2025
e0ec65c
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 11, 2025
7d2c700
clippy
grandizzy Aug 11, 2025
9d032db
More mutations - interesting byte and word
grandizzy Aug 11, 2025
442dfe5
More mutations, traces
grandizzy Aug 11, 2025
e3d4e32
Review changes
grandizzy Aug 12, 2025
40036f4
add mutators traits
grandizzy Aug 12, 2025
bc3afd3
Simplify
grandizzy Aug 12, 2025
9c363b6
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 12, 2025
4f1134b
Update crates/evm/fuzz/src/strategies/mod.rs
grandizzy Aug 12, 2025
26fe2f6
Avoid clone when mutating array and tuples
grandizzy Aug 12, 2025
3ead4b3
Update crates/evm/evm/src/executors/corpus.rs
grandizzy Aug 12, 2025
b3b06ed
Update crates/evm/evm/src/executors/corpus.rs
grandizzy Aug 12, 2025
1a4ab5e
Update crates/evm/evm/src/executors/corpus.rs
grandizzy Aug 12, 2025
423f2d9
Fix typo
grandizzy Aug 12, 2025
387db1c
use wrapping add / sub for inc/dec mutation
grandizzy Aug 12, 2025
3757c7e
Validate inc/dec, better word mutation, tests
grandizzy Aug 13, 2025
864c9a6
Remove unused validation, early return on validation
grandizzy Aug 13, 2025
cdbd275
mutators to operate on &mut [u8], instrument, cleanup
grandizzy Aug 13, 2025
50e4ae6
Mutate fixedbytes, more address mutations
grandizzy Aug 13, 2025
2afb71b
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 13, 2025
cc831da
Unify mutators
grandizzy Aug 14, 2025
9c3baa4
Merge branch 'master' into no-fuzz-proptest
grandizzy Aug 14, 2025
028e2de
Add word mutation unit test
grandizzy Aug 14, 2025
179dd37
Address nits
grandizzy Aug 14, 2025
f777b95
Update crates/evm/fuzz/src/strategies/param.rs
grandizzy Aug 14, 2025
f33b356
Update crates/evm/fuzz/src/strategies/param.rs
grandizzy Aug 14, 2025
89189da
Rename mutate_random_array/tuple_value
grandizzy Aug 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 51 additions & 8 deletions crates/config/src/fuzz.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@ pub struct FuzzConfig {
pub dictionary: FuzzDictionaryConfig,
/// Number of runs to execute and include in the gas report.
pub gas_report_samples: u32,
/// The fuzz corpus configuration.
#[serde(flatten)]
pub corpus: FuzzCorpusConfig,
/// Path where fuzz failures are recorded and replayed.
pub failure_persist_dir: Option<PathBuf>,
/// Name of the file to record fuzz failures, defaults to `failures`.
pub failure_persist_file: Option<String>,
/// show `console.log` in fuzz test, defaults to `false`
pub show_logs: bool,
/// Optional timeout (in seconds) for each property test
Expand All @@ -43,8 +44,8 @@ impl Default for FuzzConfig {
seed: None,
dictionary: FuzzDictionaryConfig::default(),
gas_report_samples: 256,
corpus: FuzzCorpusConfig::default(),
failure_persist_dir: None,
failure_persist_file: None,
show_logs: false,
timeout: None,
}
Expand All @@ -54,11 +55,7 @@ impl Default for FuzzConfig {
impl FuzzConfig {
/// Creates fuzz configuration to write failures in `{PROJECT_ROOT}/cache/fuzz` dir.
pub fn new(cache_dir: PathBuf) -> Self {
Self {
failure_persist_dir: Some(cache_dir),
failure_persist_file: Some("failures".to_string()),
..Default::default()
}
Self { failure_persist_dir: Some(cache_dir), ..Default::default() }
}
}

Expand Down Expand Up @@ -97,3 +94,49 @@ impl Default for FuzzDictionaryConfig {
}
}
}

#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct FuzzCorpusConfig {
// Path to corpus directory, enabled coverage guided fuzzing mode.
// If not set then sequences producing new coverage are not persisted and mutated.
pub corpus_dir: Option<PathBuf>,
// Whether corpus to use gzip file compression and decompression.
pub corpus_gzip: bool,
// Number of mutations until entry marked as eligible to be flushed from in-memory corpus.
// Mutations will be performed at least `corpus_min_mutations` times.
pub corpus_min_mutations: usize,
// Number of corpus that won't be evicted from memory.
pub corpus_min_size: usize,
/// Whether to collect and display edge coverage metrics.
pub show_edge_coverage: bool,
}

impl FuzzCorpusConfig {
pub fn with_test_name(&mut self, test_name: &String) {
if let Some(corpus_dir) = &self.corpus_dir {
self.corpus_dir = Some(corpus_dir.join(test_name));
}
}

/// Whether edge coverage should be collected and displayed.
pub fn collect_edge_coverage(&self) -> bool {
self.corpus_dir.is_some() || self.show_edge_coverage
}

/// Whether coverage guided fuzzing is enabled.
pub fn is_coverage_guided(&self) -> bool {
self.corpus_dir.is_some()
}
}

impl Default for FuzzCorpusConfig {
fn default() -> Self {
Self {
corpus_dir: None,
corpus_gzip: true,
corpus_min_mutations: 5,
corpus_min_size: 0,
show_edge_coverage: false,
}
}
}
28 changes: 6 additions & 22 deletions crates/config/src/invariant.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Configuration for invariant testing

use crate::fuzz::FuzzDictionaryConfig;
use crate::fuzz::{FuzzCorpusConfig, FuzzDictionaryConfig};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;

Expand All @@ -26,15 +26,9 @@ pub struct InvariantConfig {
pub max_assume_rejects: u32,
/// Number of runs to execute and include in the gas report.
pub gas_report_samples: u32,
/// Path where invariant corpus is stored, enables coverage guided fuzzing and edge coverage
/// metrics.
pub corpus_dir: Option<PathBuf>,
/// Whether corpus to use gzip file compression and decompression.
pub corpus_gzip: bool,
// Number of corpus mutations until marked as eligible to be flushed from memory.
pub corpus_min_mutations: usize,
// Number of corpus that won't be evicted from memory.
pub corpus_min_size: usize,
/// The fuzz corpus configuration.
#[serde(flatten)]
pub corpus: FuzzCorpusConfig,
/// Path where invariant failures are recorded and replayed.
pub failure_persist_dir: Option<PathBuf>,
/// Whether to collect and display fuzzed selectors metrics.
Expand All @@ -43,8 +37,6 @@ pub struct InvariantConfig {
pub timeout: Option<u32>,
/// Display counterexample as solidity calls.
pub show_solidity: bool,
/// Whether to collect and display edge coverage metrics.
pub show_edge_coverage: bool,
}

impl Default for InvariantConfig {
Expand All @@ -58,15 +50,11 @@ impl Default for InvariantConfig {
shrink_run_limit: 5000,
max_assume_rejects: 65536,
gas_report_samples: 256,
corpus_dir: None,
corpus_gzip: true,
corpus_min_mutations: 5,
corpus_min_size: 0,
corpus: FuzzCorpusConfig::default(),
failure_persist_dir: None,
show_metrics: true,
timeout: None,
show_solidity: false,
show_edge_coverage: false,
}
}
}
Expand All @@ -83,15 +71,11 @@ impl InvariantConfig {
shrink_run_limit: 5000,
max_assume_rejects: 65536,
gas_report_samples: 256,
corpus_dir: None,
corpus_gzip: true,
corpus_min_mutations: 5,
corpus_min_size: 0,
corpus: FuzzCorpusConfig::default(),
failure_persist_dir: Some(cache_dir),
show_metrics: true,
timeout: None,
show_solidity: false,
show_edge_coverage: false,
}
}
}
6 changes: 3 additions & 3 deletions crates/config/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ pub use providers::Remappings;
use providers::*;

mod fuzz;
pub use fuzz::{FuzzConfig, FuzzDictionaryConfig};
pub use fuzz::{FuzzConfig, FuzzCorpusConfig, FuzzDictionaryConfig};

mod invariant;
pub use invariant::InvariantConfig;
Expand Down Expand Up @@ -1095,7 +1095,8 @@ impl Config {
}
};
remove_test_dir(&self.fuzz.failure_persist_dir);
remove_test_dir(&self.invariant.corpus_dir);
remove_test_dir(&self.fuzz.corpus.corpus_dir);
remove_test_dir(&self.invariant.corpus.corpus_dir);
remove_test_dir(&self.invariant.failure_persist_dir);

Ok(())
Expand Down Expand Up @@ -4614,7 +4615,6 @@ mod tests {
runs: 512,
depth: 10,
failure_persist_dir: Some(PathBuf::from("cache/invariant")),
corpus_dir: None,
..Default::default()
}
);
Expand Down
3 changes: 0 additions & 3 deletions crates/evm/core/src/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,6 @@ pub const MAGIC_ASSUME: &[u8] = b"FOUNDRY::ASSUME";
/// Magic return value returned by the `skip` cheatcode. Optionally appended with a reason.
pub const MAGIC_SKIP: &[u8] = b"FOUNDRY::SKIP";

/// Test timeout return value.
pub const TEST_TIMEOUT: &str = "FOUNDRY::TEST_TIMEOUT";

/// The address that deploys the default CREATE2 deployer contract.
pub const DEFAULT_CREATE2_DEPLOYER_DEPLOYER: Address =
address!("0x3fAB184622Dc19b6109349B94811493BF2a45362");
Expand Down
Loading
Loading