Skip to content

Conversation

RomanHodulak
Copy link
Collaborator

Closes #18463

Adds a pruner for change sets.

Change sets are newly persisted in the database and need to be pruned along with the rest of the entities to preserve space on full nodes.

@github-project-automation github-project-automation bot moved this to Backlog in Reth Tracker Oct 7, 2025
@github-actions github-actions bot added A-db Related to the database A-engine Related to the engine implementation A-trie Related to Merkle Patricia Trie implementation C-enhancement New feature or request C-perf A change motivated by improving speed, memory usage or disk footprint labels Oct 7, 2025
@jenpaff jenpaff requested a review from mediocregopher October 7, 2025 16:44
@jenpaff jenpaff moved this from Backlog to In Progress in Reth Tracker Oct 7, 2025
@jenpaff jenpaff linked an issue Oct 7, 2025 that may be closed by this pull request
Copy link
Collaborator

@mediocregopher mediocregopher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One missing functionality that should be easy to fix, plus naming suggestion.

Question: when pruning runs, does it also update the pipeline sync checkpoint which is stored? Or how should the engine check what range of data is actually available to be used?

@RomanHodulak RomanHodulak force-pushed the 18460-trie-changesets-prune branch 2 times, most recently from 34e2b9f to 9a6f0f0 Compare October 8, 2025 13:31
@RomanHodulak
Copy link
Collaborator Author

Answer: The pipeline will run the pruner, then check its output and save it's checkpoint.

let segment_output = segment.prune(
provider,
PruneInput { previous_checkpoint, to_block, limiter: limiter.clone() },
)?;
if let Some(checkpoint) = segment_output.checkpoint {
segment
.save_checkpoint(provider, checkpoint.as_prune_checkpoint(prune_mode))?;
}

@mediocregopher
Copy link
Collaborator

Answer: The pipeline will run the pruner, then check its output and save it's checkpoint.

let segment_output = segment.prune(
provider,
PruneInput { previous_checkpoint, to_block, limiter: limiter.clone() },
)?;
if let Some(checkpoint) = segment_output.checkpoint {
segment
.save_checkpoint(provider, checkpoint.as_prune_checkpoint(prune_mode))?;
}

Hmm, I think this is the pruning checkpoint though, which is different than the pipeline sync checkpoint. So probably I'll need to update how the bounds check is working for engine API...

@RomanHodulak RomanHodulak force-pushed the 18460-trie-changesets-prune branch from 9a6f0f0 to d9a253b Compare October 8, 2025 16:37
@RomanHodulak RomanHodulak changed the title feat(prune): Add ChangeSets segment feat(prune): Add MerkleChangeSets segment Oct 9, 2025
Copy link
Collaborator

@mediocregopher mediocregopher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@RomanHodulak RomanHodulak marked this pull request as ready for review October 9, 2025 11:24
@RomanHodulak RomanHodulak force-pushed the 18460-trie-changesets-prune branch from d9a253b to 71936c3 Compare October 9, 2025 14:25
@RomanHodulak RomanHodulak merged commit 467c13c into 18460-trie-changesets Oct 9, 2025
38 checks passed
@RomanHodulak RomanHodulak deleted the 18460-trie-changesets-prune branch October 9, 2025 16:12
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-db Related to the database A-engine Related to the engine implementation A-trie Related to Merkle Patricia Trie implementation C-enhancement New feature or request C-perf A change motivated by improving speed, memory usage or disk footprint

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Trie ChangeSets: Pruning

2 participants