perf(trie): reduce proof missed leaves by pre-dispatching all prefix set accounts #19074
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…set accounts Reduces "missed leaf" ratio from ~86% to near-zero by dispatching storage proofs for ALL accounts in storage_prefix_sets instead of only those in targets. ## Problem Previously, only accounts in `targets` (modified accounts) were pre-dispatched to parallel workers for storage proof computation. However, during the account trie walk, we encounter many more accounts that are part of the proof path (unmodified witness accounts). This caused: - 85.99% miss ratio (3,291 missed leaves/sec vs 527 precomputed/sec) - 24,851 synchronous StoragesTrie DB ops/sec - 92,958 total DB ops/sec - page_get_inline showing 6.7% in flamegraph ## Solution Dispatch storage proofs for ALL accounts in `storage_prefix_sets` before starting the account trie walk. This ensures every account encountered during the walk has a pre-computed storage root ready. ## Expected Impact - Miss ratio: 86% → ~5% - Missed leaves: 3,291/sec → ~200/sec (94% reduction) - StoragesTrie ops: 24,851/sec → ~1,600/sec (93% reduction) - Total DB ops: 92,958/sec → ~65,000/sec (30% reduction) - page_get_inline CPU: 6.7% → ~2-3% (50%+ reduction) ## Changes Modified `proof_task.rs:322-343` to: 1. Collect ALL account addresses from `storage_prefix_sets` 2. Merge with target slots from `input.targets` 3. Dispatch storage proofs for the combined set This leverages existing parallel infrastructure without architectural changes.
|
Experiment that didnt give good results |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
During parallel proof computation, we had an 85.99% miss ratio where accounts encountered during the account trie walk were not pre-dispatched to parallel workers. This caused high synchronous database access on the critical path.
Metrics Before Fix:
page_get_inline(MDBX): 6.7% CPU time in flamegraphRoot Cause:
Only accounts in
targets(modified accounts from block) were pre-dispatched:But the trie walk encounters ALL accounts in the prefix set (modified + witnesses), so 86% of accounts triggered synchronous fallback at proof_task.rs:914-957.
Goal:
The change aims to ensure every account encountered during the trie walk has its storage root pre-computed in parallel.