feat: new mpt implementation #482
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shayanh
force-pushed
the
shayanh/mptnew
branch
from
7d37496 to
1a4bae4
Compare
| // | ||
| // More advanced improvement: either pre-execute block at guest to know exact allocations in | ||
| // advance, or allocate a separate arena specifically for updates. | ||
| let capacity = num_nodes + num_nodes / 10; |
There was a problem hiding this comment.
we should follow up about this
| nodes: Vec<NodeData<'a>>, | ||
|
|
||
| /// Cache. Hashing/encoding often needs "what would this node look like in its parent" | ||
| cached_references: Vec<RefCell<Option<NodeRef<'a>>>>, |
There was a problem hiding this comment.
btw even in Valery's PR, I wondered if we could get away using raw pointers for a little gain
There was a problem hiding this comment.
IIRC I tried using Cell instead and it was very slow for unknown reason. Raw pointers maybe can work
jonathanpwang
approved these changes
Aug 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new MPT implementation to optimize node reference calculation and data copying. The implementation is in a new crate named
mptnew, though it is mostly based on the existing MPT implementation from #427.The main contribution of this PR is a new serialization technique. MPT nodes are RLP-encoded and serialized recursively. In the guest program, deserialization is performed by traversing the serialized byte array. Since the nodes are already RLP-encoded, we avoid re-encoding them and re-storing their references. As a result, MPT deserialization and state root calculation become purely zero-copy, with node data and references maintained as pointers to the original input data. Additionally, this implementation reduces data copies by writing data directly into the bump area from the start whenever possible.
Important note: I have not yet implemented the
build_mptfeature that builds an MPT from MPT proofs. This feature is only used on the host to generate witness data. For now, I generate witness data for the new MPT implementation using the old MPT: on the host, I first build an old MPT and then serialize it in a format compatible with the new MPT. I will implement thebuild_mptfeature next and then remove the existing MPT implementation.Results on block number 23100006: