feat(trie): poc implementation of rsp trie

[lightsing]

rsp trie has proven to be efficiency, since they can be created and serialized outside zkvm, and deserialized inside zkvm.

current serial sparse trie use maps of nibbles to node, would be inefficient in such usage considering the large size of serialized data.

see also:

• rsp choose to move the trie building outside zkvm feat: build state trie from proofs in host instead succinctlabs/rsp#40
• another demo of using rsp implementation Demo of Integrating the MPT from RSP with Reth's ExecutionWitness #17416

[mattsse]

@kevaundray do you have thoughts on this?

[kevaundray]

@kevaundray do you have thoughts on this?

We currently are using the trie from Zeth (rsp trie was originally modified from that): https://github.com/boundless-xyz/zeth/blob/bc655b859bc689ef7944c7d959ef8024dd2d07e7/crates/core/src/lib.rs#L95

@Wollac Maybe you remember the benchmark numbers, was it roughly the same?

[kevaundray]

Plus one on adding the more efficient zkVM variant of the trie, though I'd lean towards the zeth version since it doesn't have an additional method that must be called by the host (given performance is roughly the same)

[Wollac]

The crate risc0-ethereum-trie mentioned by @kevaundray is exactly v2 of the referenced rsp-mpt library. It contains:

• Huge API improvements
• Speed improvements: Trie operations should be almost 50% faster. However, the speed increase for executing an entire block will obviously be much smaller.
• Logic fixes: For example, the referenced version allows the removal of an extension whose child is a digest.

These complex trie implementations are in reth only used for stateless but also have non-reth application. Therefore, I don't think they should be included in reth. Perhaps we can agree on one MPT implementation and collaborate on this.

[mattsse]

Perhaps we can agree on one MPT implementation and collaborate on this.

that'd be much appreciated

[lightsing]

In our project, we found that using rsp trie significantly reduces cycle consumption.
A closer comparison revealed that the key to this efficiency is building the trie externally and deserializing it inside the zkVM.

We initially considered testing risc0-ethereum-trie as well, but it doesn’t provide an interface to pre-build the storage tries required for our use case.
Since only the state trie can be built outside the zkVM, the potential performance gain would be limited, so we decided not to pursue it.

given performance is roughly the same

We disagree: building the trie inside the zkVM is indeed very cycle-intensive.

In our benchmark, using rsp trie delivered a 47.9 % improvement:

before:

$  make test-execute-chunk
blk 16525000->16525003, cycle 23320169, gas 729598, cycle-per-gas 32
test test_execute ... ok

after:

$  make test-execute-chunk
blk 16525000->16525003, cycle 12146445, gas 729598, cycle-per-gas 16
test test_execute ... ok

It is also critical to pre-build all required storage tries.
If only the state trie is built outside, we still need to pass node witnesses into the zkVM guest, leading to node parsing and other cycle-consuming operations.
By contrast, if every required trie were built externally, we would no longer need to pass node witnesses at all—deserializing these tries inside the zkVM would be sufficient.
The current risc0-ethereum-trie implementation does not support such comprehensive pre-construction of storage tries, which limits its efficiency for this purpose.

Questions for the @Wollac:

• Have you performed any experiments or collected data on serialization and pre-build optimizations for risc0-ethereum-trie?
• Are there any plans to add interfaces for pre-building storage tries in future releases, so that all required tries could be built externally and only deserialized within the zkVM?

[Wollac]

We initially considered testing risc0-ethereum-trie as well, but it doesn’t provide an interface to pre-build the storage tries required for our use case.
Since only the state trie can be built outside the zkVM, the potential performance gain would be limited, so we decided not to pursue it.

I don't understand this question. risc0-ethereum-trie offers the same functionality, if not more, as rsp-mpt for building a trie from RLP, EIP-1186, or ExecutionWitness. Of course, one can also manually construct a trie using the insert function. Furthermore, it is merely a generic, sparse MPT library, so it does not differentiate between state and storage, providing the same functionality for both. Handling the Ethereum world state would need to happen in outside code.

It is also critical to pre-build all required storage tries.

I'm not sure I agree. This might be true for V1 (rsp-mpt), but V2 (risc0-ethereum-trie) contains significant improvements in that area (e.g., it doesn't use the very slow rlp crate and caches RLP for faster hashing). It's also crucial to validate imported pre-built tries against MPT invariants, which rsp-mpt apparently doesn't do.
We didn't perform extensive testing in this area, and the results greatly depend on the exact use case and platform. However, in general, we got the intuition that loading the RLP data and decoding in the zkVM had similar performance to loading a pre-built trie with serde. In the context of full block execution, RLP was even a bit faster and much cleaner to use.