feature(trie): create `rawDB` trieDB engine #3282

MaksymMalicki · 2025-11-20T14:52:54Z

It introduces rawDB engine for the trie2, which is used by the new state. In this PR:

The new package with rawDB is created. The structure of the package is quite simple and follows the pattern of pathdb and hashdb both in structure and interfaces.
Contract, Class and Contract Storages are written directly to the pebbleDB and read directly from it. The writing is done in the Update() method, which takes the nodesets generated by trie2.Commit(), flattens them and writes them to the DB. For reading, the readNode() method was introduced.
For the new State, Juno defaults to rawdb as a triedb engine

Next PR to review: #3283

codecov · 2025-11-20T15:05:06Z

Codecov Report

❌ Patch coverage is 64.10256% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.19%. Comparing base (056cfce) to head (26098c7).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
core/trie2/triedb/rawdb/database.go	68.33%	16 Missing and 3 partials ⚠️
core/trie2/triedb/database.go	28.57%	5 Missing ⚠️
core/trie2/triedb/hashdb/database.go	0.00%	2 Missing ⚠️
core/trie2/triedb/pathdb/database.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3282      +/-   ##
==========================================
+ Coverage   76.07%   76.19%   +0.12%     
==========================================
  Files         326      340      +14     
  Lines       32468    32611     +143     
==========================================
+ Hits        24699    24848     +149     
+ Misses       5976     5963      -13     
- Partials     1793     1800       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

rodrigo-pino

Looks good overall

core/trie2/triedb/rawdb/database.go

core/trie2/triedb/rawdb/reader.go

core/state/history_test.go

EgeCaner

Looks good! Some nitpicks

core/trie2/triedb/rawdb/database.go

EgeCaner · 2025-11-27T11:48:58Z

core/trie2/triedb/database.go

 	config *Config
 }

 func New(disk db.KeyValueStore, config *Config) (*Database, error) {


Can we return Database interface rather than Database struct and then directly interact with underlying db through interface, avoiding switches in other functions to choose which db we are working with?

Yeah, agreed

If we are not doing this at the moment, and decided to plan it for the later, please open an issue so we wont forget.

let me repost the idea here:

As I see Database struct is just a wrapper to handle creation and interaction with different trieDBs. I was wondering what if we return lets say Database interface instead of struct and remove struct all together. I guess all tries implement NodeReader, Update, Close, NewIterator and Commit. This leaves Journal and Scheme, other than that we are just routing call to underlying DB directly. What if we just make Underlying DBs to implement Scheme() which returns pathdb etc.. and Journal. Then New function creates and returns Database interface then from there consumer will directly interact with DB through interface. This allow us to remove Database struct and replace it with interface implemented by the underlying trie then we can avoid switches and remove all the code in that file except New.

I agree with this, we can remove this abstraction layer.

its done, I removed the Database struct and I'm returning the database.TrieDB interface with underlying triedb implementation

…/juno into maksym/rawdb-clean

core/trie2/triedb/database/db.go

rodrigo-pino · 2025-11-28T15:06:55Z

core/trie2/triedb/rawdb/database.go

+func (d *Database) Close() error {
+	return nil
+}


Please add a comment explaining that this is to satisfy the io.Closer interface

added comment

rodrigo-pino · 2025-11-28T15:12:14Z

core/trie2/triedb/rawdb/database.go

+type Config struct{}
+
+type Database struct {
+	disk db.KeyValueStore
+
+	lock sync.RWMutex
+	log  utils.SimpleLogger
+}


Since Database is implementing an interface, I believe it is a good programming practice to add a compilation check for it. It helps because:

We have a compilation check on this same file

Everyone reading this code will know that this DB is intentionally following a certain(s) interface(s).

var _ database.TrieDB = (*Database)(nil)

added compilation check, thanks

Based on Ege's previous comment, #3282 (comment), shouldn't refactor this logic?

yes I removed the Database structure and return the interface database.TrieDB interface with underlying implementation from the New method

rodrigo-pino

I just realized that the code has no tests. If it is something that was done before, it's ok, but we should change it starting from this PR

core/trie2/triedb/rawdb/database.go

core/trie2/trieutils/accessors.go

core/trie2/triedb/rawdb/database.go

core/trie2/triedb/database.go

core/trie2/triedb/rawdb/types.go

EgeCaner

Some minor comments, rest looks good to me!

core/trie2/triedb/rawdb/database_test.go

core/trie2/triedb/rawdb/database.go

infrmtcs · 2025-12-01T14:53:44Z

core/trie2/triedb/rawdb/database_test.go

@@ -0,0 +1,274 @@
+package rawdb


Can be in another PR, but can we use the same test for all 3 implementations?

I'll create a different PR for this, this one covers a lot so far. It's an interesting idea, I'll have to think if its possible, because the underlying triedb implementations don't have the same behavior

infrmtcs

I finished my first round, please ping me after you address the comments, thanks

Commits from maksym/rawdb (squashed)

373032d

MaksymMalicki temporarily deployed to Development November 20, 2025 15:08 — with GitHub Actions Inactive

This was referenced Nov 20, 2025

refactor(juno): Use common interfaces in Juno #3283

Open

[New State] Common interface integration #3272

Open

rodrigo-pino reviewed Nov 21, 2025

View reviewed changes

core/trie2/triedb/rawdb/database.go Show resolved Hide resolved

core/trie2/triedb/rawdb/reader.go Show resolved Hide resolved

MaksymMalicki mentioned this pull request Nov 22, 2025

refactor(trie2): use more descriptive felt.Address type instead of felt.Felt #3295

Open

infrmtcs reviewed Nov 24, 2025

View reviewed changes

core/state/history_test.go Outdated Show resolved Hide resolved

EgeCaner reviewed Nov 27, 2025

View reviewed changes

Commits from maksym/rawdb (squashed)

4a7808a

MaksymMalicki force-pushed the maksym/rawdb-clean branch from 373032d to 4a7808a Compare November 27, 2025 12:20

MaksymMalicki temporarily deployed to Development November 27, 2025 12:25 — with GitHub Actions Inactive

MaksymMalicki added 2 commits November 27, 2025 15:37

address comments

3e61597

Merge branch 'maksym/rawdb-clean' of https://github.com/NethermindEth…

9659941

…/juno into maksym/rawdb-clean

MaksymMalicki had a problem deploying to Development November 27, 2025 14:44 — with GitHub Actions Failure

EgeCaner reviewed Nov 27, 2025

View reviewed changes

core/trie2/triedb/database/db.go Show resolved Hide resolved

comments

d639dc3

MaksymMalicki temporarily deployed to Development November 28, 2025 11:19 — with GitHub Actions Inactive

MaksymMalicki mentioned this pull request Nov 28, 2025

refactor(trie2): use more descriptive felt.Hash type instead of felt.Felt #3316

Open

rodrigo-pino approved these changes Nov 28, 2025

View reviewed changes

rodrigo-pino requested changes Nov 28, 2025

View reviewed changes

core/trie2/triedb/rawdb/database.go Outdated Show resolved Hide resolved

core/trie2/trieutils/accessors.go Show resolved Hide resolved

core/trie2/trieutils/accessors.go Show resolved Hide resolved

core/trie2/triedb/rawdb/database.go Show resolved Hide resolved

EgeCaner reviewed Nov 28, 2025

View reviewed changes

core/trie2/triedb/database.go Show resolved Hide resolved

comments

2bf4e9d

MaksymMalicki temporarily deployed to Development December 1, 2025 11:36 — with GitHub Actions Inactive

infrmtcs reviewed Dec 1, 2025

View reviewed changes

core/trie2/triedb/rawdb/types.go Show resolved Hide resolved

EgeCaner approved these changes Dec 1, 2025

View reviewed changes

core/trie2/triedb/rawdb/database_test.go Outdated Show resolved Hide resolved

core/trie2/triedb/rawdb/database_test.go Outdated Show resolved Hide resolved

core/trie2/triedb/rawdb/database_test.go Outdated Show resolved Hide resolved

infrmtcs reviewed Dec 1, 2025

View reviewed changes

core/trie2/triedb/rawdb/database.go Outdated Show resolved Hide resolved

infrmtcs reviewed Dec 1, 2025

View reviewed changes

address comments

26098c7

MaksymMalicki deployed to Development December 1, 2025 16:08 — with GitHub Actions Active

feature(trie): create rawDB trieDB engine #3282

Are you sure you want to change the base?

feature(trie): create rawDB trieDB engine #3282

Conversation

MaksymMalicki commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rodrigo-pino left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EgeCaner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EgeCaner Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodrigo-pino Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodrigo-pino left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EgeCaner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

infrmtcs left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

feature(trie): create `rawDB` trieDB engine #3282

feature(trie): create `rawDB` trieDB engine #3282

MaksymMalicki commented Nov 20, 2025 •

edited

Loading

codecov bot commented Nov 20, 2025 •

edited

Loading

EgeCaner Nov 28, 2025 •

edited

Loading

rodrigo-pino Nov 28, 2025 •

edited

Loading