How to sign the stateroot？

[Tommo-L]

Q1: Should stateroot be part of core or plugin?

As we discussed at last night meeting, most people agreed that It should be part of the core to ensure the data consistency of the all cn nodes.

Q2: Which method should we use to sign state root?

This is still under discussion, there are some options:

• Option A: Using the consensus message, binding stateroot and proposal block to complete the signature.
• Option B: The cn node just broadcasts its own stateroot signature directly.
• Option C: Using the stateroot contract, the consensus node which send the proposal block, add a stateroot transaction in the proposal block, just like the MinerTransaction in neo2.x
• Option D: Add stateroot in header

What do you think?

[Tommo-L]

For me, I think option A is the best way at present, even through don't like it. Consider that if stateroot shuold be provided to the outside, like for SPV, cross-chain, it should be verifiable, requiring the signatures of all the consensus nodes.

[shargon]

They can send the current state root in the prepare request, it will be signed automatically. But then there is no difference with include it in the header.

[ZhangTao1596]

They can send the current state root in the prepare request, it will be signed automatically. But then there is no difference with include it in the header.

They can sign header and stateroot separately.

[roman-khimov]

it should be verifiable, requiring the signatures of all the consensus nodes.

That's exactly why it should be in the header, because that's what CNs naturally agree upon and sign. Also, speaking of outside users, any P2P-distributed solution would require those users implementing P2P protocol support which may not be something they want to do.

And to me that's also very important from the other perspective, ensuring user's trust in the network. This whole state is something that matters to users, making header contain state hash gives clear guarantee that whatever this state is it won't suddenly change. If there is a need to change the state because of some bug or whatever then is should be done in a clear and explicit manner in subsequent blocks (like described in #1287), but not by rewriting the history.

So, Option D, definitely.

[shargon]

My favorite version is send the StateRoot of the previous block in the proposal, and include it in the header.

[Tommo-L]

Store in block footer?

|  header: version, hash, ... |
|  body: txs |
|  footer: stateRoot |

All consensus nodes sign the whole block, and set block.hash = header.hash.

[ZhangTao1596]

We should consider of vm upgrade compatiblity first before decide where to put root.
For example, if we accept the versioning vm proposaled by @shargon . Same height will get the same result no matter which version the node is. We can put root anywhere without inconvenience.

[roman-khimov]

My favorite version is send the StateRoot of the previous block in the proposal, and include it in the header.

I've been thinking about it for a while and although it has the obvious downside of state lagging one block behind, it still gives us clear predictable behaviour: for transaction in block N you get a state at block N + 1. And it fits our current transaction handling model nicely, it's very easy to implement. So it probably really is the best option we have at the moment.

Store in block footer?

Wouldn't that complicate life for light nodes/wallets? I think it would be nice for them to be able to operate just using headers. Also, stateRoot should probably be added to MerkleRoot calculation then (the same way ConsensusData is now), so I'm not sure what this move to footer would buy us.

For example, if we accept the versioning vm proposaled by @shargon . Same height will get the same result no matter which version the node is.

This versioning is essential IMO, but not just for VM, we should also consider whole contract execution environment like native contracts or syscalls (that might also be changed in some incompatible way for whatever reasons).

[Tommo-L]

Description
Previously, we store stateRoot in p2p message for two main reasons:

• If we separate state persistence from block persistence, the consensus node can quickly process transactions without waiting for state writes and smart contract execution. original posted at Detach state persistence and block persistence #302 (comment)
  (If we store the previous block's state root in block header, it also may not have the problem)
• As shown in the figure below, if a bug in the VM caused an incorrect calculation of 1+1=3, we want to fix it.

For the above problem, there are 4 ways to deal with:

Solution1

Ignore. For the future, we will correct it.

Affect

• Layer1: neo-core need to support different neovm version and syscalls
• Layer2+: do nothing
• "Victim": have to face reality

Applicable scope:

• Tiny bug
• Tolerable impact
• "Victim" < layer2+

Solution2

Affect

• Layer1: neo-core need to support different neovm version and syscalls
• Layer2+: They all need to rollback, It may also cause double spend or loss of assets on layer2+, like dex, exchange, lease, AMM, etc.
• "Victim": It'll be happy

Applicable scope:

• Serious problem
• "Victim" > layer2+

Solution3

Roll back the block and re-execute the old block with the new version of vm.

Affect

• Layer1: neo-core need to support different neovm version and syscalls, but it may has an additional negative effect.
  • Transactions that failed in the past may be executed successfully.
  • Transactions that successed in the past may be executed failure.
  • Some related variables will also change.
• Layer2+: They all need to re-check the status of their transactions, their account balance, and dAPP, etc. It may lead to new problems, such as double spend or loss of assets on layer2+, like dex, exchange, lease, AMM, etc.
• "Victim": It'll be happy

Applicable scope:

• Serious problem
• Secondary impact is relatively small
• "Victim" > layer2+

Solution4

Affect

• Layer1: Just use the latest node for recalculation and start from the first block. But it also may has an additional negative effect.
  • Transactions that failed in the past may be executed successfully.
  • Transactions that successed in the past may be executed failure.
  • Some related variables will also change.
• Layer2+: They all need to re-check the status of their transactions, their account balance, and dAPP, etc. It may lead to new problems, such as double spend or loss of assets on layer2+, like dex, exchange, lease, AMM, etc.
• "Victim": It'll be happy

Applicable scope:

• Serious problem
• "Victim" > layer2+

Summary

• Solution1,2,3 are easy to accept in some cases.
• Solution1, Solution2, Solution3 need neo-core to support different neovm version and syscalls.
• Solution3, Solution4 may have additional effect, and it's a little uncertain and unpredictable to users.

I think that we put state root in block header will better than p2p message, which will benifit upgrade. What do you think?

[shargon]

I vote for Solution 3

[vncoelho]

We discussed this before,@igormcoelho emphasized that failed should not be converted. Perhaps we need a Solution 4,which is 3 with restrictions.
I will talk to Igor and remember the past discussions.

[roman-khimov]

Oh, it'd be so easy to discuss that in one room with a whiteboard, but anyway.

My take on solutions 3 and 4 is that they're dangerous as they would change the state for old blocks which I think is not acceptable for several reasons:

• it breaks state root, recalculated state is going to be different
• it's not cross-chain friendly because if the other blockchain is to refer to some state root and we're to change it that link is going to be lost
• probably the most important is that it breaks user's trust in the system, as a user I expect the state of the system to remain consistent no matter how the software changes, all changes to the state must be explicit and traceable

Solution 2 basically has the same issues, but it also directly contradicts one of the core Neo features which is one block finality.

And I don't think that we're only left with solution №1, we should take it and amend with one important addition from #1287 which is corrective transactions to get a solution number 5.

Solution 5
It would look something like this (sorry, it would take me like a week to redraw it following your style):

It basically follows solution number one in that we're releasing VM version 2 and all blocks starting with some number N get processed using this new VM. But at the same time if we know that the bug fixed in VM version 2 has caused some wrong states we explicitly correct them with another corrective transaction in one of subsequent blocks. So if accounts A1 and A2 have wrong balances we adjust them by this transaction.

Obviously it's a very powerful (and dangerous) mechanism, so this transaction would be signed by CNs or even whole governance committee (now that we have it). And it would only be applied when there is a need to, if some wrong cryptocuties were generated because of this bug we may as well decide that they're way too cute to change them (and thus they just won't be corrected). But if there is a change in the state, the user would know why it has happened and it at the same time won't break any references to old state.

Affects

• Layer1: neo-core needs to add support for VM/syscalls versioning and corrective transactions
• Layer2+: will follow the corrective transaction changes, it still may cause some problems here, but IMO they'd be easier to solve than with any other solution because the state change is explicit.
• "Victim": 👍

Applicable scope: just about anything.