[qs] introduce BatchInfoExt enum to support extensible BatchInfo

[ibalajiarun]

Description

This PR introduces BatchInfoExt enum that will replace BatchInfo. This should allow easy addition of new fields to BatchInfo. Currently, there are two variants: V1 which is same as BatchInfo and V2 which includes extra fields to specify BatchKind::{Normal, Encrypted}. Normal is a normal batch and Encrypted is a batch with encrypted transactions.

When network message is received for ProofOfStore<BatchInfo>, it's converted into ProofOfStore<BatchInfoExt> and processed internally. Similarly, for SignedBatchInfo. In this PR, BatchInfo is converted into BatchInfoExt::V1. When a network message is sent out, it's converted into ProofOfStore<BatchInfo> and SignedBatchInfo<BatchInfo>.

Most internal message passing, except that deals with Batch messages, uses BatchInfoExt instead of BatchInfo.

[ibalajiarun]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [consensus] Introduce OptQS::V2 Payload #18087
• [qs] Generic Batch struct and new DB schema for Batch with Ext #18086
• [qs] send proof v2 support with flag #18085
• [qs] introduce BatchInfoExt enum to support extensible BatchInfo #18084 👈 (View in Graphite)
• [qs] Make ProofOfStore and SignedBatchInfo generic over BatchInfo #18062
• [types] Introduce new payload for encrypted mempool #17919
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

✅ Forge suite compat success on c6ae38ec4156111b3c14951f8665cdb3ec87b984 ==> c07419d7552b9079ea54c4c27d3c52aefb4ed56c

Compatibility test results for c6ae38ec4156111b3c14951f8665cdb3ec87b984 ==> c07419d7552b9079ea54c4c27d3c52aefb4ed56c (PR)
1. Check liveness of validators at old version: c6ae38ec4156111b3c14951f8665cdb3ec87b984
compatibility::simple-validator-upgrade::liveness-check : committed: 13005.60 txn/s, latency: 2629.19 ms, (p50: 2700 ms, p70: 3000, p90: 3200 ms, p99: 3500 ms), latency samples: 429560
2. Upgrading first Validator to new version: c07419d7552b9079ea54c4c27d3c52aefb4ed56c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 4306.92 txn/s, latency: 7863.59 ms, (p50: 8900 ms, p70: 9000, p90: 9000 ms, p99: 9100 ms), latency samples: 153720
3. Upgrading rest of first batch to new version: c07419d7552b9079ea54c4c27d3c52aefb4ed56c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4279.67 txn/s, latency: 7901.98 ms, (p50: 8900 ms, p70: 9100, p90: 9200 ms, p99: 9300 ms), latency samples: 151460
4. upgrading second batch to new version: c07419d7552b9079ea54c4c27d3c52aefb4ed56c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7568.20 txn/s, latency: 4489.15 ms, (p50: 4900 ms, p70: 5000, p90: 5100 ms, p99: 5200 ms), latency samples: 253860
5. check swarm health
Compatibility test for c6ae38ec4156111b3c14951f8665cdb3ec87b984 ==> c07419d7552b9079ea54c4c27d3c52aefb4ed56c passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on c07419d7552b9079ea54c4c27d3c52aefb4ed56c

two traffics test: inner traffic : committed: 13837.99 txn/s, submitted: 13838.10 txn/s, expired: 0.11 txn/s, latency: 2717.65 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 4500 ms), latency samples: 5148320
two traffics test : committed: 100.03 txn/s, latency: 859.46 ms, (p50: 800 ms, p70: 900, p90: 1000 ms, p99: 1700 ms), latency samples: 1680
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.293, avg: 2.075", "ConsensusProposalToOrdered: max: 0.174, avg: 0.171", "ConsensusOrderedToCommit: max: 0.163, avg: 0.113", "ConsensusProposalToCommit: max: 0.335, avg: 0.284"]
Max non-epoch-change gap was: 1 rounds at version 229300 (avg 0.00) [limit 4], 1.19s no progress at version 233278 (avg 0.07s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.52s no progress at version 1986325 (avg 0.52s) [limit 16].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking