doc: add account spec pages

doc/runtime/accounts.md

@@ -0,0 +1,257 @@
+# Accounts
+
+## Account Accesses
+
+The following types of account accesses exist:
+
+1. [External account reads](#external-account-reads)
+2. [Transaction validity checks](#transaction-validity-checks)
+3. [Transaction account accesses](#transaction-account-accesses)
+4. [Program executions](#program-executions)
+5. [Sysvar reads](#sysvar-reads)
+6. [System accesses](#system-accesses)
+
+### External account reads
+
+Read-only users may observe any account at any time by interfacing with
+the account database of a Solana node.  These *external account read*
+accesses only reveal [*stored accounts*](#stored-accounts).
+
+Typical methods for *external account reads* are the `getAccount()` or
+`getProgramAccounts()` RPC methods, or the `update_account()` Geyser
+callback.
+
+The consensus component of a validator is another source of external
+account reads.
+
+### Transaction validity checks
+
+Reads from [*stored accounts*](#stored-accounts) may be necessary to
+determine whether a transaction is valid or not.  Specific types of
+reads include
+- fee payer check (any stored account)
+- nonce check (special read interest in [*nonce accounts*](#nonce-accounts))
+- ALT lookup (special read interest in [*ALTs*](#alts))
+
+*Transaction validity checks* also deduct the *transaction fee* from the
+first signer account of the transaction (the *fee payer*).
+
+### Transaction account accesses
+
+Every Solana transaction specifies one or more accounts to access.
+A transaction can indirectly specify account accesses via [ALTs](#alts).
+
+These accounts then become available for read and write to programs
+invoked by a transaction.  During transaction execution, various access
+checks on writes are performed.
+
+Writes to certain types of accounts are categorically restricted
+(marks the transaction invalid, fails the transaction, or partially
+reverts the write after the transaction completes).  These include:
+- [Transparent Accounts](#transparent-accounts)
+- [System managed accounts](#system-managed-accounts)
+
+### Program executions
+
+A *program execution* is a type of account access that results from
+transaction execution.  The *program account* to execute is specified
+in the transaction.  This access type is distinct from a [*transaction
+account access*](#transaction-account-accesses).
+
+A *program account* is either a [*BPF program account*](#bpf-program-accounts)
+or a [*native program*](#builtin-accounts).
+
+### Sysvar reads
+
+Certain accounts can be read at any time during runtime execution.  This
+includes the slot transition, native programs, and syscall handlers.
+
+These accounts are either [*slot sysvars*](#slot-sysvars) or
+[*ephemeral sysvars*](#ephemeral-sysvars).
+
+### System accesses
+
+When executing a slot, the runtime alternates between transaction
+execution and miscellaneous system processing.  The latter includes any
+slot execution phase during which transactions are not executed.  These
+steps perform various writes to [*stored accounts*](#stored-accounts).
+
+System accesses are also responsible for maintaining a cryptographic
+hash over the stored set of accounts.  (See [SIMD-0215](https://github.com/solana-foundation/solana-improvement-documents/blob/main/proposals/0215-accounts-lattice-hash.md))
+
+## Account Taxonomy
+
+Firedancer defines the following taxonomy for Solana accounts.  This
+taxonomy is defined by which properties are shared between different
+types of accounts.
+
+1. [Transparent accounts](#transparent-accounts)
+    1. [Ephemeral sysvars](#ephemeral-sysvars)
+2. [Stored accounts](#stored-accounts)
+    1. [User writable accounts](#user-writable-accounts)
+        1. [Basic accounts](#basic-accounts)
+        2. [Nonce accounts](#nonce-accounts)
+        3. [ALTs](#alts)
+        4. [BPF program accounts](#bpf-program-accounts)
+        5. [Vote accounts](#vote-accounts)
+        6. [Fee collector accounts](#fee-collector-accounts)
+        7. [Stake accounts](#stake-accounts)
+        8. [Incinerator](#incinerator)
+    2. [System managed accounts](#system-managed-accounts)
+        1. [Slot sysvars](#slot-sysvars)
+        2. [Builtin accounts](#builtin-accounts)
+
+### Transparent Accounts
+
+Transparent accounts are generated entirely from runtime execution
+context such as the bank or the currently executed instruction.
+Transparent accounts are not stored in the account database.
+
+When read via a [*transaction account access*](#transaction-account-accesses)
+or a [*sysvar read*](#sysvar-reads), the account is generated on-the-fly.
+Transactions may not specify this account as writable in an access list.
+
+[*External account reads*](#external-account-reads) observe a
+non-existent account when trying to read *transparent accounts*.
+
+[*System writes*](#system-writes) to transparent accounts are inherently
+undefined, since the written data would not persist.
+
+### Ephemeral Sysvars
+
+Ephemeral sysvars are transparent accounts that are accessible through
+*sysvar reads*.
+
+For more information, see [sysvars.md](./sysvars.md#slot-sysvars)
+
+### Stored Accounts
+
+Accounts are *stored accounts* if they get read and written from/to the
+account database.
+
+All stored accounts are accessible via [*external account reads*](#external-account-reads)
+and [*transaction account accesses*](#transaction-account-accesses).
+
+### User writable accounts
+
+User writable accounts can be modified via any [*transaction account access*](#transaction-account-accesses).
+
+Most modifications are subject to permission checks.  Increasing the
+lamport value of an account always passes the permission checks of a
+*user writable account*.
+
+[*System writes*](#system-writes) occasionally modify these accounts for
+rent collection.
+
+### Basic accounts
+
+An account is a *basic account* if it does not fall under any other
+subcategory of [*user writable account*](#user-writable-accounts).
+
+### Nonce accounts
+
+*Nonce accounts* are user writable stored accounts that have special
+meaning during [*transaction validity checks*](#transaction-validity-checks).
+
+*Transaction validity checks*, *transaction account accesses*, and
+*system accesses* read and write nonce accounts.  Special contracts
+between these components exist to relax data dependencies.
+
+### ALTs
+
+*ALTs* act as a dictionary compression scheme when specifying account
+addresses in transactions.  Therefore, they have special meaning during
+[*transaction validity checks*](#transaction-validity-checks).
+
+In order to relax data dependencies between the *transaction validation*
+and *transaction execution* phases of block execution, ALTs must first
+be locked before they can be updated.
+
+### BPF program accounts
+
+There are two type of *BPF program accounts*: *User BPF programs* and
+*Core BPF programs*.  Not to be confused with *native programs*, which
+are a separate concept.
+
+*User BPF programs* hold bytecode programs deployed by users and their
+supporting metadata.
+
+Before the runtime can [*execute a program*](#program-executions), it
+must first derive executable bytecode by running the "ELF loader" and
+"bytecode verifier" steps.  Because these steps are rather expensive,
+the runtime facilitates transparent caching of executable bytecode.
+
+This is done using logic in the BPF Loader native programs:
+- The number of executable revisions of the same program is limited to
+  one per slot.
+- Program byte code is immutable while it is executable
+  (i.e. potentially in the cache)
+- Transitioning a program between immutable/executable, and
+  mutable/non-executable incurs a one slot delay.
+
+### Core BPF programs
+
+*Core BPF programs* are similar to [user BPF programs](#user-bpf-programs)
+but provide essential runtime facilities.  Typically, core BPF program
+updates are only possible via feature gate activations (which perform
+writes via [*system accesses*](#system-accesses)).
+
+### Vote accounts
+
+*Vote accounts* are essential to the Solana consensus layer.
+
+Validators frequently dispatch writes via transactions to *vote
+accounts* by invoking consensus-specific logic in the *vote program*,
+see [*transaction account accesses*](#transaction-account-accesses).
+
+The runtime periodically dispatches [*system accesses*](#system-accesses)
+for *vote accounts* to convert *vote credits* into *stake rewards*.
+
+Various other validator components simultaneously dispatch [*external account reads*](#external-account-reads).
+These include consensus (voting, fork choice), and network components
+(stake weighted QoS in repair and gossip).
+
+### Fee collector accounts
+
+A *fee collector account* gains *fee rewards* during runtime
+[*system accesses*](#system-accesses).
+
+Each slot, the *vote authority* of the leader *vote account* is
+designated as the *fee collector*.
+
+### Stake accounts
+
+*Stake accounts* implement stake delegation to [*vote accounts*](#vote-accounts).
+Stake accounts are infrequently accessed via [*transaction account
+accesses*](#transaction-account-accesses).  Stake accounts are also
+updated approximately once per epoch using [*system accesses*](#system-accesses)
+to accrue *inflation rewards*.
+
+### Incinerator
+
+The *incinerator* has a hardcoded account address `1nc1nerator11111111111111111111111111111111`.
+
+At the end of every slot, a [*system access*](#system-accesses) sets the
+account's lamport balance to zero, and updates the bank's capitalization
+accordingly.
+
+### Slot sysvars
+
+Readable for any access type.  Only [*system accesses*](#system-accesses)
+can write sysvar data.
+
+See [sysvars.md](./sysvars.md#slot-sysvars).
+
+### Builtin accounts
+
+Readable for any access type.  Only [*system accesses*](#system-accesses)
+write sysvar data.
+
+*Builtin accounts* are *stored accounts* that are usually just
+placeholders.  For example, the data content of `Stake11111111111111111111111111111111111111`
+(the stake program) is just `stake_program_v3` at the moment.
+
+Some *builtin accounts* have special behavior when accessed via a
+[*program execution*](#program-executions).  For example, the "system
+program" invokes a hardcoded runtime instruction processor.  These
+builtins are also called *native programs*.

doc/runtime/sysvars.md

@@ -0,0 +1,90 @@
+# Sysvars
+
+This page documents Firedancer's internal handling of Solana sysvars.
+
+Sysvars are special accounts that are implicitly read and written by the
+runtime.  Any program execution can read a sysvar, even when that sysvar
+account was not referenced in that transaction's account list.
+
+Any sysvars and sysvar categories specified in this document are valid
+as of June 2025.  This document might be outdated for sysvars added
+later.
+
+## Caching
+
+The Agave runtime maintains a "sysvar cache".  This cache holds data
+structures deserialized from the 'data' field of sysvar accounts.
+
+It is assumed that transactions / programs cannot directly change sysvar
+data.  Sysvar data is only changed when transitioning between program
+executions, transactions, or slots.
+
+The design of this cache is dictated by the cache invalidation policy.
+The invalidation policy, in turn, is dictated by the possible places
+where the sysvar can update.
+
+As of now (June 2025), sysvars can be categorized like so:
+
+- **Ephemeral Sysvars**: Updated within a transaction (outside program execution)
+- **Slot Sysvars**: Updated at the start or end of a slot (outside transaction execution)
+
+Please note that "sysvar categories" are a Firedancer-specific concept
+and are not defined in the SVM specification.
+
+## Slot sysvars
+
+The following sysvars are written outside of transaction execution, and
+can only be read during transaction execution.  These can be cached
+trivially:
+
+| Name                 | Account                                       |
+|----------------------|-----------------------------------------------|
+| `clock`              | `SysvarC1ock11111111111111111111111111111111` |
+| `epoch_schedule`     | `SysvarEpochSchedu1e111111111111111111111111` |
+| `fees`               | `SysvarFees111111111111111111111111111111111` |
+| `recent_blockhashes` | `SysvarRecentB1ockHashes11111111111111111111` |
+| `rent`               | `SysvarRent111111111111111111111111111111111` |
+| `slot_hashes`        | `SysvarS1otHashes111111111111111111111111111` |
+| `slot_history`       | `SysvarS1otHistory11111111111111111111111111` |
+| `stake_history`      | `SysvarStakeHistory1111111111111111111111111` |
+| `epoch_rewards`      | `SysvarEpochRewards1111111111111111111111111` |
+| `last_restart_slot`  | `SysvarLastRestartS1ot1111111111111111111111` |
+
+Firedancer uses a **write-through** cache policy for these sysvars.
+
+All writes to the above sysvars use the `fd_slot_sysvar_write` API, which:
+- Writes bank fields corresponding to this sysvar
+- Writes to account database
+
+## Transaction sysvars
+
+The following sysvars are generated from runtime context on read:
+
+| Name           | Account                                       |
+|----------------|-----------------------------------------------|
+| `instructions` | `Sysvar1nstructions1111111111111111111111111` |
+
+Firedancer uses a complex cache policy for these sysvars.
+- Primarily backed by short-lived buffers, unique for each transaction
+- Account writes via user transactions are banned
+  (FIXME clarify what happens when this is attempted)
+- Never written to the account database
+- Account reads via user transactions are redirected to a custom
+  callback (does not access the account database)
+
+## Unresolved issues
+
+**Lamport changes**
+
+Unclear if lamport updates have consequences for the sysvar cache (which
+only caches data, but not metadata).
+
+Particularly interesting for `Sysvar1nstructions1111111111111111111111111`.
+
+**Crafted snapshot attack**
+
+A Solana snapshot might be hacked to contain invalid sysvar data (that
+fails bincode decoding).  It is unclear how the Agave runtime handles
+accesses for those invalid sysvars.
+
+Unclear what happens if TxnSysvar accounts are restored from snapshots.