cryptonote_core: support intermediate PoW hashes in v17

See docs/BLOCK_HASHING.md

Author: jeffro256

docs/BLOCK_HASHING.md

@@ -0,0 +1,52 @@
+# Block hashing
+
+## Background
+
+In Bitcoin, block IDs and Proof-of-Work (PoW) hashes are calculated the same way, which is why on-chain block IDs always start
+with many 0s. However, in Monero, block IDs and PoW hashes are calcuated using different hash functions and slightly different
+inputs. This is because the hash functions used for PoW (CryptoNight and RandomX) are much slower to run than other
+cryptographic hash functions, specifically Keccak256, which Monero uses for block IDs. By contrast, Bitcoin uses SHA-256 for
+both block IDs and PoW. The reason that CryptoNight and RandomX were chosen for PoW in Monero is to protect against
+application-specific integrated circuits (ASICs) from dominanting the network hashrate like in Bitcoin. In theory, this
+would restore the principle of "1 CPU, 1 vote" outlined in the Satoshi whitepaper, and bring greater decentralization to the
+Monero exosystem. This document aims to describe exactly how block IDs and PoW hashes are calculated in the Monero reference
+codebase.
+
+## CryptoNight epoch
+
+To be reductive, the underlying design of Cryptonight is to 1) fill a 2MB buffer of memory using a memory-hard function, 2)
+perform over a millon random reads on that buffer to permutate some state, and then 3) do a final hash on the state using
+a random choice between four independent hash fuctions. There are 4 variants of CryptoNight: v0, v1, v2, and v4. This is the
+data flow for how block hashing in the CryptoNight epoch (Monero fork versions v1-v11) is performed:
+
+![CryptoNight data flow](resources/cryptonight-data-flow.png)
+
+## RandomX epoch
+
+Like, CryptoNight, RandomX also uses a memory hard function to fill a large space of memory called a "cache". However, this
+space is 256MB, not 2MB. It can be further expanded to 2GB to trade memory usage for compute speed. Unlike CryptoNight's cache,
+the RandomX cache is computed only once every 2048 blocks, not once per block. Monero uses a "seed" block ID of a previous block
+in the chain to fill the memory space. And to make ASICs even harder, RandomX uses random instruction execution as well. This
+is the data flow for how block hashing in the RandomX epoch (Monero fork versions v12-present) is performed:
+
+![RandomX data flow](resources/randomx-data-flow.png)
+
+## RandomX, with commitments, epoch
+
+Because RandomX can be slow to run, especially while building the cache (nearly 1/2 a second on my machine), it inadvertantly
+can introduce a Denial-of-Service (DoS) vector. For this reason, RandomX hash commitments can be added, so the PoW can be
+partially verified using only Blake2B, a much lighter hash function. For simplicity's sake, we can also remove the number of
+transactions in the block from the block hashing blob. This is the data flow for how block hashing works with these changes:
+
+![RandomX commitment data flow](resources/randomx-commitment-data-flow.png)
+
+The way that the hashes are calculated allows us to do partial PoW verification like so:
+
+![Partial RandomX verification data flow](resources/partial-pow-verify-data-flow.png)
+
+A node can be given simply the block header, block content hash, and intermediate PoW hash, and verify that some PoW was done
+using only `randomx_calculate_commitment()` (Blake2B undernath). Passing this partial verification requires PoW to done using
+Blake2B up to the relevant difficulty. While doing so is much easier than doing the full RandomX PoW, it still requires
+significant work. It is very important that we can calculate the block ID from the same information (or a subset thereof)
+that we calculate PoW from, since each block header contains the previous block ID. Binding the block headers as such makes
+faking intermediate PoW on chains of blocks exponentially harder the longer the chain is.

docs/resources/cryptonight-data-flow.png

@@ -105,6 +105,7 @@ void rx_seedheights(const uint64_t height, uint64_t *seed_height, uint64_t *next
 
 void rx_set_main_seedhash(const char *seedhash, size_t max_dataset_init_threads);
 void rx_slow_hash(const char *seedhash, const void *data, size_t length, char *result_hash);
+void rx_commitment(const void *data, size_t length, const void *rx_hash, char *result_commitment);
 
 void rx_set_miner_thread(uint32_t value, size_t max_dataset_init_threads);
 uint32_t rx_get_miner_thread(void);

docs/resources/partial-pow-verify-data-flow.png

@@ -487,6 +487,10 @@ void rx_slow_hash(const char *seedhash, const void *data, size_t length, char *r
   CTHR_RWLOCK_UNLOCK_WRITE(secondary_cache_lock);
 }
 
+void rx_commitment(const void *data, size_t length, const void *rx_hash, char *result_commitment) {
+  randomx_calculate_commitment(data, length, rx_hash, result_commitment);
+}
+
 void rx_set_miner_thread(uint32_t value, size_t max_dataset_init_threads) {
   miner_thread = value;
 

docs/resources/randomx-commitment-data-flow.png

@@ -1439,7 +1439,8 @@ namespace cryptonote
     blobdata blob = t_serializable_object_to_blob(static_cast<block_header>(b));
     crypto::hash tree_root_hash = get_tx_tree_hash(b);
     blob.append(reinterpret_cast<const char*>(&tree_root_hash), sizeof(tree_root_hash));
-    blob.append(tools::get_varint_data(b.tx_hashes.size()+1));
+    if (b.major_version < HF_VERSION_POW_COMMITMENT)
+      blob.append(tools::get_varint_data(b.tx_hashes.size()+1));
     return blob;
   }
   //---------------------------------------------------------------

docs/resources/randomx-data-flow.png

@@ -192,6 +192,7 @@
 #define HF_VERSION_BULLETPROOF_PLUS             15
 #define HF_VERSION_VIEW_TAGS                    15
 #define HF_VERSION_2021_SCALING                 15
+#define HF_VERSION_POW_COMMITMENT               17
 
 #define PER_KB_FEE_QUANTIZATION_DECIMALS        8
 #define CRYPTONOTE_SCALING_2021_FEE_ROUNDING_PLACES 2