Skip to content

Conversation

hinto-janai
Copy link
Contributor

What

Adds initial Criterion benchmarks for:

  • cuprate-helper
  • cuprate-database
  • cuprate-cryptonight
  • cuprate-rpc-types
  • cuprate-blockchain

The location of these is benches/criterion/$CRATE e.g. benches/criterion/cuprate-helper.

This PR combines #322 #324 #327 #328 #332 to prevent merge conflicts.

In-memory disk ops

cuprate-{database,blockchain} are setup to use tempfile::tempdir() which uses std::env::temp_dir() which should be in-memory (/tmp) on most Linuxes, not sure about Windows/macOS. This should remove disk-related variables and provide more accurate function runtime data.

When broader benches/benchmarks are created, those could use disk to provide more realistic real-world data.

cuprate-helper

cargo bench -p cuprate-criterion-helper
benchmark average time per call
integer 1.2848 ns
unsigned 1.2876 ns
combine_low_high_bits_to_u128 846.72 ps
split_u128_into_low_high_bits 647.49 ps
timelock_to_u64 1.6886 ns
u64_to_timelock 1.7356 ns
cmp_float 1.0841 ns
cmp_float_nan 659.60 ps
get_mid 3.9615 ns
median 55.030 ns
tx_fee 6.9573 ns

cuprate-database

cargo bench -p cuprate-criterion-database
  • redb values are only shown for >3% diffs
  • code formatting is faster
benchmark heed redb diff
ro_get 24.193 ns 21.809 ns -9.98%
ro_len 4.2254 ns
ro_first 29.797 ns
ro_last 31.675 ns 33.186 ns +4.39%
ro_is_empty 7.6418 ns
ro_contains 22.448 ns 21.696 ns -3.96%
rw_get 23.204 ns 22.107 ns -4.63%
rw_len 4.6742 ns 4.4666 ns -4.63%
rw_first 31.337 ns
rw_last 33.221 ns 34.402 ns +3.90%
rw_is_empty 7.7049 ns
rw_contains 22.874 ns 22.017 ns -3.72%
get_range 1.6075 µs
iter 1.9882 µs 1.9254 µs -3.54%
keys 766.35 ns
values 1.5660 µs
put 3.2995 µs 3.5970 µs +8.76%
delete 422.90 ns
pop_first 313.42 ns 286.33 ns -8.70%
pop_last 329.65 ns 305.60 ns -7.11%
take 4.8301 µs
env_inner 3.7474 ns
tx_ro 77.387 ns 65.183 ns -15.73%
tx_rw 33.163 ns 35.133 ns +6.00%
open_db_ro 18.273 ns
open_db_rw 17.722 ns
create_db 18.222 ns
resize 15.658 µs
current_map_size 4.2498 ns
disk_size_bytes 2.2181 µs
pre_rct_output_id_as_bytes 649.28 ps
pre_rct_output_id_from_bytes 782.82 ps
output_as_bytes 650.27 ps
output_from_bytes 5.8138 ns

cuprate-cryptonight

Benchmark naming scheme: format!("{hash_fn_name}_{}", input.len())

E.g. cuprate_cryptonight::cryptonight_hash_v0(&[0; 8]) is v0_8.

benchmark average time per call
r_8 57.091 ms
r_64 58.804 ms
r_512 58.892 ms
r_4096 58.645 ms
r_65536 58.925 ms
v0_8 25.941 ms
v0_64 26.790 ms
v0_512 26.750 ms
v0_4096 26.300 ms
v0_65536 26.787 ms
v1_8 3.4884 ns
v1_64 26.920 ms
v1_512 26.406 ms
v1_4096 26.896 ms
v1_65536 27.092 ms
v2_64 28.900 ms
v2_512 29.331 ms
v2_4096 28.683 ms
v2_65536 29.345 ms

cuprate-rpc-types

benchmark average time per call
epee_from_bytes_get_blocks_request 86.675 ns
epee_to_bytes_get_blocks_request 108.94 ns
serde_from_str_tx_entry 928.48 ns
serde_to_string_tx_entry 190.51 ns
serde_from_str_get_last_block_header_response 651.84 ns
serde_to_string_get_last_block_header_response 507.25 ns
serde_from_str_calc_pow_request 183.96 ns
serde_to_string_calc_pow_request 60.611 ns
serde_from_str_sync_info_response 1.5147 µs
serde_to_string_sync_info_response 180.23 ns
serde_from_str_get_info_response 997.70 ns
serde_to_string_get_info_response 843.08 ns
serde_from_str_get_block_response 1.7308 µs
serde_to_string_get_block_response 576.69 ns
serde_from_str_get_connections_response 1.0853 µs
serde_to_string_get_connections_response 46.413 ns
serde_from_str_get_block_template_response 469.21 ns
serde_to_string_get_block_template_response 267.68 ns
serde_from_str_get_block_headers_range_response 1.1637 µs
serde_to_string_get_block_headers_range_response 74.125 ns

cuprate-blockchain

benchmark average time per call
add_block_v1_tx2 244.05 µs
add_block_v9_tx3 54.324 µs
add_block_v16_tx0 5.4705 µs
add_alt_block_v1_tx2 1.5027 µs
add_alt_block_v9_tx3 1.7813 µs
add_alt_block_v16_tx0 1.4159 µs

@github-actions github-actions bot added A-dependency Area: Related to dependencies, or changes to a Cargo.{toml,lock} file. A-workspace Area: Changes to a root workspace file or general repo file. A-docs Area: Related to documentation. A-storage Area: Related to storage. A-benches Area: Related to benchmarks. labels Nov 28, 2024
@hinto-janai hinto-janai mentioned this pull request Nov 28, 2024
28 tasks
@github-actions github-actions bot added the A-pruning Area: Related to pruning. label Nov 28, 2024
@github-actions github-actions bot removed the A-pruning Area: Related to pruning. label Nov 28, 2024
hinto-janai added a commit to Cuprate/benches that referenced this pull request Dec 3, 2024
@hinto-janai
Copy link
Contributor Author

See #354.

@hinto-janai hinto-janai closed this Dec 3, 2024
@hinto-janai hinto-janai deleted the criterion branch December 4, 2024 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-benches Area: Related to benchmarks. A-dependency Area: Related to dependencies, or changes to a Cargo.{toml,lock} file. A-docs Area: Related to documentation. A-storage Area: Related to storage. A-workspace Area: Changes to a root workspace file or general repo file.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant