-
Notifications
You must be signed in to change notification settings - Fork 242
Open
Description
With current settings rocksdb produces a lot of WAL log files that are not removed in timely manner. It leads to substantial memory consumption.
Behavior governed by two settings set_max_total_wal_size
and set_wal_ttl_seconds
- If both set to 0, logs will be deleted asap and will not get into the archive.
- If wal_ttl_seconds is 0 and wal_size_limit_mb is not 0, WAL files will be checked every 10 min and if total size is greater then wal_size_limit_mb, they will be deleted starting with the earliest until size_limit is met. All empty files will be deleted.
- If wal_ttl_seconds is not 0 and wall_size_limit_mb is 0, then WAL files will be checked every wal_ttl_seconds / 2 and those that are older than wal_ttl_seconds will be deleted.
- If both are not 0, WAL files will be checked every 10 min and both checks will be performed with ttl being first.
By default it's option 1
set_max_total_wal_size(0)
set_wal_ttl_seconds(0)
Suggestion.
Switch to option 4
set_max_total_wal_size(some_value)
set_wal_ttl_seconds(some_value)
which gives more predictable cleanup, it will happen according to set_wal_ttl_seconds
value, and potentially smaller memory consumption, garbage will not accumulate.
bobinson
Metadata
Metadata
Assignees
Labels
No labels