v1.30.1

This is a patch release.

This release fixes a regression from v1.29.0 when Dragonfly crashes during pipelining (#5012).
It is strongly advised to upgrade for those who use v1.30.0.

What's Changed

• fix: "chore: get rid of possible recursion when unwinding structured reply (#5012)"
• fix(hset_family): Fix val being overwritten by TTL (#5094)
• fix: update module versions (#5126)
• fix: make search keywords case insensitive (#5129)

Full Changelog: v1.30.0...v1.30.1

v1.30.0

Dragonfly v1.30.0

Important stability fixes and changes:

1. More JSON/SEARCH stability fixes: #4973 #5026 #5036 #5040
2. Performance improvements around pipleining #4994
3. Memcached suppport for GETS #5087 as well as supporting tls without authentication #5085

What's Changed

• feat(dfly_bench): allow regulated throughput in 3 modes by @romange in #4962
• feat(server): Add support for command aliasing by @abhijat in #4932
• fix: FT.SEARCH KNN crash fixed by @vyavdoshenko in #4973
• Add workflow to run mastodon ruby test suite by @abhijat in #4946
• chore(acl_family): Move some variables outside loop by @abhijat in #4975
• feat: availability_zone flag support was added by @vyavdoshenko in #4976
• chore(hset_family): Support resp3 format for hrandfield by @abhijat in #4978
• Fix test hypothesis by @cunla in #4927
• chore: time sensitive throttle for scan command by @kostasrim in #4954
• feat(tools/replay): Add pipeline latency distribution data by @Pothulapati in #4990
• refactor(debug-cmd): migrate ParsePopulateArgs to CmdArgParser by @XueSongTap in #4838
• fix: potential vulnerable cloned functions by @npt-1707 in #4996
• chore: split unit test steps in ci workflow by @kostasrim in #5002
• feat: allow sharding by cluster slot id by @romange in #5006
• fix: potential crash with multi-sharded pfmerge by @romange in #5008
• chore: Pipelining fixes by @romange in #4994
• fix: bitops_family crash fixed by @vyavdoshenko in #4989
• chore: fix some clang-tidy warnings by @romange in #5009
• chore: clean ups around command squasher by @romange in #5011
• chore: StoredCmd to support both owned and external arguments by @romange in #5010
• fix: RENAME in cluster mode crash by @vyavdoshenko in #4995
• fix: regtest failures by @romange in #5013
• fix(glob): Correctly match longer strings with '*' or '**' pattern by @mkaruza in #5001
• refactor: update contributing doc by @BorysTheDev in #5016
• fix: HNSW schema deleting document crash by @vyavdoshenko in #4987
• chore: add DCHECK to easier tests debugging by @BorysTheDev in #5018
• chore: polishing. put basic usage info to stdout using one call to a void messing lines by @vyavdoshenko in #5019
• feat(server): support partial sync from last master by @adiholden in #5015
• feat(pytest): Generate huge val string in seeder by @mkaruza in #4957
• fix: JSON.ARRPOP command crash fixed by @vyavdoshenko in #5026
• chore: more performance improvements around MGET and pipelining by @romange in #5022
• feat: add copy cmd by @BorysTheDev in #5032
• fix: json depth limit has been changed to 64 by @vyavdoshenko in #5036
• chore: show moving P99 latency in dfly_bench by @romange in #5027
• chore: get rid of possible recursion when unwinding structured reply by @romange in #5012
• fix: Support for additional flags in expireat, pexpire and pexpireat commands by @H4R5H1T-007 in #5007
• chore: bump up max_busy_read_usec in tests by @romange in #5039
• fix: make FreeMemWithEvictionStep atomic by @kostasrim in #4885
• fix(stream_family): Fix stream memory tracking issues by @BagritsevichStepan in #5024
• chore: skip failing test_migration_rebalance_node by @kostasrim in #5042
• chore: add dfly_bench to release workflow by @kostasrim in #5045
• refactor: add WARNING regarding max_client_iobuf_len flag usage by @BorysTheDev in #5029
• refactor: reduce number of cluster logs by @BorysTheDev in #5044
• chore(json_family): Clean up code. FIRST PR by @BagritsevichStepan in #5049
• fix: send response error for long sequence instead of command crash by @vyavdoshenko in #5048
• feat: introduce 'debug iostats' command by @romange in #5051
• fix: json.set recursive processing crash by @vyavdoshenko in #5040
• bug(server): fix lns mismatch in replication by @adiholden in #4967
• feat(dfly_bench): introduce connect only flow by @romange in #5052
• chore: skip test_bug_in_json_memory_tracking by @kostasrim in #5066
• feat(memory_cmd): Add WITHOUTKEY option for the MEMORY USAGE command. SECOND PR by @BagritsevichStepan in #5068
• chore: dash table clean ups by @romange in #5064
• chore: further extend the compression analysis by @romange in #5065
• fix(json_family): Fix memory tracking for the JSON.SET command. THIRD PR by @BagritsevichStepan in #5069
• chore: introduce huffman encoder class by @romange in #5076
• chore(metrics): add rdb_bgsave_in_progress and rdb_last_bgsave_status by @kostasrim in #5061
• chore: futher dash table clean ups by @romange in #5072
• chore: add password flag to dfly_bench by @kostasrim in #5074
• chore: reorganize compact object mask bits by @romange in #5077
• feat: add migrated keys statistic by @BorysTheDev in #5043
• chore: Add HuffmanDecoder class by @romange in #5078
• fix: remove code duplicate in GetSlice by @romange in #5079
• chore: allow skipping cluster probing by @romange in #5082
• feat: add huffman coding to compact object by @romange in #5080
• chore: allow setting huffman tables via DEBUG COMPRESSION SET by @romange in #5083
• fix: Support TLS for Memcached without password authentication by @vyavdoshenko in #5085
• test: tune test_cluster_migration_while_seeding by @BorysTheDev in #5088
• feat(memcache): Add support for GETS by @abhijat in #5087
• feat(github-workflow): upload dev binaries to artifact buckets by @Abhra303 in #5031

Huge thanks to all the contributors! ❤️

New Contributors

• @XueSongTap made their first contribution in #4838
• @npt-1707 made their first contribution in #4996

Full Changelog: v1.29.0...v1.30.0

v1.29.0

Dragonfly v1.29.0

Important stability fixes and changes:

1. Improvements and bug fixes around FT.Search: #4810 #4868 #4942 #4844 #4930 #4941#4837
2. Lua stack buffer overflow crash #4853
3. Reduce the negative effect of replication/slot migration on throughput and latency #4821 #4910

What's Changed

• fix: support dump for external objects by @romange in #4794
• feat: add sleep between migration connecting attempts by @BorysTheDev in #4798
• feat test: add test for pipeline queue overlimit by @adiholden in #4791
• fix(json_family): Fix memory tracking for JSON by @BagritsevichStepan in #4777
• fix(hset_family): Fix crash on scan after expiry set by @abhijat in #4802
• feat: ignore managed_service_info flag for cluster by @BorysTheDev in #4803
• fix(hll_family): Fixed PFMERGE wrong merge operation by @mkaruza in #4796
• chore(bptree): introduce ability comparing keys to non-key types by @romange in #4805
• chore(cluster_mgr): introduce SlotRange class by @romange in #4814
• fix: python tests were excluded for epoll case. by @vyavdoshenko in #4813
• feat(server): Add support for aliasing commands by @abhijat in #4782
• chore: remove pointer tagging for ScoreSdsPolicy by @romange in #4815
• chore: refactor debug populate by @romange in #4823
• fix: python test was excluded for epoll case. by @vyavdoshenko in #4824
• feat(dfly_bench): allow generation of keys without hashslots for set/get by @romange in #4818
• feat(dfly_bench): Handle moved slots in cluster by @mkaruza in #4761
• feat(cluster_mgr): add populate command by @romange in #4816
• feat(docker): push dragonfly image to gcp artifact registry by @Abhra303 in #4716
• fix(dfly_bench): Get MOVED error from parsed response by @mkaruza in #4827
• fix(pytest): Save into S3 temp directory by @mkaruza in #4806
• feat(server): flush slots traverse better yield by @adiholden in #4821
• fix: Backward compatibility for fields without the "@" sign in the FT.AGGREGATE command by @vyavdoshenko in #4810
• feat(command docs): Enable fallback tab completion in cli by @abhijat in #4842
• chore(zset_family/score_map): Replace sds arguments with string_view by @abhijat in #4738
• fix: Added query to log on error by @vyavdoshenko in #4843
• fix(replication_test): Temporary skip test_bug_in_json_memory_tracking test by @BagritsevichStepan in #4845
• refactor: rename cntx variable to exec_st by @BorysTheDev in #4847
• feat(metrics): Update metrics for aliased commands by @abhijat in #4819
• chore: turn on more logs for test_replication_timeout_on_full_sync by @BorysTheDev in #4848
• fix(rdb_load): Handle JSON loading failure when parsing fails by @BagritsevichStepan in #4801
• fix(snapshot): Append trailing slash for GCP by @mkaruza in #4846
• fix(lua) : Prevent Lua stack buffer overflow crash (#4853) by @KIMDONGYEON00 in #4854
• fix: increase timeout for test_replication_timeout_on_full_sync by @BorysTheDev in #4851
• fix(test): test_cluster_slot_ownership_changes by @adiholden in #4852
• fix(server_family): Remove redundant logging in the CLIENT command by @BagritsevichStepan in #4859
• fix test: test_cluster_slot_ownership_changes by @adiholden in #4865
• chore: Remove version attribute from compose file by @ajnart in #4869
• fix: FT.SEARCH commas support for ranges by @vyavdoshenko in #4844
• chore: turn on more logs for test_migration_timeout_on_sync by @BorysTheDev in #4870
• chore(search_family): Add logging for monostate queries by @BagritsevichStepan in #4872
• feat(server): SCAN command add ATTR options by @lichuang in #4766
• feat: Support synonyms in queries. Add FT.SYNUPDATE, FT.SYNDUMP by @vyavdoshenko in #4837
• feat(server): add connections send timeout by @adiholden in #4874
• fix: add version id for dev container builds by @romange in #4878
• fix: local dashboard show rapid changes in QPS by @romange in #4886
• fix: '@text:prefix*' matching for fields. by @vyavdoshenko in #4868
• fix: test_migration_rebalance_node by @BorysTheDev in #4891
• feat: debug compression by @romange in #4879
• fix: docker release should be updated latest version using semantic version of latest tag by @vyavdoshenko in #4887
• fix: lsn check failure by @kostasrim in #4881
• fix: skip heartbeat if shard is under global lock by @kostasrim in #4882
• fix: update timeout for connection test by @vyavdoshenko in #4895
• fix: make pause all for migration finalization by @BorysTheDev in #4897
• chore: test snapshot in replica while seeding master by @kostasrim in #4867
• feat(server): Move bumpup logic out of FindInternal by @mkaruza in #4877
• pytest: temporarily skip test while its failures are investigated by @abhijat in #4900
• test: tune test_cluster_migration_while_seeding to reduce false/posit… by @BorysTheDev in #4901
• fix: take_over_counters by @kostasrim in #4890
• fix(hset_family): Ensure empty hash sets are removed by @abhijat in #4873
• fix: test_replica_snapshot_with_big_values_while_seeding by @kostasrim in #4902
• fix(rdb): Print tag as int in error by @abhijat in #4908
• feat(server): Exit process if error is report during load snapshot by @mkaruza in #4907
• fix: invalid master_last_io_seconds_ago metric during stable sync by @vyavdoshenko in #4892
• fix(pytest): Remove invalid argument from pytest by @mkaruza in #4912
• fix(hset): Fix size account discrepancy when setting TTL by @abhijat in #4913
• perf: Added benchmarks for different operations in String Set. by @H4R5H1T-007 in #4866
• refactor: add ability to reinit incoming migration object by @BorysTheDev in #4756
• fix: cancel blocking command during migration finalization by @BorysTheDev in #4904
• fix(pytest): Wait exception on instance start and stop by @mkaruza in #4916
• chore: Cleanup DbSlice::CallChangeCallbacks by @mkaruza in #4917
• fix: Skip test_take_over_counters for epoll by @abhijat in #4919
• test: add logs for cluster_mgr_test.py by @BorysTheDev in #4920
• chore: improve SlotRanges::Merge algorithm by @BorysTheDev in #4921
• fix(stream_family): Fix memory tracking for the STREAMs. SECOND PR by @BagritsevichStepan in #4781
• fix(set_family): Update object time during SET FIELDEXPIRE by @mkaruza in #4903
• fix: Dropping stale connections updated by @vyavdoshenko in #4909
• fix: prefix search with added synonyms fixed by @vyavdoshenko in #4930
• fix: test_migration_timeout_on_sync by @BorysTheDev in #4933
• fix(connection): Close MONITOR connection if pipeline...

v1.28.2

This is a patch release.

What's Changed

• server(bug): fix json mget crash on invalid path (#4940)
• fix: ZRANGE with LIMIT parameters crash fixed (#4944)
• fix(set_family): Transfer TTL flag from DenseLink object in delete (#4947)
• chore: add metrics for heartbeat evictions (#4952)

Full Changelog: v1.28.1...v1.28.2

v1.28.1

This is a patch release.

Update urgency low, unless you use TLS sockets.

What's Changed

• fix: TLS short write bug (romange/helio@bfb655c)
• fix(hset_family): Fix crash on scan after expiry set (#4802)
• feat(server): flush slots traverse better yield (#4821)
• feat(cluster): add sleep between migration connecting attempts (#4798)

v1.27.4

This is a patch release.

Update urgency low, unless you use TLS sockets or streams.

What's Changed

• fix: TLS short write (romange/helio@bfb655c)
• fix(stream_family): Fix replication for the XADD and XTRIM commands (#4591)
• Fix auto journaling in transaction (#4737)
• fix(hset_family): Fix crash on scan after expiry set (#4802)
• fix(cluster): flush slots traverse better yield (#4821)
• fix(cluster): control cluster migration speed with flag (#4734)

Full Changelog: v1.27.3...v1.27.4

v1.28.0

Dragonfly v1.28.0 - Spring Edition 🌵

This marks our second release for the year, with lots of bugs closed and probably some new bugs introduced.

Important fixes and changes:

1. Deadlocked transaction bugs #4590 #4647 #4685
2. Sharded pubsub (API-7) : #4702 #4518 #4529
3. Fix Streams replication #4591 #4737
4. Numerous cluster improvements #4727 #4786 #4767

What's Changed

• fix(search_family): Add options test for the FT.AGGREGATE command by @BagritsevichStepan in #4479
• fix(stream_family): Fix replication for the XADD and XTRIM commands by @BagritsevichStepan in #4591
• chore(search_family): Simplify FT.SEARCH reply code by @BagritsevichStepan in #4553
• feat(zset family): Implement ZDIFFSTORE by @mkaruza in #4582
• fix(seeder): Support stream types in the Seeder by @BagritsevichStepan in #4574
• fix: debian path in dragonfly.service by @romange in #4594
• fix: tls replication without ca verification by @kostasrim in #4539
• fix(server): list move on single shard wake blocking transaction by @adiholden in #4590
• fix: race when bumping items while loading a snapshot by @kostasrim in #4564
• fix: multi/exec CROSSSLOT error processing by @BorysTheDev in #4598
• test: added test_keys_expiration_during_migration by @BorysTheDev in #4558
• chore: fix test_snapshot by @kostasrim in #4607
• chore: fix glob translation with \ at the end by @romange in #4608
• feat: support lz4 compression method by @romange in #4610
• fix: test_keys_expiration_during_migration by @BorysTheDev in #4613
• chore: add zstd_test by @romange in #4614
• chore: simplify transaction code by @romange in #4602
• fix: epoll reg tests running on iouring by @kostasrim in #4612
• chore: use macos-14 for the daily macos build by @kostasrim in #4609
• chore: introduce docker release pipeline by @romange in #4618
• chore: connection pipeline cache does not shrink by @kostasrim in #4491
• chore: implement sharded pub/sub by @kostasrim in #4518
• chore: introduce background deletions of DenseSet objects by @romange in #4496
• fix: revert reflex based matching due to performance problems by @romange in #4626
• chore: allow yielding in Scan command by @kostasrim in #4624
• refactor: simplify journal and restore streamer cancellation by @BorysTheDev in #4549
• server: rdb loader big string loading in chunks by @adiholden in #4623
• chore: remove redundant calls to GetSlice by @romange in #4635
• server: redis parser reject big bulk read by @adiholden in #4627
• chore: Improve loading of sorted sets by @romange in #4631
• fix: invalid BUSYKEY response by @romange in #4637
• chore: propagate reserve data upon restore by @romange in #4641
• chore: get rid of sds in SortedMap::AddElem by @romange in #4638
• fix: improve test_send_delay_metric test by @romange in #4642
• fix: do not flush replicaof if failed to connect by @kostasrim in #4636
• fix: performance degradation with Reflex matching by @romange in #4634
• chore: get rid of lispack stats by @romange in #4644
• chore: add listpack statistics to objhist by @romange in #4650
• chore: allow sampling of topk hottest keys. by @romange in #4649
• feat: implement unique keys counting using hll by @romange in #4652
• fix: pull latest helio by @romange in #4651
• chore: unsubscribe pub/sub connections after cluster migration by @kostasrim in #4529
• fix: bring back the helm step to docker release action by @romange in #4657
• fix: do not crash upon inconsistent expire iterator by @romange in #4658
• chore: replace try assert false with pytest.raises by @kostasrim in #4662
• server: disable single shard tx optimization on scheduling by @adiholden in #4647
• fix: race condition in ReplicaOfInternal by @kostasrim in #4653
• Support Cluster configuration for dfly_bench by @romange in #4664
• CI: add slow regression tests by @adiholden in #4646
• fix: replication error message and scan command yield by @kostasrim in #4665
• feat: add migration error statistic by @BorysTheDev in #4643
• chore: refactor ServerFamily::Info by @romange in #4671
• fix(search_family): Add option tests for the FT.CREATE command by @BagritsevichStepan in #4678
• chore: introduce DEBUG COMPRESSION by @romange in #4620
• chore: cluster related clean ups by @romange in #4683
• fix: SendInvalidationTrackingMessage should not block. by @romange in #4680
• fix: reduce stack usage on Fedora by @romange in #4690
• fix(server): deadlock with replicaof inside multi by @adiholden in #4685
• fix(metrics): add corresponding comments to command metrics by @Yomanz in #4674
• feat: Added a flag to ignore key expiry. by @H4R5H1T-007 in #4667
• feat(cluster_mgr): Add argument to set path to dragonfly binary by @mkaruza in #4695
• fix: possible preemption under FiberAtomicGuard by @romange in #4692
• chore: add debug printings to SetExisting by @romange in #4694
• chore: Add debug logs to help tracking transactional deadlocks by @romange in #4669
• chore: snapshot/tiering improvements by @romange in #4661
• chore: disable failing epoll tests by @kostasrim in #4699
• feat: client unpause and client help by @kostasrim in #4630
• fix: ZMSCORE return value if key does not exist by @vyavdoshenko in #4697
• fix: prevent tx_executor reading from sock if operation is canceled by @BorysTheDev in #4704
• feat(set_family): Update TTL for existing fields in SADDEX by @abhijat in #4700
• chore: update CONTRIBUTING.md by @kostasrim in #4703
• chore: minor clean ups before introducing ProvidedBuffers by @romange in #4709
• refactor: clean cluster code by @BorysTheDev in #4707
• fix: Remove the dfly logo print from pytest run #4682 by @vyavdoshenko in #4706
• fix(json_family): Fix JSON.SET handling for nested fields by @BagritsevichStepan in #4710
• fix(dfly_bench): support dns resolution for cluster hosts by @romange in #4715
• fix: buffer overrun in GetRandomHex by @romange in #4717
• feat(set_family): Add support for KEEPTTL to SAddEx by @abhijat in #4712
• fix: preemption in atomic section of heartbeat by @kostasrim in #4720
• Revert "chore: minor clean ups before introducing ProvidedBuffers (#4… by @adiholden in #4719
• feat(server): Add PUBSUB SHARDCHANNELS/SHARDNUMSUB by @mkaruza in #4702
• fix(dfly_bench): track hit rate for mget command by @romange in https://github.com/dragonflydb/dr...

v1.27.3

This is a patch release.

What's Changed

• fix: invalid reference inside GetQueueBackpressure (#4789)
• fix(cluster) : moved error port number on migration finish (#4776)

Full Changelog: v1.27.2...v1.27.3

v1.27.2

This is a patch release.

What's Changed

• fix: add corresponding comments to command metrics (#4674)
• fix: deadlock with replicaof inside multi (#4685)
• feat: allow cluster READONLY command (#4727)
• fix: disable single shard tx optimization on scheduling (#4647)
• fix: preemption in atomic section of heartbeat (#4720)

Full Changelog: v1.27.1...v1.27.2

v1.27.1

This is a patch release.

What's Changed

• fix: wrong binary path in deb package (#4593)

Update urgency: low
Full Changelog: v1.27.0...v1.27.1