Dragonfly v1.21.0

Some prominent changes include:

• Alpha release of SSD Data tiering - enabled with flag --prefix some/path/basename
• Very basic support of multi-tenancy @3260
• HSETEX now supports NX option, see our docs.
• Added support for JSON.MERGE and for JSON.MSET.
• valkey's replica-announce-ip and --cluster--announc--ip are consolidated via --announce-ip flag.

What's Changed

• feat(server): master stop sending exec opcode to replica by @adiholden in https://github.com/dragonflydb/dragonfly/pull/3289
• chore: optimize zpopminmax operations by @romange in https://github.com/dragonflydb/dragonfly/pull/3291
• fix(json_family): Fix error in JsonFamilyTest.MGet by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/3285
• fix: define macro WITH_AWS in cmake when flag is ON by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3296
• chore: Add 'memory arena show' command by @romange in https://github.com/dragonflydb/dragonfly/pull/3298
• fix: missing logs on pytest failures #3255 by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3272
• chore: refactor compact_object and introduce materialize method by @romange in https://github.com/dragonflydb/dragonfly/pull/3300
• fix(acl): loading interleaved plain and hashed passwords by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3297
• chore: Add CompactObj Raw methods by @romange in https://github.com/dragonflydb/dragonfly/pull/3303
• chore: On invalid TTL, print the TTL first by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3306
• chore: moving functions + renaming in tiered storage by @romange in https://github.com/dragonflydb/dragonfly/pull/3304
• fix(tests): Add missing awaits by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3273
• chore: refactoring around tiered storage by @romange in https://github.com/dragonflydb/dragonfly/pull/3299
• feat: yield when serialization is in progress by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3220
• refactor: Use DbContext, OpArgs and Transaction to access DbSlice by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3311
• chore: Separate tiered serialization format from object values by @romange in https://github.com/dragonflydb/dragonfly/pull/3310
• chore: add more community links to README by @Niennienzz in https://github.com/dragonflydb/dragonfly/pull/3308
• fix(cluster): Join on specified attempt id by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3305
• chore(acl): add test with requirepass and aclfile by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3312
• chore: skip test_cluster_flushall_during_migration by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3316
• fix (pytest): generate unique random dbfilename for tests by @adiholden in https://github.com/dragonflydb/dragonfly/pull/3317
• chore: bypass decoding/encoding of the data when performing offloading by @romange in https://github.com/dragonflydb/dragonfly/pull/3315
• refactor: acl helpers and global tables by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3309
• chore(tiering): Fixes by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3225
• fix(migration): Use transactions! by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3266
• fix: forbid DFLYCLUSTER commads set for emulated cluster mode by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3307
• feat(namespaces): Initial support for multi-tenant by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3260
• chore: improve dfly_bench stats by @romange in https://github.com/dragonflydb/dragonfly/pull/3320
• chore: remove replace_deleted flag from hnswlib by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3323
• feat(test): Improve benchmark workflow by @adiholden in https://github.com/dragonflydb/dragonfly/pull/3330
• fix: Proper shutdown sequence with Namespaces by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3333
• fix(test): copy logs for failed test during TEARDOWN phase by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3331
• fix(json_family): fix JSON.STRAPPEND command for JSON legacy mode by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/3264
• chore(tiering): add protection against overruning memory budget by @romange in https://github.com/dragonflydb/dragonfly/pull/3327
• chore: Add coordinated omission mode by @romange in https://github.com/dragonflydb/dragonfly/pull/3332
• chore: implement sequential pass without the overlapping traffic by @romange in https://github.com/dragonflydb/dragonfly/pull/3335
• feat: add an option to flush serialized entries on threshold limit by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3241
• feat(hset_family): Add NX option to HSETEX by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/3295
• Fix blocking commands moved error by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3334
• fix: ub in RegisterOnChange and regression tests for big values by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3336
• fix: Cancel outgoing migration when retrying / closing by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3339
• chore: Make KeyIndex iterable by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3326
• fix: AllocationTracker::Remove return value was reversed by @romange in https://github.com/dragonflydb/dragonfly/pull/3341
• chore: remove redundant metrics from memory stats by @romange in https://github.com/dragonflydb/dragonfly/pull/3345
• fix: corruption in replication stream by @romange in https://github.com/dragonflydb/dragonfly/pull/3344
• chore: clean up TaskQueue since we do not need multiple fibers for it by @romange in https://github.com/dragonflydb/dragonfly/pull/3348
• chore: pull helio by @romange in https://github.com/dragonflydb/dragonfly/pull/3350
• chore: Log connection context when issuing dangerous cmds by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3352
• chore: small rename and add dcheck on LocalBlockingCounter by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3356
• refactor: reduce number of logs for cluster by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3357
• chore: fixes to dfly_bench by @romange in https://github.com/dragonflydb/dragonfly/pull/3353
• chore: fix test_parser_memory_stats flakiness by @romange in https://github.com/dragonflydb/dragonfly/pull/3354
• chore(server): Introduce StringSetWrapper by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3347
• fix: do not upload offload values on a first hit by @romange in https://github.com/dragonflydb/dragonfly/pull/3360
• chore(tiering): Range functions + small refactoring by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3207
• fix: failure in test_cluster_fuzzymigration by @romange in https://github.com/dragonflydb/dragonfly/pull/3363
• fix(server): Require >=1 args to GETEX by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3366
• fix(transaction): Fix namespace access by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3364
• chore: disable compression on big values by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3358
• fix: protect OnJournalEntry with ConditionGuard by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3367
• chore: Introduce CoolQueue by @romange in https://github.com/dragonflydb/dragonfly/pull/3365
• chore: small fixes around tiering by @romange in https://github.com/dragonflydb/dragonfly/pull/3368
• chore: add a test for HeapSize() function by @romange in https://github.com/dragonflydb/dragonfly/pull/3349
• chore: simplify computation of used_mem_current by @romange in https://github.com/dragonflydb/dragonfly/pull/3372
• chore: disable cluster_fuzzymigration by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3373
• chore(replica): remove unused methods in the Replica class by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/3374
• fix(transaction): Properly store block cancel status by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3371
• chore: Track db_slice table memory instantly by @romange in https://github.com/dragonflydb/dragonfly/pull/3375
• chore: add mem test for big values and default the flag by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3369
• update: replication_acks_interval flag to 1000 by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3378
• chore: dfly_bench - print ongoing error counts by @romange in https://github.com/dragonflydb/dragonfly/pull/3382
• chore: introduce a cool queue that gradually retires cool items by @romange in https://github.com/dragonflydb/dragonfly/pull/3377
• chore: update cached stats inside PollExecution by @romange in https://github.com/dragonflydb/dragonfly/pull/3376
• fix: Fix test_take_over_seeder by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3385
• chore: reenable evictions upon insertion to avoid OOM rejections by @romange in https://github.com/dragonflydb/dragonfly/pull/3387
• fix: remove fiber guard from non atomic section by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3381
• chore: do not preempt on db_slice::RegisterOnChange by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3388
• fix: disable inline transactions when db_slice has registered callbacks by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3391
• fix: test_big_value_serialization_memory_limit shutdown timeout by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3390
• chore: set serialization_max_chunk_size to 1 byte by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3379
• chore: tiered fixes by @romange in https://github.com/dragonflydb/dragonfly/pull/3393
• fix(acl): remove none from acl categories by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3392
• chore: tiering - make Modify work with cool storage by @romange in https://github.com/dragonflydb/dragonfly/pull/3395
• fix: Fix unsupported object type rejson-rl in RedisInsight by @BagritsevichStepan in https://github.com/dragonflydb/dragonfly/pull/3384
• Revert "chore: set serialization_max_chunk_size to 1 byte (#3379)" by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3398
• chore(tiering): Move cool entry warmup to DbSlice by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3397
• fix: reenable macos builds by @romange in https://github.com/dragonflydb/dragonfly/pull/3399
• chore: Don't print password to log on replica AUTH failure by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3403
• chore: Support setting the value of replica-priority by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3400
• feat: stabilize non-coordinated omission mode by @romange in https://github.com/dragonflydb/dragonfly/pull/3407
• chore: cancel slot migrations on shutdown by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3405
• chore: add db_slice lock to protect segments from preemptions by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3406
• chore: increase timeout of regression tests by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3412
• fix: crash with NS in multi/exec #3410 by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3415
• fix: json.merge exception crash by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3409
• fix(connection): Count memchached pipelined commands by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3413
• chore: remove verbose printing of tests by @romange in https://github.com/dragonflydb/dragonfly/pull/3420
• chore: Tiered fixes by @romange in https://github.com/dragonflydb/dragonfly/pull/3401
• feat: Support non-root paths for json.merge by @romange in https://github.com/dragonflydb/dragonfly/pull/3419
• chore: skip cluster tests if redis-server wasn't found by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3416
• chore: expose metric that shows how many task submitters are blocked by @romange in https://github.com/dragonflydb/dragonfly/pull/3427
• chore: optimize SendStringArrInternal even more by @romange in https://github.com/dragonflydb/dragonfly/pull/3425
• fix(server): Implement SCRIPT GC command by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3431
• chore: retire TEST_EnableHeartBeat by @romange in https://github.com/dragonflydb/dragonfly/pull/3435
• feat(server): Support replica-announce-ip/port by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3421
• test(cluster): Migration replication test by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3417
• chore: improve replication locks by @romange in https://github.com/dragonflydb/dragonfly/pull/3436
• chore: reorganize EngineShard::Heartbeat by @romange in https://github.com/dragonflydb/dragonfly/pull/3437
• chore: fix memcached pipeline test by @dranikpg in https://github.com/dragonflydb/dragonfly/pull/3438
• chore: simplify master replication cancelation interface by @romange in https://github.com/dragonflydb/dragonfly/pull/3439
• chore: reset serialization_max_chunk_size to 0 by @kostasrim in https://github.com/dragonflydb/dragonfly/pull/3432
• feat: DEBUG REPLICA PAUSE now pauses fullsync by @romange in https://github.com/dragonflydb/dragonfly/pull/3441
• test: fix test_disconnect_replica by @BorysTheDev in https://github.com/dragonflydb/dragonfly/pull/3442
• fix: cluster_mgr.py to use CLUSTER MYID by @chakaz in https://github.com/dragonflydb/dragonfly/pull/3444
• chore: disable serialization_max_chunk_size in regtests by @romange in https://github.com/dragonflydb/dragonfly/pull/3445
• fix: properly seriailize meta buffer in SendStringArrInternal by @romange in https://github.com/dragonflydb/dragonfly/pull/3455

Full Changelog: https://github.com/dragonflydb/dragonfly/compare/v1.20.0...v1.21.0