v0.9 🚀🚀🚀

We have got a big one for you! Today we are releasing version 0.9 of SpacetimeDB. It's been a while since our last official release and we have got a huge number changes and performance improvements.

Most notably performance has improved by 10x - 100x or more for certain parts of the database! Subscription evaluation in particular has gotten almost 100 times faster! This makes an enormous difference for people shipping production games. 🏃🏃🏃

Version 0.9 is the same version that we ran the Closed Alpha for the MMOPRG BitCraft, which had 700 players playing concurrently in a single database! We are battle testing SpacetimeDB with our production game!

We're so excited to see what people build with this new version!

As a reminder you can upgrade from the previous release with the spacetime upgrade command. If you are on macOS and you've installed through brew we recommend using brew install clockworklabs/tap/spacetime to upgrade.

Highlights

• Vastly improved metrics gathering performance overhead
• Stabilized new transaction log format for a 2-3x reduction in transaction log size and much faster reply times
• Reduced unnecessary copying and cloning in core database paths
• Improved threading model for optimal performance
• Fixed a critical issue for misbehaving clients or clients with very poor latency
• Multi-column index selection for the query! macro
• Huge number of critical performance and bug fixes!

What's Changed

• Bump Rust to 1.76.0 by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/811
• test: Add performance benchmark for incremental join query by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/795
• perf(subscriptions): Add metric counter for subscribe calls by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/817
• ResultInspectExt is obsolete by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/818
• chore(datastore): Remove debug instrumentation by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/812
• Fix benchmarks by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/819
• This fixes replaying of the transaction log to no longer check constraints by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/806
• ScanIterByColRange: Avoid unnecessary ProductValue allocations by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/804
• Update upgrade-version tool by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/826
• chore(836): Remove tracing instrumentation from database iterators by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/837
• test(829): Benchmark incremental update for table subscription by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/830
• Verify that the upgrade-version tool still works for every PR by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/825
• To make eval subscription concurrent by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/807
• Remove unused pre-publish.duck file by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/841
• Fix: Publish fails if rustup not present by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/814
• test: Deduplicate utilities for creating tables by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/842
• Add workflow to tag latest docker image on release by @drogus in https://github.com/clockworklabs/SpacetimeDB/pull/756
• test(835): Benchmark full subscription eval for table scans and join by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/839
• Core: depend on "serde" from lib by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/845
• Reduce allocations for prometheus with_label_values by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/810
• fix: Wrong algebraic type in query benchmarks by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/849
• fix: Record durations for aborted and read transactions by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/850
• Fix publish issues 0.8.2 by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/853
• Fix identity subcommand ordering by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/857
• v0.8.2 Version Upgrade by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/858
• core: Ignore indexed-only constraints when updating by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/734
• Use a version label for tag builds by @drogus in https://github.com/clockworklabs/SpacetimeDB/pull/751
• Fix issues caused by disabling metrics by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/856
• Add delete_by_{nonunique_column} method to #[spacetimedb(table)] by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/859
• Fix deadlock by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/854
• Fix bench errors and include in CI by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/855
• Fix consistency issue by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/861
• Refactor some ReadColumn stuff + relational_db tests by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/847
• Added extra check to create_table transactionality test by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/866
• system_tables, mut_tx, and friends: bye bye to_product_value by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/851
• Fix bootstrapping ids for relating objects like indexes by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/777
• dedup some with_label_values + reduce cloning in bootstrapping by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/867
• Set name for Rayon threads by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/870
• Adds an isolation_level argument to begin_mut_tx calls by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/864
• chore: Remove instrumentation from DataKey deserializers by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/852
• Replace DbProgram with a leaner version that compile directly to queries without environment by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/860
• Use a copy-on-write structure for RelValue by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/869
• Fix module import warnings by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/820
• Add GitHub workflow to check for PRs with merge-blocking labels by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/862
• Script to run perf against SpacetimeDB by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/882
• HostThreadPool: don't run Rayon threads in Tokio blocking threads by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/881
• Rename types in benchmarks by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/803
• num_threads = available_concurrency by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/887
• Metric for request RTT by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/872
• serialise disconnect logic by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/886
• Run initial subscriptions evals on rayon by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/888
• Revert "Run initial subscriptions evals on rayon" by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/894
• C#: reuse Consume helper by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/704
• refactor(890): Improve magic constant for index join selection by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/895
• VM: Wrap Header in Arc to avoid cloning by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/897
• Parallelize QuerySet::eval by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/891
• Remove row_pk from client API; hash rows not row_ids on client by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/840
• perf(832): Remove redundant row deduplication in subscriptions by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/863
• acquire remove subscription lock in spawn_blocking by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/903
• Adjust benchmarks yaml to run callgrind + with better comments by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/907
• perf(831): Remove row_pk computation from query by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/908
• core: Fix host controller to not replace module if lifecyle hooks failed by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/904
• Correctly show the error for AmbiguousField and simplify the code by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/910
• Adding an index selector that take in account multi-column indexes (and improve the query! macro) by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/694
• Revert 694 by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/914
• Fix the RowCount estimation for select operator by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/900
• Remove Tables from query plans by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/912
• iter_by_col_eq: take AV by ref + impl RangeBounds<..> for &AV by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/925
• perf(747): Single query execution for multiple subscriptions by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/917
• Re-land mult-col index selection for queries by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/918
• Use recv_many in ws_client_actor by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/913
• proto changes by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/924
• Single-table subscription queries: plan once, run repeatedly by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/928
• Improve generate help text by @jdetter in https://github.com/clockworklabs/SpacetimeDB/pull/934
• Avoid Header::find_pos_by_name in eval_incr. by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/935
• Kick clients that are backing up their message channel by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/930
• chore: Remove debug instrumentation from hot path by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/936
• Incremental joins: compile once, run repeatedly. by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/938
• chore: Header does not need to be Ord by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/942
• Remove perfcnt for now by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/941
• perf: Make op_type special case fast for selections by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/943
• chore: Remove instrumentation from subscription eval hot path by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/945
• perf: Update magic constant for join rewrite by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/944
• Truly remove perfcnt by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/946
• Also poll handle_queue while flushing by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/947
• refactor: Incremental evaluation tests for index joins by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/952
• Increase sdk tests' timeout to 60 seconds by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/956
• fix: Incremental evaluation for index joins by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/957
• test(954): Incremental evaluation for index joins by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/958
• perf(816): Compile inner joins ahead of time for incremental evaluation by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/964
• Fix bench for location, restore single-column indexes by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/967
• Rewrite smoketests as python unittests by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/692
• Code motion: QueryCode => QueryExpr; CrudCode => CrudExpr by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/975
• iter_filtered_chunks: avoid PVs by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/978
• Simplify RelOps<'a> for CatalogCursor<I> by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/979
• spacetime publish: Add --wasm-file flag by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/883
• Nix From<IndexScan> for ColumnOp & fix the multi-col bug by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/980
• Add a Testing section to the PR template by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/898
• Misc tweaks to tools/perf.sh by @bfops in https://github.com/clockworklabs/SpacetimeDB/pull/937
• fix BufReader for Cursor impl by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/982
• Remove #[tracing::instrument...] from InstanceEnv::*_by_col_eq by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/983
• core: Fix schema checks in database updates, again by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/974
• Log a warning when doing iter_by_col_range without an index by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/971
• Distinguish between inner and semijoins in QueryExpr AST. by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/969
• perf(833): Do not clone query plan during execution by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/981
• perf(813): Avoid materialization of product values in subscriptions by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/959
• perf(933): Clone bsatn instead of product values in incremental update by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/951
• fix: Optimize query plan in iter_filtered_chunks by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/939
• Don't do alignment computations during BFLATN ser/de by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/986
• Remove useless usages of RowCount by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/987
• Nuke to_mem_table_with_op_type by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/990
• incr-join, find_updates: avoid unncecessary clones & use partition by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/988
• test: Subscriptions should not drop read lock early by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/995
• fix(996): Do not release database lock early for subscriptions by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/997
• fix: Consistency test for subscription message ordering by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1007
• Show the error text of the server when a sql call fails on cli by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/1004
• fix(1009): Multi-column index selection for query macro by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1012
• test: Multi-column index selection through query macro by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1001
• BFLATN -> BSATN fast-path for fixed-length rows by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1005
• Bump Rust to 1.77 + fix warnings + use Bound::map by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1020
• perf(1024): Remove serialization from tx execution thread by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1027
• Add AlgebraicValue::take + move test-code in btree_index to tests by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1028
• Split DatabaseTableUpdate in deletes/inserts vecs by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1019
• Fixed an issue which caused metrics to only be recorded for on-disk databases by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/901
• Binary WebSocket API: Brotli-compress all outgoing messages by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1026
• (1/3) Commitlog: Base implementation "sans I/O" by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/919
• (2/3) Commitlog: Add I/O based on regular files by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/920
• (3/3) Commitlog: Add canonical txdata payload by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/921
• Make 'op_type' an Enum by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/1032
• Prune deps from bindings dependency tree by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/1014
• Bump to reqwest 0.12 (uses hyper 1.0) by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/1031
• Implement 'sql' parsing for simple 'Enums' by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/1018
• C#: add experimental NativeAOT-LLVM support by @RReverser in https://github.com/clockworklabs/SpacetimeDB/pull/713
• Batch metrics with ctx by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/916
• Remove dead code in subscription by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1039
• Consistently use --identity -i for identity args to CLI by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1041
• use nohasher_hash and ahash instead of siphash13 by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1040
• Preserve debuginfo (e.g. symbols) in wasm-opt by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/1013
• Reject large SQL queries by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1037
• IndexSemijoin::next: do a bit less cloning and work by @Centril in https://github.com/clockworklabs/SpacetimeDB/pull/1044
• Detect unsatisfiable range queries; warn and short-circuit. by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1036
• Metrics were not measuring the time it took to commit the tx by @cloutiertyler in https://github.com/clockworklabs/SpacetimeDB/pull/1045
• fix(1051): Reset queue length metrics on module restart by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1053
• fix: Disable unused metrics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1056
• refactor(1058): Remove feature flag for disabling metrics by @joshua-spacetime in https://github.com/clockworklabs/SpacetimeDB/pull/1059
• Drop commitlog logging to trace to avoid spamming host logs by @gefjon in https://github.com/clockworklabs/SpacetimeDB/pull/1073
• disable iter metrics by @Shubham8287 in https://github.com/clockworklabs/SpacetimeDB/pull/1074
• Move lib::{name,recovery} to client-api-messages by @coolreader18 in https://github.com/clockworklabs/SpacetimeDB/pull/570
• core: Make init_module_host consistent with update_module_host by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/977
• Durability: Traits and implementation in terms of commitlog by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/922
• Implement 'sql' parsing for 'Identity, Address' in hex format by @mamcx in https://github.com/clockworklabs/SpacetimeDB/pull/786
• Swap the location of tags in the BFLATN encoding by @kazimuth in https://github.com/clockworklabs/SpacetimeDB/pull/1063
• core: Integrate new commitlog + durability by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/926
• core: More compact TxData by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/950
• core,commitlog: Re-instantiate commitlog disk usage reporting by @kim in https://github.com/clockworklabs/SpacetimeDB/pull/955

Full Changelog: https://github.com/clockworklabs/SpacetimeDB/compare/v0.8.2-beta-hotfix7...v0.9.0-beta