MyGit

v18.0.0-rc1

vitessio/vitess

版本发布时间: 2023-10-03 22:24:06

vitessio/vitess最新发布版本:v21.0.0-rc2(2024-10-23 13:13:47)

Release of Vitess v18.0.0-rc1

Summary

Table of Contents

Major Changes

Breaking Changes

Local examples now use etcd v3 storage and API

In previous releases the local examples were explicitly using etcd v2 storage (etcd --enable-v2=true) and API (ETCDCTL_API=2) mode. We have now removed this legacy etcd usage and instead use the new (default) etcd v3 storage and API. Please see PR #13791 for additional info. If you are using the local examples in any sort of long-term non-testing capacity, then you will need to explicitly use the v2 storage and API mode or migrate your existing data from v2 to v3.

New command line flags and behavior

VTOrc flag --allow-emergency-reparent

VTOrc has a new flag --allow-emergency-reparent that allows the users to toggle the ability of VTOrc to run emergency reparent operations. Users that want VTOrc to fix the replication issues, but don't want it to run any reparents should start using this flag. By default, VTOrc will be able to run EmergencyReparentShard. Users must specify the flag to false to change the behaviour.

VTOrc flag --change-tablets-with-errant-gtid-to-drained

VTOrc has a new flag --change-tablets-with-errant-gtid-to-drained that allows users to choose whether VTOrc should change the tablet type of tablets with errant GTIDs to DRAINED. By default, it is disabled.

This feature allows users to configure VTOrc such that any tablet that encounters errant GTIDs is automatically taken out of the serving graph. These tablets can then be inspected for what the errant GTIDs are, and once fixed, they can rejoin the cluster.

ERS sub flag --wait-for-all-tablets

Running EmergencyReparentShard from the vtctldclient has a new sub-flag --wait-for-all-tablets that makes EmergencyReparentShard wait for a response from all the tablets. Originally EmergencyReparentShard was meant only to be run when a primary tablet is unreachable. We have realized now that there are cases when the replication is broken but all the tablets are reachable. In these cases, it is advisable to call EmergencyReparentShard with --wait-for-all-tablets so that it does not ignore one of the tablets.

VTGate GRPC stream execute session flag --grpc-send-session-in-streaming

This flag enables transaction support on StreamExecute api. Once enabled, VTGate StreamExecute gRPC api will send session as the last packet in the response. The client should enable it only when they have made the required changes to expect such a packet.

It is disabled by default.

Experimental Foreign Key Support

A new field foreignKeyMode has been added to the VSchema. This field can be provided for each keyspace. The VTGate flag --foreign_key_mode has been deprecated in favour of this field.

There are 3 foreign key modes now supported in Vitess -

  1. unmanaged - This mode represents the default behaviour in Vitess, where it does not manage foreign keys column references. Users are responsible for configuring foreign keys in MySQL in such a way that related rows, as determined by foreign keys, reside within the same shard.
  2. managed [EXPERIMENTAL] - In this experimental mode, Vitess is fully aware of foreign key relationships and actively tracks foreign key constraints using the schema tracker. Vitess takes charge of handling DML operations with foreign keys cascading updates, deletes and verifying restrict. It will also validate parent row existence. This ensures that all the operations are logged in binary logs, unlike MySQL implementation of foreign keys. This enables seamless integration of VReplication with foreign keys. For more details on what operations Vitess takes please refer to the design document for foreign keys.
  3. disallow - In this mode Vitess explicitly disallows any DDL statements that try to create a foreign key constraint. This mode is equivalent to running VTGate with the flag --foreign_key_mode=disallow.

Upgrade process

After upgrading from v17 to v18, the users should specify the correct foreign key mode for all their keyspaces in the VSchema using the new property. Once this change has taken effect, the deprecated flag --foreign_key_mode can be dropped from all the VTGates.

VTAdmin

vtadmin-web updated to node v18.16.0 (LTS)

Building vtadmin-web now requires node >= v18.16.0 (LTS). Breaking changes from v16 to v18 are listed in https://nodejs.org/en/blog/release/v18.0.0, but none apply to VTAdmin. Full details on v18.16.0 are listed on https://nodejs.org/en/blog/release/v18.16.0.

Deprecations and Deletions

Deprecated Command Line Flags

Throttler related vttablet flags:

Cache related vttablet flags:

Buffering related vtgate flags:

Cache related vtgate flags:

VTGate flags:

VDiff v1:

VDiff v2 was added in Vitess 15.0 and marked as GA in 16.0. The legacy v1 client command is now deprecated in Vitess 18.0 and will be removed in 19.0. Please switch all of your usage to the new VDiff client command ASAP.

Deprecated Stats

The following EmergencyReparentShard stats are deprecated in v18.0 and will be removed in v19.0:

These metrics are replaced by new reparenting stats introduced in v18.0.

VTBackup stat DurationByPhase is deprecated. Use the binary-valued Phase stat instead.

Deleted Command Line Flags

Flags in vtcombo:

Flags in vtctldclient ApplySchema:

Flags in vtctl ApplySchema:

Flags in vtgate:

Flags in vttablet:

Flags in vtctld:

Flags in vtorc:

Deleted v3 planner

The Gen4 planner has been the default planner since Vitess 14. The v3 planner was deprecated in Vitess 15 and has now been removed in this release.

Deleted k8stopo

The k8stopo has been deprecated in Vitess 17, also see https://github.com/vitessio/vitess/issues/13298. With Vitess 18 the k8stopo has been removed.

Deleted vtgr

The vtgr has been deprecated in Vitess 17, also see https://github.com/vitessio/vitess/issues/13300. With Vitess 18 vtgr has been removed.

Deleted query_analyzer

The undocumented query_analyzer binary has been removed in Vitess 18, see https://github.com/vitessio/vitess/issues/14054.

New stats

VTGate Vindex unknown parameters

The VTGate stat VindexUnknownParameters gauges unknown Vindex parameters found in the latest VSchema pulled from the topology.

VTBackup Phase stat

In v17, the vtbackup stat DurationByPhase stat was added measuring the time spent by vtbackup in each phase. This stat turned out to be awkward to use in production, and has been replaced in v18 by a binary-valued Phase stat.

Phase reports a 1 (active) or a 0 (inactive) for each of the following phases:

To calculate how long vtbackup has spent in a given phase, sum the 1-valued data points over time and multiply by the data collection or reporting interval. For example, in Prometheus:

sum_over_time(vtbackup_phase{phase="TakeNewBackup"}) * <interval>

VTBackup PhaseStatus stat

PhaseStatus reports a 1 (active) or a 0 (inactive) for each of the following phases and statuses:

Backup and restore metrics for AWS S3

Requests to AWS S3 are instrumented in backup and restore metrics. For example:

vtbackup_backup_count{component="BackupStorage",implementation="S3",operation="AWS:Request:Send"} 823
vtbackup_backup_duration_nanoseconds{component="BackupStorage",implementation="S3",operation="AWS:Request:Send"} 1.33632421437e+11
vtbackup_restore_count{component="BackupStorage",implementation="S3",operation="AWS:Request:Send"} 165
vtbackup_restore_count{component="BackupStorage",implementation="S3",operation="AWS:Request:Send"} 165

VTCtld and VTOrc reparenting stats

New VTCtld and VTorc stats were added to measure frequency of reparents by keyspace/shard:

Also, the reparent_shard_operation_timings stat was added to provide per-operation timings of reparent operations.

VTTablet

New ResetSequences rpc

A new VTTablet RPC ResetSequences has been added, which is being used by MoveTables and Migrate for workflows where a sequence table is being moved (https://github.com/vitessio/vitess/pull/13238). This has an impact on the Vitess upgrade process from an earlier version if you need to use such a workflow before the entire cluster is upgraded.

Any MoveTables or Migrate workflow that moves a sequence table should only be run after all vitess components have been upgraded, and no upgrade should be done while such a workflow is in progress.

New Dry-run/monitoring-only mode for the transaction throttler

A new CLI flag --tx-throttler-dry-run to set the Transaction Throttler to monitoring-only/dry-run mode has been added. If the transaction throttler is enabled with --enable-tx-throttler and the new dry-run flag is also specified, the tablet will not actually throttle any transactions; however, it will increase the counters for transactions throttled (vttablet_transaction_throttler_throttled). This allows users to deploy the transaction throttler in production and gain observability on how much throttling would take place, without actually throttling any requests.

Docker

Bookworm added and made default

Bookworm was released on 2023-06-10, and will be the new default base container for Docker builds. Bullseye images will still be built and available as long as the OS build is current, tagged with the -bullseye suffix.

Buster removed

Buster LTS supports will stop in June 2024, and Vitess v18.0 will be supported through October 2024. To prevent supporting a deprecated buster build for several months after June 2024, we are preemptively removing Vitess support.

Durability Policies

New Durability Policies

2 new inbuilt durability policies have been added to Vitess in this release namely semi_sync_with_rdonly_ack and cross_cell_with_rdonly_ack. These policies are exactly like semi_sync and cross_cell respectively, and differ just in the part where the rdonly tablets can also send semi-sync ACKs.


The entire changelog for this release can be found here.

The release includes 361 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @Juneezee, @adsr, @ajm188, @app/dependabot, @app/github-actions, @app/vitess-bot, @arvind-murty, @austenLacy, @brendar, @davidpiegza, @dbussink, @deepthi, @derekperkins, @ejortegau, @frouioui, @harshit-gangal, @hkdsun, @jfg956, @jspawar, @mattlord, @maxenglander, @mdlayher, @notfelineit, @olyazavr, @pbibra, @peterlyoo, @rafer, @rohit-nayak-ps, @shlomi-noach, @systay, @timvaillancourt, @vmg, @yields

相关地址:原始地址 下载(tar) 下载(zip)

1、 vitess-18.0.0-rc1-6ab165a.tar.gz 539.12MB

2、 vitess-18.0.0_rc1-6ab165a.x86_64.rpm 538.7MB

3、 vitess_18.0.0-rc1-6ab165a_amd64.deb 539.74MB

查看:2023-10-03发行的版本