2.3.3 Release Note

Bug fix

Core

• [Translation] [Spark] Fix SeaTunnelRowConvertor fail to convert when schema contains row type (#5170)

Connector

• [Connector] [Jdbc] Fix the problem of JdbcOutputFormat database connection leak (#4802)
• [Connector-V2] [AmazonDynamoDB] Fix the problem that all table data cannot be obtained (#5146)
• [Connector-V2] [CDC-base] Fix cdc connection does not close (#4922)
• [Connector-V2] [Clickhouse] Fix clickhouse source connector read Nullable() type is not null,example:Nullable(Float64) while value is null the result is 0.0 (#5080)
• [Connector-V2] [ClickhouseFile] Fix ClickhouseFile write file failed when field value is null (#4937)
• [Connector-V2] [Doris] Add streamload Content-type for doris URLdecode error (#4880)
• [Connector-V2] [E2E] Fix maven scope (#4901)
• [Connector-V2] [File-Base] Fix Hadoop Kerberos authentication related issues (#5171)
• [Connector-V2] [Flink-runtime] Flink register table Environment The running mode is set tojob.mode (#4826)
• [Connector-V2] [HbaseSink] Fix default timestamp (#4958)
• [Connector-V2] [Hive] Support user-defined hive-site.xml (#4965)
• [Connector-V2] [Jdbc] Support postgresql json type (#5194)
• [Connector-V2] [Jdbc] Fix XA Transaction bug (#5020)
• [Connector-V2] [Jdbc] Modify error comment in JdbcSourceConfigFactory (#5088)
• [Connector-V2] [Jdbc-Oracle] Fix the Oracle number type mapping problem (#5209)
• [Connector-V2] [Jdbc-Postgres] PostgreSQL versions below 9.5 are compatible use cdc sync problem (#5120)
• [Connector-V2] [JindoOssFile] Fix plugin-mapping.properties (#5215)
• [Connector-V2] [LocalFileSink] Fix LocalFile Sink file_format_type (#5118)
• [Connector-V2] [Mongodb] Fix document error content and remove redundant code (#4982)
• [Connector-V2] [Mongodb] Compatible with historical parameters (#4997)
• [Connector-V2] [MongoDB-CDC] Incremental data kind error in snapshot phase (#5184)
• [Connector-V2] [MongoDB-CDC] Solve startup resume token is negative (#5143)
• [Connector-V2] [MongoDB-CDC] Refine data format to adapt to universal logic (#5162)
• [Connector-V2] [MongoDB-CDC] Fix array index anomalies caused by #5057 (#5195)
• [Connector-V2] [MySql-CDC] Fix jdbc connection leak for mysql (#5037)
• [Connector-V2] [MySql-CDC] Fix mysql bit column to java byte (#4817)
• [Connector-V2] [MySql-CDC] Fix listener not released when BinlogClient reuse (#5011)
• [Connector-V2] [MySql-CDC] ServerId from int to long (#5033) (#5035)
• [Connector-V2] [Paimon] Bump paimon-bundle version to 0.4.0-incubating (#5219)
• [Connector-V2] [Redis] Fix redis nodes format error (#4981)
• [Connector-V2] [StarRocks] Fix StarRocksJsonSerializer will transform array/map/row to string (#5281)
• [Connector-V2] [StarRocks] Fix code style (#4966)

Zeta(ST-Engine)

• [Zeta] Resolved the issue causing checkpoints to halt on tolerable-failure=0 (#5263)
• [Zeta] Fix the issue of two identical IDs appearing when executing seatunnel.sh -l as the job resumes (#5191)
• [Zeta] Fix SinkFlowLifeCycle without init lastCommitInfo (#5152)
• [Zeta] Fix MultipleTableJobConfigParser ignore env option (#5067)
• [Zeta] Fix checkpoint error report without msg (#5137)
• [Zeta] [REST-API] Fix class not found exception (#5063)
• [Zeta] [REST-API] Fix browser long type intercept (#5267)
• [Zeta] Fix task state memory leak (#5139)
• [Zeta] Fix when execute seatunnel.sh, console may print error (#4984)
• [Zeta] Fixed the issue of duplicated metrics caused by job fault tolerance or restore (#5214)
• [Zeta] Avoid Redundant Job Submissions by Checking Job Status (#5229)
• [Zeta] Fix the serialization issue of GetMetricsOperation during multi-node operation (#5206)
• [Zeta] Fix after the savepoint job is restored, the checkpoint file cannot be generated #4985 (#5051)
• [Zeta] Fixed multi-table job data loss and latency issues (#149) (#5031)
• [Zeta] Fix the HdfsStorage can not delete checkpoint file #5046 (#5054)
• [Zeta] Fix if the initialization task throws an error and the task can not return to master (#4888)

TEST

• [Test] Split updated modules integration test for part 5 (#5208)
• [Test] Add code style check when docs changed (#5183)
• [Test] Turn on the testSavepoint test (#5199)
• [Test] Fix the CI Job name error (#5032)
• [Test] [E2E] Fix testGetErrorInfo case error (#5282)
• [Test] [Kafka] Kafka e2e error (#5200)
• [Test] [MongoDB-CDC] Fix mongodb cdc e2e instability (#5128)
• [Test] Declare files that will always have UNIX line endings on checkout (#5221)
• [Test] Fixed a minor bug (#5274)
• [Test] Fixed IMap file storage e2e bug (#5237)
• [Test] Split updated modules integration test for part 4 (#5028)
• [Test] Fix zeta ci error (#5254)

Others

• Update the project description (#4967)
• Update version to 2.3.3-SNAPSHOT (#5043)
• Fix com.google.common.base.Preconditions to seatunnel shade one (#5284)
• Fix style error (#5280)
• Fix code style (#5092)

Improve

Core

• [API & Zeta] Using connector custom serializer encode/decode states (#5238)
• [API]Improve savemode api (#4767)

Connector

• [CDC Base] Add some split parameters to the optionRule (#5161)
• [CDC Base] Support exactly-once of cdc and fix the BinlogOffset comparing bug (#5057)
• [CDC Base] Add a fast sampling method that supports character types (#5179)
• [CDC Base] Change driver scope to provider (#5002)
• [CDC Base] Remove driver for cdc connector (#4952)
• [Connector-V2] [File] Optimize files commit order (#5045)
• [Connector-V2] [HiveSink] Remove drop partition when abort (#4940)
• [Connector-V2] [Jdbc] Remove scheduler in JDBC sink #4736 (#5168)
• [Connector-V2] [Jdbc] Check url not null throw friendly message (#5097)
• [Connector-V2] [Jdbc] Support for Decimal types as splict keys (#4634)
• [Connector-V2] [Mongodb] Sink support transaction update/writing (#5034)
• [Connector-V2] [Mongodb] Optimize reading logic (#5001)
• [Connector-V2] [Neo4j] Supports neo4j sink batch write and update docs (#4841)
• [Connector-V2] [OSS-Jindo] Optimize jindo oss connector (#4964)
• [Connector-V2] [Tablestore sink] Remove scheduler in Tablestore sink (#5272)

Transform

• [Transform-V2] [SQL] Support use catalogTableName as SQL expression (#5273)

CI

• [CI/CD] Remove 'paths-ignore', enable the code style check for markdown files (#5286)

Zeta(ST-Engine)

• [Zeta] Upgrade the key log output level (#4993)
• [Zeta] Don't trigger handleSaveMode when restore (#5192)
• [Zeta] Move checkpoint notify complete in checkpoint stage (#5185)
• [Zeta] Add sleep for Task to reduce CPU cost (#5117)
• [Zeta] Add an interface for batch retrieval of JobMetrics (#4576)
• [Zeta] Move imap storage file dependency packages to submodules (#5218)
• [Zeta] Improve CheckpointCoordinator log error when report error from task (#178) (#5134)
• [Zeta] Improve CheckpointCoordinator notify complete when restore (#5136)

Feature

Connector

• [Connector-V2] [CDC-base] Support tables without primary keys (with unique keys) (#163) (#5150)
• [Connector-V2] [CDC-base] Support disable/enable exactly once for INITIAL (#4921)
• [Connector-V2] [CDC-base] Change the time zone to the default time zone (#5030)
• [Connector-V2] [CDC-base] Support string type shard fields (#5147)
• [Connector-V2] [Clickhouse] Add clickhouse connector time zone key,default system time zone (#5078)
• [Connector-V2] [Clickhouse] Clickhouse writes with checkpoints (#4999)
• [Connector-V2] [Doris] Add doris factory type convertor (#5061)
• [Connector-V2] [File] Add config of 'file_filter_pattern', which used for filtering files (#5153)
• [Connector-V2] [File] Add cos source&sink (#4979)
• [Connector-V2] [Hive] Hive add option because web need (#5154)
• [Connector-V2] [Jdbc] Add JDBC Catalog auto create table (#4917)
• [Connector-V2] [Jdbc] Add oceanbase dialect factory (#4989)
• [Connector-V2] [Jdbc] JDBC source support string type as partition key (#4947)
• [Connector-V2] [Jdbc-DM] Add DM upsert support (#5073)
• [Connector-V2] [Kafka] Support read debezium format message from kafka (#5066)
• [Connector-V2] [Kafka] Connector-kafka source support data conversion extracted by kafka connect source (#4516)
• [Connector-V2] [Mongodb] Mongodb support cdc sink (#4833)
• [Connector-V2] [Mongodb-CDC] Support source mongodb cdc (#4923)
• [Connector-V2] [Mysql-CDC] Conversion of tinyint(1) to bool is supported (#5105)
• [Connector-V2] [Redis] Support redis to set expiration time (#4975)

Zeta(ST-Engine)

• [Zeta] [CDC-base] Support schema evolution framework(DDL) (#5125)
• [Zeta] [REST-API] Add REST API To Submit Job (#5107)
• [Zeta] Checkpoint support hdfs ha mode (#4942)
• [Zeta] The expiration time of a historical Job can be config (#5180)
• [Zeta] Support history service record job execute error (#5114)
• [Zeta] Configuration files support user variable replacement (#4969)

Transform

• [Transform-V2] [SQL] Support 'select *' and 'like' clause for SQL Transform plugin (#4991)

CI

• [CI] Update action config to support run CI on fork repo (#5065)
• [E2E] [FtpFile] Add ftp file e2e test case (#4647)

Docs

• [Connector-V2][Clickhouse] Reconstruct the clickhouse connector doc (#5085)
• [Connector-V2][Doris] Reconstruct the Doris connector document (#4903)
• [Connector-V2][HDFS] Refactor connector-v2 docs using unified format HDFS (#4871)
• [Connector-V2][Http] Reconstruct the Http connector document (#4962)
• [Connector-V2][Hudi] Reconstruct the Hudi connector document (#4905)
• [Connector-V2][Jdbc] Change the line boundary store value type to BigDecimal (#4900)
• [Connector-V2][Jdbc-DB2、Jdbc-Vertica] Improve DB2 Source Vertica Source & DB2 Sink Vertica Sink document (#5102)
• [Connector-V2][Jdbc-Oracle] Add JDBC Oracle Documentation (#5239)
• [Connector-V2][Kafka] Reconstruct the kafka connector document (#4778)
• [Connector-V2][MyHours] Reconstruct the MyHours connector document (#5129)
• [Connector-V2][PostgreSQL] Refactor connector-v2 docs using unified format PostgreSQL #4590 (#4757)
• [Connector-V2][S3File] Improve S3File Source & S3File Sink document (#5101)
• [Connector-V2][SelectDB-Cloud] Reconstruct the SelectDB-Cloud connector document (#5130)
• [Connector-V2][StarRocks] Reconstruct the StarRocks connector document (#5132)
• [Connector-V2][StarRocks] StarRocks nodeUrls property name fix (#4951)
• [README] Remove useless github workflow, and adjust description of 'engineering structure' (#4305)
• [Zeta] Add savepoint doc (#5081)
• Update bug-report.yml (#5160)
• Add Value types in Java to Schema feature (#5087)
• Documentation and partial word optimization (#4936)
• Change repository name from incubator-seatunnel to seatunnel (#4868)
• Update .asf.yaml (#5242)
• Update the Readme (#4968)
• Set Up with Kubernetes, dockerfile document error in constructing docker image (#5022)
• Fix Dockerfile and seatunnel-flink.yaml in Set Up with Kubernetes (#4793)
• Change the transform website url (#4954)
• Update quick-start-seatunnel-engine.md (#4943)
• Remove incubating keyword in document (#5257)