RELEASE NOTE

Version information

About this version

In V3.1.2, we provided free GUI developer tool (ODC), maintenance and monitoring tool (OCP), data migration tool (OMS), OBDUMPER, and OBLOADER for OceanBase users. OceanBase supported online cross-version upgrade. You can specify the target upgrade version by using OBD. OBD will automatically find the best upgrade solution and do the upgrade for you. The upgrade process does not affect your business. We optimized the update of hot row data and improved the performance by 300%. We optimized the delimination mechanism of SQL PLAN for partition tables. We optimized the underlying communication code. We added multiple SQL MODE functions and system functions.

New features

• Developer center (ODC): OceanBase Developer Center Community Version is a development platform for OceanBase database. ODC supports database objects visual management, intelligent syntax, session management, recycle bin management, data import or export, and resource and permission management. Key features of ODC are:

  • Multi-window development mode: Provided SQL window, anonymous block window, and command line window development modes. You can choose your development mode based on your habits. The SQL window and anonymous block window supports advanced features such as intelligent auto-completion, syntax highlighting, and formatting, etc.
  • Rich result set management: Supported the interactive operation of result set tabulation. Supported exporting all or part of the query result set in SQL window.
  • Flexible import and export: Supported SQL, CSV, and other format data files. You can import or export a single table, multi-table, or all data in a database at one time. Supported importing or exporting the defined objects and tables separately or together.
  • Collaboration management: Supported built-in control platform. You can edit the roles and permissions for user groups, and manage connection resources to improve your collaboration efficiency.

• Maintenance and monitoring tool (OCP): OceanBase Cloud Platform (OCP) is a cluster management tool for OceanBase Database. With OCP, you can manage and operate your hosts, networks, software packages, and the full lifecycle for your clusters and tenants. With OCP, you can also decrease your IT O&M cost. Key features of OCP are:

  • Cluster management: Supported cluster installation, operation, and maintenance, performance monitoring, configuration, upgrade, deletion, adding or removing hosts.
  • Tenant management: Supported creating tenant, tenant topology, performance monitoring, session management, and parameter management.
  • Monitoring and alerts: Supported monitoring clusters, tenants, and hosts. You can use the built-in alerts or customize your alerts. You can use templates to configure HTTP or customize script channels for various message channels.
  • System management: Supported viewing tasks and managing running tasks. You can customize parameters based on your business.
  • Security: Supported creating, altering, and deleting database users and roles, granting and revoking permissions for roles. You can manage OCP users' settings, passwords, and alarm subscriptions through User Center.

• OceanBase Migration Service (OMS): You can use the OceanBase Data Migration Tool (OMS) Community Edition to migrate data from heterogeneous databases to OceanBase databases. OMS supports online data migration and real-time synchronization of incremental data. All changes to OceanBase database are synchronized in real-time to the source database before migration. You can use the management console to create data migration projects, customize task configuration and monitor the OMS transfer components. Key features of OMS are:

  • Database objects migration: Supported exporting, switching, and creating tables, indexes, constraints.
  • Full data migration.
  • Incremental data migration: Supported real-time incremental data synchronization from the source database to the target database through log parsing.
  • Full data verification: Supported full data verification between source data and target data. Provided scripts to correct the inconsistent data.
  • Reverse incremental migration: When the business switches the application to the (source) target data, the incremental data in the target database is synchronized back to the source database in real-time.
  • Supported MySQL versions: 5.6, 5.7.
  • Supported customized alarm settings.
  • O&M monitoring: Supported monitoring on hosts and components.

• OBLOADER/OBDUMPER: You can use OBLOADER or OBDUMPER to import or export data. To improve concurrent performance for OBLOADER or OBDUMPER, you can specify the concurrency number, the paging query records number, and the batch number based on your kernel load capacity. OBLOADER and OBDUMPE support exporting SQL by partition, specified conditions, single table, multi-table, full database, and customized queries. OBLOADER and OBDUMPE support data pre-processing, field mapping import. OBLOADER and OBDUMPE are compatible with SQL, CSV, CUT, etc.

• Supported online rotating upgrade across versions: You can upgrade your OceanBase database with the obd cluster upgrade command. OBD automatically identifies the upgrade path between matching versions and does a rotating upgrade for each Zone. For more information, see OBD document.

• Optimized data hotspot row: To update hotspot data, OceanBase used lock for write and released row-level locks before filling logs. This increased the concurrent update performance from 3000TPS to 9000TPS and expanded the upper limit for hotspot data processing.

• Optimized the elimination mechanism for partition table SQL PLAN: In distributed scenario plan evaluation, there is no accurate feedback on the number of rows in the remote node partition. And there is a risk of plan degradation. OceanBase uses the final SQL execution time feedback to negatively optimize the execution plan. When the average execution time of the sampled query exceeds twice the first execution time, the PLAN is targeted to be eliminated and a new SQL PLAN is generated to solve the bad performance caused by the degradation of the SQL PLAN.

• Optimized the communication clock stability: OceanBase uses CPU clock (SYS_clock_gettime) to get the timeout time for its underlying communication framework. However, the upper SQL layer uses OS clock (get_timeofday) to record the timeout time. The difference between these two times may cause the upper business layer to reset the original error code for the bottom layer. With the timeout difference tolerance design, OceanBase reproduces the system error and improves the error locating efficiency.

• Compatibility with MySQL: Supported some new system functions:

  • VALIDATE_PASSWORD_STRENGTH(): This function returns an integer to indicate how strong the password is. The return value ranges from 0 (weak) to 100 (strong). For example, SELECT VALIDATE_PASSWORD_STRENGTH('Password@123');。
  • UUID_SHORT(): Returns a “short” universal identifier as a 64-bit unsigned integer. For example, SELECTUUID_SHORT();。
  • TO_BASE64() and FROM_BASE64(): TO_BASE64() returns the argument converted to a base-64 string. For example, SELECT TO_BASE64('12345'). FROM_BASE64() decodes base64 encoded string and return result. For example, SELECT FROM_BASE64('MTIZNDU=');.
  • DAY(): Returns the day of the month (0-31). For example, SELECT DAY('2021-12-12');.
  • ANY_VALUE(): Suppress ONLY_FULL_GROUP_BY value rejection.

• Open sourced OBKV TABLE JAVA Client: Provided direct access to the underlying OceanBase database stored table data through the TABLE API interface protocol. You can call this API as a JAR package in development. TABLE API supports authentication, add, delete, check, and append operations. For more information, see GitHub repo

• Open sourced OBKV HBASE API: Based on the basic interface protocol provided by the TABLE API, we added the wide table model (WideColumn) for the data model, optimized the filter logic, supported KEY partitioning, RANGE partitioning, prefix partitioning, and TTL. Compatibility with HBase API client is coming soon.

• Supported UOS platform operating system: OceanBase and obproxy are fully compatible with the mainstream X86 platform operating system. For more information, see OceanBase GitHub repo.

• Enhanced administrator tool ob_admin: supported parsing and explicitly exporting block_file files, archive files, backup meta information, and data. The input command is ob_admin dumpsst/archive_tool/dump_backup. For more information, see ob_admin document.

• Enhanced OceanBase Community Edition documentation. Documentation includes architecture design principles, performance tuning, troubleshooting, daily maintenance guidance,etc.

Bugfixes

• Fixed __all_virtual_memory_info returs incomplete tenant information. This is caused by MEMOEY APPEND.
• Fixed global index creation failure caused by killing observer. OceanBase triggers retry for this scenario to make sure a successful index creation.
• Fixed the error triggered by writing long column values.
• Fixed compiled error on CentOS 8.3 and 7.9 platform. For more information, see Issue 714 and Issue 715.

版本信息

概要说明

在 V3.1.2 版本，我们为 OceanBase 用户提供了免费的图形界面开发者工具（ODC）、运维监控云平台（OCP）、数据迁移工具（OMS）、导数工具（OBLOADER 和 OBDUMPER） 。OceanBase 支持在线跨版本轮转升级，通过 OBD 指定需要升级到的版本，自动完成最佳升级路径解析和应用升级，全过程无需人工干预且不影响业务运行。优化了热点行数据更新，性能提升了 300%。优化了分区表 SQL PLAN 淘汰机制。优化了底层通信代码等。新增了多个 SQL MODE 和系统函数。

特性更新

• 开发者中心（ODC）：OceanBase 社区版开发者中心（ODC）是为 OceanBase 数据库量身打造的开发平台。为数据库开发者提供了数据库对象的可视化管理、智能语法、会话管理、回收站管理、导入导出、资源和权限统一管理等多种能力。ODC 核心功能如下：

  • 多窗口开发模式：ODC 提供 SQL 窗口、匿名块窗口和命令行窗口开发模式。您可以按照自身习惯选择打开对应模式窗口进行开发，SQL 窗口与匿名块窗口支持智能联想提示、语法高亮、格式化等高级功能。
  • 丰富的结果集管理：支持对结果集表格化的交互操作，可导出全部和部分的 SQL 窗口内的查询结果集。
  • 灵活的导入导出：支持 SQL、CSV 和其他数据格式的文件，可以一次导入和导出单表、多表或全库数据。支持定义对象和表数据单独或混合导入导出。
  • 协作管理：内置管控平台，您可以编辑用户群体的角色和权限，管理连接资源，提升企业协作管理效率。

• 运维监控工具（OCP）：OceanBase 社区版运维监控工具（OCP）是为 OceanBase 数据库量身打造的企业级数据库集群管理平台工具。您可以使用 OCP 管理和运维主机、网络、软件包等资源，管理集群和租户的全生命周期，降低 IT 运维成本。OCP 的核心功能如下：

  • 集群管理：支持集群的安装、运维、性能监控、配置、升级、删除、主机添加和删除等功能。
  • 租户管理：支持租户的创建、结构拓扑图、性能监控、会话管理和参数管理等。
  • 监控告警：支持集群、租户和主机三种维度的监控告警。您可以使用内置告警项或自定义配置告警项，可以选择 HTTP 通道或者脚本通道传递告警消息。
  • 系统管理：支持查看和管理运行中的任务，根据业务自定义参数配置。
  • 安全 ：支持创建、修改或删除数据库用户和角色，赋予和回收角色的权限。您可以通过用户中心管理 OCP 用户的个人设置、密码和告警订阅等信息。

• 数据迁移工具（OMS）：您可以使用 OceanBase 社区版数据迁移工具（OMS）实现异构数据库与 OceanBase 数据库的数据迁移。OMS 支持将异构数据库在线不停服地迁移至 OceanBase 数据库，同时在应用切换至 OceanBase 数据库后，将 OceanBase 数据库上所有的变更数据实时同步至切换前的源端数据库。您可以使用管理控制台创建数据传输项目、自定义任务配置和监控运维传输组件。OMS 的核心功能如下：

  • 支持数据库对象迁移：包括表、约束、索引的自动导出、转换和创建。
  • 支持全量数据迁移。
  • 支持增量数据迁移：通过日志解析将源端实时的增量数据迁移至目标端。
  • 支持全量数据校验：源端和目标端表数据全字段的数据比对，对不一致的数据提供订正 SQL 语句。
  • 支持切换后反向增量迁移：业务将应用切换至(源)目标端数据源时，将目标端增量数据实时同步回源端。
  • 支持的 MySQL 版本包括：5.6、5.7。
  • 支持用户自定义告警设置。
  • 支持运维监控：包括主机监控、组件监控和组件运维。

• 数据导入导出工具（OBLOADER/OBDUMPER）：您可以使用 OBLOADER 和 OBDUMPER 快速导入和导出数据。您可以根据内核的负载能力指定并发数、分页查询的记录条数及分批处理数来提升导入和导出的并发性能。OBLOADER 和 OBDUMPER 支持按照分区、指定条件、单表、多表、全库、以及自定义查询 SQL 导出，支持数据预处理、字段映射导入，兼容 SQL，CSV、 CUT 等多种文本格式。

• 支持在线跨版本轮转升级：您可以通过 obd cluster upgrade 命令升级 OceanBase 数据库。OBD 自动识别匹配版本间的升级路径，对每个 Zone 进行轮转升级。更多信息，参考 OBD 使用指导。

• 数据热点行优化：在热点数据更新场景中，OceanBase 通过对 lock for write 加锁逻辑、填充日志前释放行级锁优化，使得 OceanBase 在处理热点数据的并发更新性能从 3000TPS 提升到 9000TPS，提升了热点数据处理能力上限。

• 优化了分区表 SQL PLAN 的淘汰机制：在分布式场景计划评估中，无法准确获得远端节点分区的行数反馈，存在计划退化的风险。OceanBase 采用最终 SQL 执行时间反馈负向优化执行计划，当采样的查询平均执行时间超过首次执行时间的 2 倍时，则针对性淘汰该 PLAN 并重新生成新的 SQL PLAN，解决因 SQL PLAN 劣化导致的性能下降问题。

• 通信时钟稳定性优化：OceanBase 底层通信框架使用 CPU 时钟（SYS_clock_gettime）获取超时时间，而上层 SQL 业务逻辑是使用操作系统时钟（get_timeofday）记录超时时间的，这两种时间精度上的误差会有极低概率导致上层业务逻辑重置底层的原始错误码，OceanBase 通过设计可容忍超时误差计算逻辑，准确还原出系统运行报错的第一现场，提升问题分析诊断效率。

• MySQL兼容：新增支持系统函数：

  • VALIDATE_PASSWORD_STRENGTH()：该函数用来给密码的安全性进行评分，返回 0-100。如 SELECT VALIDATE_PASSWORD_STRENGTH('Password@123');。
  • UUID_SHORT()：该函数可以生成一个 64 位无符号整数。如 SELECTUUID_SHORT();。
  • TO_BASE64() 和 FROM_BASE64()：该函数提供基于 base-64 算法的字符编码与解码能力。如 SELECT TO_BASE64('12345') 和 SELECT FROM_BASE64('MTIZNDU=');。
  • DAY()：该函数返回给定日期的月份的日期部分。如 SELECT DAY('2021-12-12');。
  • ANY_VALUE()：该函数用来抑制 ONLY_FULL_GROUP_BY 的值被拒绝。

• 开源 OBKV TABLE JAVA Client：提供通过 TABLE API 接口协议直接访问 OceanBase 数据库底层存储表数据的操作方式，用户开发程序可以以 JAR 包的形式调用该接口，支持认证、对行增删改查追加等操作。详见GitHub代码仓库(https://github.com/oceanbase/obkv-table-client-java)

• 开源 OBKV HBASE API：基于 TABLE API 提供的基本接口协议，在数据模型上增加宽表模型（WideColumn），优化 filter 逻辑，支持 KEY 分区、RANGE 分区和前缀分区，支持 TTL。兼容 HBase API 客户端持续开源中。

• 新增配置支持统信 UOS 平台操作系统：observer 与 obproxy 全面兼容主流 X86 平台操作系统。兼容列表详见：https://github.com/oceanbase/oceanbase

• 增强管理员工具 ob_admin：支持解析和显式输出 block_file 文件、归档文件、备份元信息和数据。命令输入形式为ob_admin dumpsst/archive_tool/dump_backup，详细参考帮助命令。

• 完善 OceanBase 社区版本文档。文档包括架构设计原理、性能调优、故障处理、日常维护指导等。

问题修复：

• 修复 __all_virtual_memory_info 虚拟表因 MEMOEY APPEND 失败导致显示租户信息不全的问题。
• 修复因 kill observer 实例导致创建 GLOBAL INDEX 失败的问题。该场景下系统自动发起DDL操作重试保证索引创建成功。
• 修复写入超长列值触发的 SCHEMA 刷新的防御性报错。
• 修复在 CentOS 8.3 和 7.9 平台上的编译错误。更多信息，参考 Issue 714 和 Issue 715。