Enhancements

• #10077 Add support for QUIC TLS password protected certificate file.

• #10128 Add support for OCSP stapling for SSL MQTT listeners.

• #10156 Change the priority of the configuration:

  1. If it is a new installation of EMQX, the priority of configuration is ENV > emqx.conf > HTTP API.
  2. If EMQX is upgraded from an old version (i.e., the cluster-override.conf file still exists in EMQX's data directory), then the configuration priority remains the same as before. That is, HTTP API > ENV > emqx.conf.

  Deprecated data/configs/local-override.conf.

  Stabilizing the HTTP API for hot updates.

• #10164 Add CRL check support for TLS MQTT listeners.

• #10206 Decouple the query mode from the underlying call mode for buffer workers.

  Prior to this change, setting the query mode of a resource such as a bridge to sync would force the buffer to call the underlying connector in a synchronous way, even if it supports async calls.

• #10207 Use 'label' from i18n file as 'summary' in OpenAPI spec.

• #10210 Unregister Mnesia post commit hook when Mria is being stopped. This fixes hook failures occasionally occurring on stopping/restarting Mria.

  Mria PR

• #10224 Add the option to customize clusterIP in Helm chart, so that a user may set it to a fixed IP.

• #10263 Add command 'eval-ex' for Elixir expression evaluation.

• #10278 Refactor the directory structure of all gateways.

• #10306 Add support for async query mode for most bridges.

  Before this change, some bridges (Cassandra, MongoDB, MySQL, Postgres, Redis, RocketMQ, TDengine) were only allowed to be created with a sync query mode.

• #10318 Now, the rule engine language's FROM clause supports both strings enclosed in double quotes (") and single quotes (').

• #10336 Add /rule_engine API endpoint to manage configuration of rule engine.

• #10354 More specific error messages when configure with bad max_heap_size value. Log current value and the max value when the message_queue_too_long error is thrown.

• #10358 Hide flapping_detect/conn_congestion/stats configuration. Deprecate flapping_detect.enable.

• #10359 Metrics now are not implicitly collected in places where API handlers don't make any use of them. Instead, a separate backplane RPC gathers cluster-wide metrics.

• #10373 Deprecate the trace.payload_encode configuration. Add payload_encode=[text,hidden,hex] option when creating a trace via HTTP API.

• #10381 Hide the auto_subscribe configuration items so that they can be modified later only through the HTTP API.

• #10385 Hide data items(rule_engine/bridge/authz/authn) from configuration files and documentation.

• #10391 hide exhook/rewrite/topic_metric/persistent_session_store/overload_protection from the docs and configuration file.

• #10404 Change the default queue mode for buffer workers to memory_only. Before this change, the default queue mode was volatile_offload. When under high message rate pressure and when the resource is not keeping up with such rate, the buffer performance degraded a lot due to the constant disk operations.

• #10140 Integrate Cassandra into bridges as a new backend. At the current stage:

  • Only support Cassandra version 3.x, not yet 4.x.

• #10143 Add RocketMQ data integration bridge.

• #10165 Support escaped special characters in InfluxDB data bridge write_syntax. This update allows to use escaped special characters in string elements in accordance with InfluxDB line protocol.

• #10294 When configuring a MongoDB bridge, you can now use the ${var} syntax to reference fields in the message payload within the collection field. This enables you to select the collection to insert data into dynamically.

• #10337 Add schema registry feature.

  With schema registry, one can encode and decode special serialization formats in payloads when transforming messages in Rule Engine. Currently, only Apache Avro is supported.

• #10363 Implement Microsoft SQL Server bridge.

Bug Fixes

• #10145 Fix bridges API to report error conditions for a failing bridge as status_reason. Also when creating an alarm for a failing resource we include this error condition with the alarm's message.

• #10172 Fix the incorrect default ACL rule, which was:

  {allow, {username, "^dashboard?"}, subscribe, ["$SYS/#"]}.
  

  However, it should use {re, "^dashboard$"} to perform a regular expression match:

  {allow, {username, {re,"^dashboard$"}}, subscribe, ["$SYS/#"]}.
  

• #10174 Upgrade library esockd from 5.9.4 to 5.9.6. Fix an unnecessary error level logging when a connection is closed before proxy protocol header is sent by the proxy.

• #10195 Add labels to API schemas where description contains HTML and breaks formatting of generated documentation otherwise.

• #10196 Use lower-case for schema summaries and descritptions to be used in menu of generated online documentation.

• #10209 Fix bug where a last will testament (LWT) message could be published when kicking out a banned client.

• #10211 Hide broker.broker_perf config and API documents. The two configs route_lock_type and trie_compaction are rarely used and requires a full cluster restart to take effect. They are not suitable for being exposed to users. Detailed changes can be found here: https://gist.github.com/zmstone/01ad5754b9beaeaf3f5b86d14d49a0b7/revisions

• #10225 Allow installing a plugin if its name matches the beginning of another (already installed) plugin name. For example: if plugin "emqx_plugin_template_a" is installed, it must not block installing plugin "emqx_plugin_template".

• #10226 Don't crash on validation error in /bridges API, return 400 instead.

• #10242 Fixed a log data field name clash. Piror to this fix, some debug logs may report a wrong Erlang PID which may affect troubleshooting session takeover issues.

• #10257 Fixed the issue where auto_observe was not working in LwM2M Gateway.

  Before the fix, OBSERVE requests were sent without a token, causing failures that LwM2M clients could not handle.

  After the fix, LwM2M Gateway can correctly observe the resource list carried by client, furthermore, unknown resources will be ignored and printing the following warning log:

  2023-03-28T18:50:27.771123+08:00 [warning] msg: ignore_observer_resource, mfa: emqx_lwm2m_session:observe_object_list/3, line: 522, peername: 127.0.0.1:56830, clientid: testlwm2mclient, object_id: 31024, reason: no_xml_definition
  

• #10286 Enhance logging behaviour during boot failure. When EMQX fails to start due to corrupted configuration files, excessive logging is eliminated and no crash dump file is generated.

• #10297 Keeps eval command backward compatible with v4 by evaluating only Erlang expressions, even on Elixir node. For Elixir expressions, use eval-ex command.

• #10300 Fixed an issue where a build made with Elixir could not receive uploaded plugins until the plugins folder was created manually to receive the uploaded files.

• #10315 Fix crash checking limit and page parameters in /mqtt/delayed/messages API call.

• #10317 Do not expose listener level authentications before extensive verification.

• #10323 For security reasons, the value of the password field in the API examples is replaced with ******.

• #10410 Fix config check failed when gateways are configured in emqx.conf. This issue was first introduced in v5.0.22 via #10278, the boot-time config check was missing.

• #10201 In TDengine, removed the redundant database name from the SQL template.

• #10270 Clickhouse has got a fix that makes the error message better when users click the test button in the settings dialog.

• #10324 Previously, when attempting to reconnect to a misconfigured Clickhouse bridge through the dashboard, users would not receive an error message. This issue is now resolved, and error messages will now be displayed