e5.5.0

Enhancements

• #12085 EMQX has been upgraded to leverage the capabilities of OTP version 26.1.2-2. NOTE: Docker images are still built with OTP 25.3.2.

• #12189 Enhanced the ACL claim format in EMQX JWT authentication for greater versatility. The updated format now supports an array structure, aligning more closely with the file-based ACL rules.

  For example:

  [
  {
    "permission": "allow",
    "action": "pub",
    "topic": "${username}/#",
    "qos": [0, 1],
    "retain": true
  },
  {
    "permission": "allow",
    "action": "sub",
    "topic": "eq ${username}/#",
    "qos": [0, 1]
  },
  {
    "permission": "deny",
    "action": "all",
    "topics": ["#"]
  }
  ]
  

  In this new format, the absence of a matching rule does not result in an automatic denial of the action. The authorization chain can advance to other configured authorizers if a match is not found in the JWT ACL. If no match is found throughout the chain, the final decision defers to the default permission set in authorization.no_match.

• #12267 Added a new timeout parameter to the cluster/:node/invite interface, addressing the issue of default timeouts. The previously set 5-second default timeout often led to HTTP API call timeouts because joining an EMQX cluster usually requires more time.

  In addition, EMQX added a new API /cluster/:node/invite_async to support an asynchronous way to invite nodes to join the cluster and introduced a new cluster/invitation API to inspect the join status.

• #12272 Introduced updates to the retain API in EMQX:

  • Added a new API DELETE /retainer/messages to clean all retained messages.
  • Added an optional topic filter parameter topic in the query string for the API GET /retainer/messages. For example, using a query string topic=t/1 filters the retained messages for a specific topic, improving the efficiency of message retrieval.

• #12277 Added mqtt/delayed/messages/:topic API to remove delayed messages by topic name.

• #12278 Adjusted the maximum pagination size for paginated APIs in the REST API from 3000 to 10000.

• #12289 Authorization caching now supports the exclusion of specific topics. For the specified list of topics and topic filters, EMQX will not generate an authorization cache. The list can be set through the authorization.cache.excludes configuration item or via the Dashboard. For these specific topics, permission checks will always be conducted in real-time rather than relying on previous cache results, thus ensuring the timeliness of authorization outcomes.

• #12329 Added broker.routing.batch_sync configuration item to enable a dedicated process pool that synchronizes subscriptions with the global routing table in batches, thus reducing the frequency of cross-node communication that can be slowed down by network latency. Processing multiple subscription updates collectively, not only accelerates synchronization between replica nodes and core nodes in a cluster but also reduces the load on the broker pool, minimizing the risk of overloading.

• #12333 Added a tags field for actions and connectors. Similar to the description field (which is a free text annotation), tags can be used to annotate actions and connectors for filtering and grouping.

• #12072 GreptimeDB data integration now supports asynchronous data write operations to provide better performance.

• #12194 Improved Kafka producer performance.

• #12247 The bridges for InfluxDB have been split so they are available via the connectors and actions APIs. They are still backward compatible with the old bridge API.

• #12299 Exposed more metrics to improve observability:

  Montior API:

  • Added retained_msg_count field to /api/v5/monitor_current.
  • Added license_quota field to /api/v5/monitor_current
  • Added retained_msg_count and node_uptime fields to /api/v5/monitor_current/nodes/{node}.
  • Added retained_msg_count, license_quota and node_uptime fields to /api/v5/monitor_current/nodes/{node}.

  Prometheus API:

  • Added emqx_cert_expiry_at and emqx_license_expiry_at to /api/v5/prometheus/stats to display TLS listener certificate expiration time and license expiration time.
  • Added /api/v5/prometheus/auth endpoint to provide metrics such as execution count and running status for all authenticatiors and authorizators.
  • Added /api/v5/prometheus/data_integration endpoint to provide metrics such as execution count and status for all rules, actions, and connectors.

  Limitations: Prometheus push gateway only supports the content in /api/v5/prometheus/stats?mode=node.

  For more API details and metric type information, please see swagger api docs.

• #12196 Improved network efficiency during routes cleanup.

  Previously, when a node was down, a delete operation for each route to that node must be exchanged between all the other live nodes. After this change, only one match and delete operation is exchanged between all live nodes, significantly reducing the number of necessary network packets and decreasing the load on the inter-cluster network. This optimization must be especially helpful for geo-distributed EMQX deployments where network latency can be significantly high.

• #12354 The concurrent creation and updates of data integrations are now supported, significantly increasing operation speeds, such as when importing backup files.

• #12396 Enhanced the user import feature in the authentication/:id/import_users Interface:

  • Added a new parameter ?type=plain for easier importing of users with plaintext passwords, complementing the existing functionality that supports hashed passwords.
  • Enhanced support for content-type: application/json, allowing HTTP Body submissions in JSON format. This extends the current capability that exclusively supports multipart/form-data for CSV files.

• #11902 Enhanced EMQX's capability to facilitate MQTT message bridging through the one-way Nari SysKeeper 2000 network isolation gateway.

Bug Fixes

• #12232 Fixed an issue when cluster commit log table was not deleted after a node was forced to leave a cluster.

• #12243 Fixed a family of subtle race conditions that could lead to inconsistencies in the global routing state.

• #12269 Improved error handling in the /clients interface; now returns a 400 status with more detailed error messages, instead of a generic 500, for query string validation failures.

• #12285 Updated the CoAP gateway to support short parameter names for slight savings in datagram size. For example, clientid=bar can be written as c=bar.

• #12303 Fixed the message indexing in retainer. Previously, clients with wildcard subscriptions might receive irrelevant retained messages not matching their subscription topics.

• #12305 Corrected an issue with incomplete client/connection information being passed into emqx_cm, which could lead to internal inconsistencies and affect memory usage and operations like node evacuation.

• #12306 Fixed an issue preventing the connectivity test for the Connector from functioning correctly after updating the password parameter via the HTTP API.

• #12359 Fixed an issue causing error messages when restarting a node configured with some types of data bridges. Additionally, these bridges were at risk of entering a failed state upon node restart, requiring a manual restart to restore functionality.

• #12404 Fixed an issue where restarting a data integration with heavy message flow could lead to a stop in the collection of data integration metrics.

• #12282 Improved the HTTP API error response for MySQL bridge creation failures. It also resolved a problem with removing MySQL Sinks containing undefined columns in their SQL.

• #12291 Fixed inconsistencies in EMQX’s handling of configuration updates involving sensitive parameters, which previously led to stray "******" strings in cluster configuration files.

• #12301 Fixed an issue with the line protocol in InfluxDB, where numeric literals were being stored as string types.

• #12317 Removed the resource_opts.batch_size field from the MongoDB Action schema, as it is not yet supported.

Breaking Changes

• #12283 Fixed the resource_opts configuration schema for the GCP PubSub Producer connector so that it contains only relevant fields. This affects the creation of GCP PubSub Producer connectors via HOCON configuration (connectors.gcp_pubsub_producer.*.resource_opts) and the HTTP APIs POST /connectors / PUT /connectors/:id for this particular connector type.