Changes

v1.83.0

Released at 29-10-2022

Update note 1: the indexdb/tagFilters cache type at /metrics has been renamed to indexdb/tagFiltersToMetricIDs in order to make its puropose more clear.

Update note 2: vmalert: the crlfEscape template function becames obsolete starting from this release. It can be safely removed from alerting templates, since \n chars are properly escaped with other *Escape functions now. See this and this issue for details.

• FEATURE: VictoriaMetrics enterprise: add support for automatic vmstorage nodes discovering and updating at vmselect and vminsert. See these docs.

• FEATURE: VictoriaMetrics enterprise: allow configuring multiple retentions for distinct sets of time series. See these docs, this and this feature request.

• FEATURE: VictoriaMetric cluster enterprise: add support for multiple retentions for distinct tenants - see these docs and this and this feature request.

• FEATURE: allow limiting memory usage on a per-query basis with -search.maxMemoryPerQuery command-line flag. See this feature request.

• FEATURE: allow referring environment variables inside command-line flags via %{ENV_VAR} syntax. For example, if AUTH_KEY=top-secret environment variable is set, then -metricsAuthKey=%{AUTH_KEY} command-line flag is automatically expanded to -storageDataPath=top-secret at VictoriaMetrics startup. See these docs for details.

• FEATURE: allow referring environment variables inside other environment variables via %{ENV_VAR} syntax. For example, if A=a-%{B}, B=b-%{C} and C=c env vars are set, then VictoriaMetrics components automatically expand them to A=a-b-c, B=b-c and C=c on startup.

• FEATURE: vmagent: drop all the labels with __ prefix from discovered targets in the same way as Prometheus does according to this article. Previously the following labels were available during metric-level relabeling: __address__, __scheme__, __metrics_path__, __scrape_interval__, __scrape_timeout__, __param_*. Now these labels are available only during target-level relabeling. This should reduce CPU usage and memory usage for vmagent setups, which scrape big number of targets.

• FEATURE: vmagent: improve the performance for metric-level relabeling, which can be applied via metric_relabel_configs section at scrape_configs, via -remoteWrite.relabelConfig or via -remoteWrite.urlRelabelConfig command-line options.

• FEATURE: vmagent: allow specifying full url in scrape target addresses (aka __address__ label). This makes valid the following -promscrape.config:

  scrape_configs:
  - job_name: abc
    metrics_path: /foo/bar
    scheme: https
    static_configs:
    - targets:
      # the following targets are scraped by the provided full urls
      - 'http://host1/metric/path1'
      - 'https://host2/metric/path2'
      - 'http://host3:1234/metric/path3?arg1=value1'
      # the following target is scraped by <scheme>://host4:1234<metrics_path>
      - host4:1234
  

  See the corresponding issue.

• FEATURE: vmagent: allow controlling staleness tracking on a per-scrape_config basis by specifying no_stale_markers: true or no_stale_markers: false option in the corresponding scrape_config.

• FEATURE: vmalert: add strvalue and stripDomain template functions in order to improve compatibility with Prometheus.

• FEATURE: vmalert: add jsonEscape and htmlEscape template functions.

• FEATURE: vmui: limit the number of plotted series. This should prevent from browser crashes or hangs when the query returns big number of time series. See this feature request.

• FEATURE: vmui: reduce memory usage when querying big number of time series. See this issue.

• FEATURE: vmui: add responsive styles for small screens. See this issue and this pull request.

• FEATURE: log error if some environment variables referred at -promscrape.config via %{ENV_VAR} aren't found. This should prevent from silent using incorrect config files.

• FEATURE: immediately shut down VictoriaMetrics apps on the second SIGINT or SIGTERM signal if they couldn't be finished gracefully for some reason after receiving the first signal.

• FEATURE: improve the performance of /api/v1/series endpoint by eliminating loading of unused TSID data during the API call.

• FEATURE: vmbackupmanager: add functionality for automated restore from backup. See these docs.

• BUGFIX: MetricsQL: properly merge buckets with identical le values, but with different string representation of these values when calculating histogram_quantile and histogram_share. For example, http_request_duration_seconds_bucket{le="5"} and http_requests_duration_seconds_bucket{le="5.0"}. Such buckets may be returned from distinct targets. Thanks to @647-coder for the pull request.

• BUGFIX: vmalert: change severity level for log messages about failed attempts for sending data to remote storage from error to warn. The message for about all failed send attempts remains at error severity level.

• BUGFIX: vmalert: fix panic if vmalert runs with -clusterMode command-line flag in multitenant mode. The issue has been introduced in v1.82.0.

• BUGFIX: vmalert: properly escape string passed to quotesEscape template function, so it can be safely embedded into JSON string. This makes obsolete the crlfEscape function. See this and this issue.

• BUGFIX: vmagent: do not show invalid error message in Kubernetes service discovery: cannot parse WatchEvent json response: EOF. The invalid error message has been appeared in v1.82.0.

• BUGFIX: vmagent: properly add exported_ prefix to metric labels, which clashing with scrape target labels if honor_labels: true option isn't set in scrape_config. Previously some exported_ prefixes were missing in the resulting metric labels. See this issue. The issue has been introduced in v1.82.0.

• BUGFIX: vmselect: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"} . This metric is used for monitoring rollup cache usage with the query vm_cache_size_bytes{type="promql/rollupResult"} / vm_cache_size_max_bytes{type="promql/rollupResult"} in the same way as this is done for other cache types.

See full changelog.

How to run VictoriaMetrics

Unpack the victoria-metrics-*.tar.gz archive and read these docs.

vmutils-*.tag.gz archive contains the following tools:

• vmagent
• vmalert
• vmauth
• vmbackup
• vmctl
• vmrestore

vmutils-*-enterprise.tar.gz archive contains the following additional enterprise tools:

• vmbackupmanager
• vmgateway

The corresponding docker images are available here. Cluster version is available here.