• We now have proper support for out-of-tree builds, and this is now the preferred method for building Netdata.
• Build configuration is now measurably faster than it was previously.
• Netdata can now be built using Ninja instead of Make, further speeding up the build process. The installer and updater script will automatically use Ninja instead of Make when possible, and developers, package maintainers, and users who are building by hand are encouraged to explicitly use it themselves when building Netdata by specifying the -G Ninja option to CMake during the configuration process.
• Overall maintenance of the build system will be significantly easier going forwards. This means we should be able to fix any issues involving it much more quickly, and contributions from external developers should be much easier.
• A number of features we have wanted to add to our build infrastructure will be much easier to add now.

Most users should not be directly affected by this change other than benefiting from the faster build times, only those who were building locally by hand (not using the netdata-installer.sh script or the kickstart script) will need to change things.

We have made this change for three reasons:

• It makes handling of bugs in the Go plugin much easier. Instead of possibly needing to track issues and PRs across two repositories, now everything should end up tracked coherently in one repository.
• It lets us significantly simplify a number of parts of our CI and installation code, allowing for greater reliability and easier maintenance.
• It provides a testbed for infrastructure for handling of Go in the main repo, which is significant as we have been internally looking at reimplementing some other components of the agent in Go.

Users of native packages and static builds should see no difference at all from this change.

Building the agent locally will now require a working Go toolchain supporting a particular minimum version of the Go language (currently 1.21) if the Go plugin needs to be built. The plugin itself can still be disabled to avoid this requirement, but this is not recommended.

The installer code will attempt to ensure that a sufficiently up-to-date Go toolchain is installed when installing or updating the agent. If such a toolchain is not found, it will attempt to automatically install a copy of the official toolchain from https://go.dev/dl/ in /usr/local/go. If that attempt fails, the Go plugin will be DISABLED automatically at build time.

• Add macOS support for collecting resource usage of processes (apps.plugin) (#17180, @ktsaou)
• Improve identification of applications in docker service discovery (go.d.plugin) (#17174, @ilyam8)
• Execute local-listeners periodically rather than just once at startup (go.d.plugin) (#17160, @ilyam8)
• Add service discovery for applications running inside Docker containers (go.d.plugin) (#17152, @ilyam8)
• Implement dynamic configuration for configuring data collection jobs (go.d.plugin) (#17064, @ilyam8)
• Update message IDs for systemd and dbus (systemd-journal.plugin) (#16987, @ktsaou)
• Report EDAC ECC errors s total counts since boot instead of rates (proc/sys_devices_system_edac_mc) (#16970, @ilyam8)
• Add aggregated view (network-viewer.plugin) (#16940, @ktsaou)
• Add filtering by username (network-viewer.plugin) (#16911, @ktsaou)
• Add Network Viewer plugin (#16872, @ktsaou)
• Add CPU throttling % column to the containers-vms function (cgroups.plugin) (#16800, @ilyam8)
• Add the ndsudo binary, a helper tool for assisting in the execution of privileged commands (#16614, @ktsaou)
• Disable CPU per core metrics by default (proc.plugin) (#16572, @ilyam8)

• Fix incorrect family value of the ZFS ZPool state chart (proc/proc_spl_kstat_zfs) (#17054, @ilyam8)
• Fix race conditions (diskspace.plugin) (#16786, @ktsaou)
• Fix allocated memory after it has been freed (diskspace.plugin) (#16784, @ktsaou)
• Fix priority per-core CPU charts (proc/proc_stat) (#16749, @ilyam8)
• Fix missing CPU frequency chart (proc/proc_stat) (#16732, @ilyam8)
• Fix an issue where cgroup_check_for_new_every was incorrectly multiplied by update_every (cgroups.plugin) (#16719, @ilyam8)

• Add mongodb-community-server image to docker service discovery configuration (go.d.plugin) (#17173, @ilyam8)
• Add an option to disable service discovery (go.d.plugin) (#17171, @ilyam8)
• Allow array/object to be null json schemas (go.d.plugin) (#17166, @ilyam8)
• Update file path pattern in jsonschema (go.d.plugin) (#17164, @ilyam8)
• Add support for multi-config templates in the service discovery configuration (go.d.plugin) (#17157, @ilyam8)
• Improve go.d.plugin dyncfg config schemas (#17124, @ilyam8)
• Fix incorrect chart priority for discovered configs (go.d.plugin) (#17115, @ilyam8)
• Add notice log level (go.d.plugin) (#17112, @ilyam8)
• Fix pulsar tests (go.d/pulsar) (#17093, @ilyam8)
• Set max chart id length to 1200 (go.d.plugin) (#17062, @ilyam8)
• Improve aggregated view (network-viewer.plugin) (#16960, @ktsaou)
• Show unknown container (network-viewer.plugin) (#16900, @ktsaou)
• Reorganise code to prepare for functions (ebpf.plugin) (#16788, @thiagoftsm)
• Fix missing aral_freez call (eBPF) (#16765, @thiagoftsm)
• Cleanup network devices rename (proc/proc_net_dev) (#16745, @ktsaou)
• Improve ebpf-socket function column names (ebpf.plugin) (#16727, @ilyam8)
• Add double-linked network interfaces collection delay (#16701, @ilyam8)
• Cleanup code and improve reliability (ebpf.plugin) (#16669, @thiagoftsm)
• Update to create a separate chart for each systemd service rather than a chart dimension (ebpf.plugin) (#16630, @thiagoftsm)
• Include 'lxcfs.service/.control' in the list of filtered cgroups (cgroups.plugin) (#16620, @ilyam8)
• Exit if unable to locate journal data directories (systemd-journal.plugin) (#16592, @ilyam8)

• Remove deprecated alert fields from stock alarms (#17113, @ilyam8)
• Fix filtering by severity for gotify notifications (#17069, @ilyam8)
• Remove deprecated alert fields: "charts", "os", "host", "plugin" and "module" (#17048, @ktsaou)
• Add a new alert to notify about systemd timer units that have failed (#16845, @tkatsoulas)
• Implement dynamically configured alerts (#16779, @ktsaou)
• Add a new alert to detect unexpected HTTP headers (#16736, @ilyam8)

• Fix installing incorrect systemd service files in native deb installations (#17159, @tkatsoulas)
• Add a macOS build check in the CI pipeline (#17139, @tkatsoulas)
• Remove the "nut" package suggestion in native deb installations (#17129, @ilyam8)
• Fix incorrect ownership of cups.plugin in native deb installations (#17087, @tkatsoulas)
• Fix incorrect ownership of ioping.plugin in native deb installations (#17086, @tkatsoulas)
• Fix an issue where the ebpf.plugin was recommended for unsupported architectures (#17085, @tkatsoulas)
• Fix pre/post install script names for the network-viewer plugin in native deb installations (#17084, @tkatsoulas)
• Improve message in kickstart if a static build can’t be found (#17081, @Ferroin)
• Fix determining repo root in Coverity scan script (#17024, @Ferroin)
• More concretely utilize local modules in our CMake code (#17022, @Ferroin)
• Update eBPF packages (#17012, @thiagoftsm)
• Integrate Go plugin with build system (#17005, @Ferroin)
• Bump the version of the installed Go toolchain to 1.22.0 (#17004, @Ferroin)
• Include Go plugin sources in main repository (#16997, @Ferroin)
• Move CO-RE headers (integration between eBPF and Network Viewer) (#16978, @thiagoftsm)
• Use C++14 by default when building on systems that support it (#16972, @Ferroin)
• Split network viewer plugin to its own package (#16949, @Ferroin)
• Fix "Fluent-Bit" installation (#16924, @ilyam8)
• Build network-viewer only on Linux (#16910, @vkalintiris)
• Set build type to release with debug information, ensuring optimized builds (#16889, @vkalintiris)
• Add ARMv6 static builds (#16853, @Ferroin)
• Fix issue with fetching the latest tag on macOS in kickstart.sh (#16844, @ilyam8)
• Make the kickstart checksum placeholder value more specific (#16843, @tkatsoulas)
• Fix directory handling in Go toolchain handling script (#16828, @Ferroin)
• Add script to ensure a usable Go toolchain is installed (#16815, @Ferroin)
• Apply ASCII-based comparisons to commands in kickstart script that rely on a particular language setting (#16806, @tkatsoulas)
• Remove help text that no longer applies from netdata-installer.sh (#16805, @vkalintiris)
• Fix incorrect major version check in updater (#16803, @Ferroin)
• Change default build directory in installer to build (#16768, @Ferroin)
• Add cap_dac_read_search capability to go.d.plugin (#16754, @ilyam8)
• Improve removal of the netdata user from groups in uninstaller.sh (#16742, @ilyam8)
• Update the default netdata.conf configuration file used for native packages (#16734, @ilyam8)
• Fix handling of hardening flags with Clang (#16731, @Ferroin)
• Disable logs-management plugin when installing on macOS (#16708, @ilyam8)
• Remove Ubuntu 23.04 from the CI (#16694, @tkatsoulas)
• Fix enable/disable options for Prometheus remote write in netdata-installer.sh (#16690, @tkatsoulas)
• Disable logs-management plugin when building static packages (#16684, @ilyam8)
• Fix an issue where install-required-packages.sh wouldn't run due to elevated privileges on macOS (#16675, @ilyam8)
• Remove unused contrib/rhel directory (#16672, @ilyam8)
• Update eBPF packages (#16671, @thiagoftsm)
• Add extra build flags to CMakeLists.txt (#16641, @Ferroin)
• Assorted cleanup of native packaging code (#16640, @Ferroin)
• Make web directory configurable through CMake variables (#16638, @ilyam8)
• Improve handling of basic permissions for most scripts on install (#16629, @Ferroin)
• Improve robustness of netdata-updater.sh (#16613, @candlerb)
• Remove v1 dashboard version check from installer.sh (#16603, @ilyam8)
• Fix "target_ram" calculation in netdata-installer.sh (#16602, @ilyam8)
• Improve enable_feature function in installer.sh (#16601, @ilyam8)
• Allow passing cmake options with NETDATA_CMAKE_OPTIONS (#16598, @vkalintiris)
• Add Alpine Linux 3.19 to CI (#16579, @Ferroin)
• Remove Netdata packages from APT cache when attempting to install (#16566, @Ferroin)
• Assorted kickstart script fixes (#16537, @Ferroin)
• Remove openSUSE 15.4 from CI (#16449, @tkatsoulas)
• Remove fedora 37 from CI (#16422, @tkatsoulas)
• Add CMake build system (#15996, @vkalintiris)
• Add check to avoid auto-installing new major versions of Netdata (#15898, @Ferroin)
• Improve support running the Docker entrypoint code as a non-root user (#15118, @Ferroin)

• Improve "Choose your Netdata Cloud theme" doc (#17172, @Ancairon)
• Add instructions for monitoring NVIDIA GPUs to the Docker installation guide (#17167, @ilyam8)
• Add documentation for the "Integration URL" field to PagerDuty Cloud integration doc (#17149, @juacker)
• Bring back old docs that were containing missing information (#17146, @Ancairon)
• Remove unnecessary repetition of words in docs (#17131, @carrychair)
• Fix broken link in "Netdata Cloud On-Prem Installation" (#17118, @tkatsoulas)
• Fix typos and improve wording in "Backing up a Netdata Agent" (#17117, @Ancairon)
• Remove deprecated settings from "Configure alerts" (#17116, @ilyam8)
• Fix broken links in go.d.plugin markdown files (#17108, @ilyam8)
• Remove deprecated "foreach" from "Configure alerts" (#17106, @ilyam8)
• Remove distributed-data-architecture.md (#17097, @Ancairon)
• Fix broken links (#17095, @Ancairon)
• Remove docs/netdata-security.md (#17094, @Ancairon)
• Update "Plugin Functions Tables" docs (#17071, @car12o)
• Update "Sizing Netdata Agents" doc (#17057, @ktsaou)
• Fix links pointing to old go.d repo and update the integrations (#17040, @Ancairon)
• Update links to Netdata Agent start-stop-restart docs (#17037, @Ancairon)
• Include information on securing Netdata parent-child communication in "Configuring Metrics Centralization Points" (#17035, @Ancairon)
• Restructure and update documentation (#17014, @Ancairon)
• Add "Backing up a Netdata Agent" documentation (#17006, @luisj1983)
• Correct instructions on creating a startup script in the "Install Netdata on Synology" guide (#16980, @pschaer)
• Improve formatting in "How to optimize the Netdata Agent's performance" (#16925, @tkatsoulas)
• Fix links to the energy efficiency screenshots to main readme file (#16904, @Aliki92)
• Update "What's New and Coming?"based on Office Hours shared plans to main readme file (#16895, @hugovalente-pm)
• Improve readability of Webhook Cloud notification documentation (#16882, @juacker)
• Remove deprecated db mode "save" from "Database" (#16864, @Ancairon)
• Fix CNCF link (#16851, @hugovalente-pm)
• Add documentation on how to configure MS Teams Cloud notifications (#16834, @papazach)
• Added instructions on calculating replication history to "Streaming and Replication Reference" (#16816, @thiagoftsm)
• Update provisioning instructions in "Netdata Cloud On-Prem Light PoC" (#16811, @M4itee)
• Add information about the new node permissions to "Role-Based Access model" (#16791, @vkuznecovas)
• Add missing settings to "Streaming and replication reference" (#16778, @thiagoftsm)
• Fix instructions for setting up Telegram notifications (#16777, @thiagoftsm)
• Updated the kickstart URL to https://get.netdata.cloud/kickstart.sh (#16738, @ilyam8)
• Fix --distro-override parameter name in "Install Netdata with kickstart.sh" (#16726, @moschlar)
• Add the Mobile App notification Integration (#16715, @sashwathn)
• Add "Require Cloud" column to the functions table in "Netdata Functions" (#16681, @ilyam8)
• Fix typos and improve wording in "Creating Alerts with Netdata Alerts Configuration Manager" (#16679, @Ancairon)
• Fix description of "chart labels" in "Configure alerts" (#16656, @sepek)
• Fix formatting in "Creating Alerts with Netdata Alerts Configuration Manager" (#16651, @Ancairon)
• Add practical examples showcasing how to utilize journalctl for querying Netdata logs to "Netdata Logging" (#16650, @ilyam8)
• Add "Creating Alerts with Netdata Alerts Configuration Manager" (#16642, @sashwathn)
• Add instructions for installing Netdata in a rootless Docker environment (#16632, @ilyam8)
• Add energy efficiency image to main readme file (#16617, @Aliki92)
• Remove deprecated memory mode "map" and "save" (#16604, @vkalintiris)
• Update Splunk icon to a dark version for improved visibility (#16593, @juacker)
• Add documentation on how to configure Splunk Cloud notifications (#16586, @juacker)
• Add a new document explaining Gorilla compression and decompression techniques (#16553, @vkalintiris)
• Add an initial version of the "Plugin Functions Tables" documentation (#16535, @ktsaou)

• Change query label matching logic (#16827, @stelfrag)
• Setup sentry-native SDK for reporting crashes (#16798, @vkalintiris)
• Add netdata_os_info metric to Prometheus export (#16756, @colinleroy)
• Rewrite and extend dynamic configuration (#16702, @ktsaou)
• Track the progress of data queries (#16574, @ktsaou)

• Fix a crash occurring when failing to create the requested number of tiers (#16999, @stelfrag)
• Fix an issue where Netdata plugins could inherit unintended sockets or file descriptors during the forking process (#16881, @ktsaou)

• Announce dynamic configuration capability to the Cloud (#17162, @stelfrag)
• Fix a problem preventing the Agent from starting due to missing SOCK_CLOEXEC on macOS (#17151, @stelfrag)
• Add a check to ensure that the duration is not negative when sending alert log to the Cloud (#17144, @stelfrag)
• Add a check to detect self thread when exiting (#17126, @vkalintiris)
• Fix health alert dyncfg schema fullPage option (#17125, @ilyam8)
• Fix memory leak when freeing an array pattern (#17114, @stelfrag)
• Remove unused go.d.plugin files (#17110, @ilyam8)
• Improve cleanup of ephemeral hosts during agent startup (#17104, @stelfrag)
• Reorganize and cleanup database related code (#17101, @stelfrag)
• Fix ebpf compilation warnings (#17100, @stelfrag)
• Make watcher thread wait for explicit steps (#17079, @vkalintiris)
• Abort the agent if a single shutdown step takes more than 60 seconds (#17060, @vkalintiris)
• Fix a typo in the ENABLE_SENTRY cmake variable (#17059, @vkalintiris)
• Call the dyncfg interceptor when executing 'test' for a new job (#17052, @ktsaou)
• Fix alerts jsonschema prototype for latest dyncfg (#17047, @ktsaou)
• Protect type anomaly rate map with a spinlock (#17044, @vkalintiris)
• Do not use backtrace when sentry is enabled (#17043, @vkalintiris)
• Add metric and sample count into the api/v2/info response (#17042, @stelfrag)
• Improved query target cleanup (#17038, @stelfrag)
• Database and health code cleanup (#17036, @stelfrag)
• Do not fetch retention on metric release (#17033, @stelfrag)
• Increase RRD_ID_LENGTH_MAX to 1200 (#17028, @stelfrag)
• Add support for deleting orphan configurations to dyncfg (#17023, @ktsaou)
• Correctly mark protobuf as required in find_package (#17021, @Ferroin)
• Protect metric release in dimension delete callback (#17020, @stelfrag)
• Reorganize ebpf plugin code for network-viewer (#17018, @thiagoftsm)
• Allow tree for individual IDs (#17017, @ktsaou)
• Add watcher thread to report shutdown steps (#17010, @vkalintiris)
• Fix testing new jobs in dyncfg (#17009, @ktsaou)
• Abort on non-zero rc during exiting on sentry-enabled builds (#17008, @vkalintiris)
• Misc improvements (#17001, @stelfrag)
• Move diagrams/ under docs/ (#16998, @vkalintiris)
• Small code cleanup (#16996, @vkalintiris)
• Remove historical changelog and cppcheck (#16995, @vkalintiris)
• Remove config macros that are always set (#16994, @vkalintiris)
• Move web/ under src/ (#16992, @vkalintiris)
• Add spinlock to protect metric release (#16989, @stelfrag)
• Detect machine GUID change (#16979, @stelfrag)
• Move collectors/ under src/ (#16965, @vkalintiris)
• Improve agent shutdown (#16959, @stelfrag)
• Add support for testing new jobs to dyncfg (#16958, @ktsaou)
• Fix path in health integrations (#16956, @Ancairon)
• Move health/ under src/ (#16954, @vkalintiris)
• Do not declare struct meant for internal usage (#16951, @vkalintiris)
• Remove cleanup_destroyed_dictionaries call during shutdown (#16944, @stelfrag)
• Remove duplicate check (#16936, @stelfrag)
• Move daemon/ under src/ (#16933, @vkalintiris)
• Split dictionary into multiple files (#16920, @ktsaou)
• Release label key if already in use (#16916, @stelfrag)
• Add support for the info parameter to all external plugin functions (#16915, @ktsaou)
• Move exporting/ under src/ (#16913, @vkalintiris)
• Rename network functions (#16908, @ktsaou)
• Assorted build-related changes (#16906, @vkalintiris)
• Move fluent-bit & logsmanagement under src/ (#16903, @vkalintiris)
• updated permissions map comment (#16902, @ktsaou)
• Use spinlock for reference counting (#16901, @vkalintiris)
• Move aclk/ under src/ (#16899, @vkalintiris)
• Enable sentry sessions (#16898, @vkalintiris)
• Do not cancel detection thread (#16897, @vkalintiris)
• Create a top-level directory for the source code (#16896, @vkalintiris)
• Remove tags field from RRD hosts (#16894, @vkalintiris)
• Add support for using netlink when libmnl is available to local-sockets (#16893, @ktsaou)
• Fix order of opening a file and checking its inode in local-sockets (#16887, @ktsaou)
• Fix crash on query_progress initializer (#16885, @ktsaou)
• Remove references to map and save modes in stream.conf (#16874, @vkalintiris)
• Fix coverity issues (#16873, @stelfrag)
• Add support for network namespaces to local-listeners (#16867, @ktsaou)
• Fix coverity issue (#16866, @stelfrag)
• Fix an issue where alerts were applied based on order instead of the matching chart criteria (#16862, @ktsaou)
• Add support for sockets direction to local-listeners (#16861, @ktsaou)
• Improve service thread shutdown (#16841, @stelfrag)
• New Permissions System (#16837, @ktsaou)
• Add brotli and libyaml to buildinfo (#16830, @ktsaou)
• Add explicit callback types for readability (#16820, @vkalintiris)
• Improve the robustness of host prefix verification (#16813, @ilyam8)
• Move mqtt_websockets under aclk/ (#16804, @vkalintiris)
• Add additional fail reason and source during database initialization (#16794, @stelfrag)
• Use original summary for alert transition (#16793, @stelfrag)
• Free key and search, replace patterns (#16789, @stelfrag)
• Use named constants for keyword tokens (#16787, @vkalintiris)
• Remove h2o header from libnetdata (#16780, @vkalintiris)
• Delete unused variable (#16776, @vkalintiris)
• Use unsigned char for binary data in mqtt (#16775, @vkalintiris)
• Fix compiler warnings (#16774, @vkalintiris)
• Allow POST requests to be received from ACLK (#16770, @ktsaou)
• Keep transaction id of request headers (#16769, @ktsaou)
• Improvements for /api/v1/config tree and swagger documentation (#16764, @ktsaou)
• Fix compiler warnings (#16763, @ktsaou)
• Fix cmake _GNU_SOURCE warnings (#16761, @ktsaou)
• Fix sanitizer errors (#16759, @ktsaou)
• Report timestamps with progress (#16758, @ktsaou)
• Add schemas to /usr/lib/netdata/conf.d/schema.d (#16757, @ktsaou)
• Recursively merge mqtt_websockets (#16755, @vkalintiris)
• Name storage engine variables consistently (#16753, @vkalintiris)
• Address sanitizer through CMake and use it for unit tests (#16748, @vkalintiris)
• Improve the error message when accessing functions (#16692, @ktsaou)
• Fix linking issues for log2journal and netdatacli against libnetdata (#16688, @ktsaou)
• Fatal relaxation of unknown page types (#16682, @vkalintiris)
• Fix cmake missing defines (#16680, @ktsaou)
• Set log level of the too-old-data message to debug (#16663, @ilyam8)
• Improve context load (#16659, @stelfrag)
• Fix compilation error when using --disable-dbengine (#16645, @stelfrag)
• Remove code relying on autotools (#16634, @vkalintiris)
• Fix UB of unaligned loads/stores and signed shifts (#16628, @vkalintiris)
• Fix coverity issues, logically dead code and error checking (#16618, @stelfrag)
• Fix small coverity issue (#16616, @stelfrag)
• Remove CPack stuff from CMake (#16608, @vkalintiris)
• Remove includes outside libnetdata (#16607, @vkalintiris)
• Remove build/ (#16600, @vkalintiris)
• Cleanup am files (#16597, @vkalintiris)
• Handle coverity issues related to Y2K38_SAFETY (#16583, @stelfrag)
• Update naming for swagger api (#16564, @tkatsoulas)
• Code cleanup (#16542, @ktsaou)