This version represents a major revamp of the library, aiming to simplify and modernize it, resulting in the removal of a few features. Please read through the changes carefully before upgrading, as it is not backwards compatible with previous versions and some effort will be required to migrate.

I understand that these changes may inconvenience some existing users. However, they have been made with good intentions, aiming to improve and refine the logging library. This involved significant effort and dedication.

Bug fixes and releases for v3 will continue to be supported under the v3.x.x branch.

Comparison

• This version significantly improves compile times. Taking a look at some compiler profiling for a Release build with clang 15, we can see the difference. Below are the two compiler flamegraphs for building the recommended_usage example from the new version and the wrapper_lib example from the previous version.

The below flamegraph shows the difference in included headers between the two versions

Version	Compiler FlameGraph
v4.0.0
v3.8.0

A new compiler benchmark has been introduced. A Python script generates 2000 distinct log statements with various arguments. You can find the benchmark here. Compilation now takes only about 30 seconds, whereas the previous version required over 4 minutes.

Version	Compiler FlameGraph
v4.0.0
v3.8.0

• Minor increase in backend thread throughput compared to the previous version.

• Significant boost in hot path latency when logging complex types such as std::vector. The performance remains consistent when logging only primitive types or strings in both versions. Refer here for updated and detailed benchmarks.

Changes

• Improved compile times

The library has been restructured to minimize the number of required headers. Refactoring efforts have focused on decoupling the frontend from the backend, resulting in reduced dependencies. Accessing the frontend logging functions now does not demand inclusion of any backend logic components.

 "quill/Backend.h" - It can be included once to start the backend logging thread, typically in main.cpp 
                     or in a wrapper library.
 
 "quill/Frontend.h"` - Used to create or obtain a `Logger*` or a `Sink`. It can be included in limited 
                       files, since an obtained `Logger*` has pointer stability and can be passed around.
 
 "quill/Logger.h", "quill/LogMacros.h" - These two files are the only ones needed for logging and will have 
                                         to be included in every file that requires logging functionality.

• Backend formatting for user-defined and standard library types

One of the significant changes lies in the support for formatting both user-defined and standard library types. Previously, the backend thread handled the formatting of these types sent by the frontend. It involved making a copy for any object passed to the LOG_ macros as an argument using the copy constructor of a complex type instead of directly serializing the data to the SPSC queue. While this method facilitated logging copy-constructible user-defined types with ease, it also posed numerous challenges for asynchronous logging:

• Error-Prone Asynchronous Logging: Copying and formatting user-defined types on the backend thread in an asynchronous logging setup could lead to errors. Previous versions attempted to address this issue with type trait checks, which incurred additional template instantiations and compile times.
• Uncertainty in Type Verification: It is challenging to confidently verify types, as some trivially copiable types, such as struct A { int* m; }, could still lead to issues due to potential modifications by the user before formatting.
• Hidden Performance Penalties: Logging non-trivially copiable types could introduce hidden cache coherence performance penalties due to memory allocations and deallocations across threads. For instance, consider std::vector<int> passed as a log argument. The vector is emplaced into the SPSC queue by the frontend, invoking the copy constructor dynamically allocating memory as the only members copied to SPSC queue are size, capacity, and data*. The backend thread reads the object, formats it, and then invokes the destructor, which in turn synchronizes the freed memory back to the frontend.

Additionally, after years of professional use and based on experience, it has been observed that user-defined types are often logged during program initialization, with fewer occurrences on the hot path where mostly built-in types are logged. In such scenarios, the overhead of string formatting on the frontend during initialization is not an issue.

In this new version, the use of the copy constructor for emplacing objects in the queue has been abandoned. Only POD types are copied, ensuring that only raw, tangible data is handled without any underlying pointers pointing to other memory locations. The only exception to this are the pointers to Metadata, LoggerBase and DecodeFunction that are passed internally for each log message. Log arguments sent from the frontend must undergo serialization beforehand. While this approach resolves the above issues, it does introduce more complexity when dealing with user-defined or standard library types.

Built-in types and strings are logged by default, with the formatting being offloaded to the backend. Additionally, there is built-in support for most standard library types, which can also be directly passed to the logger by including the relevant header from quill/std.

The recommendation for user-defined types is to format them into strings before passing them to the LOG_ macros using your preferred method. You can find an example of this here.

It's also possible to extend the library by providing template specializations to serialize the user-defined types and offload their formatting to the backend. However, this approach should only be pursued if you cannot tolerate the formatting overhead in that part of your program. For further guidance, refer to this example.

• Header-Only library

The library is now header-only. This change simplifies exporting the library as a C++ module in the future. See here on how to build a wrapper static library which includes the backend and will minimise the compile times.

• Preprocessor flags moved to template parameters

Most preprocessor flags have been moved to template parameters, with only a few remaining as CMake options. This change simplifies exporting the library as a C++ module in the future.

• Renamed Handlers to Sinks

To enhance clarity, handlers have been renamed to sinks.

• PatternFormatter moved to Logger

The PatternFormatter has been relocated from Sink to Logger, enabling a logger object to log in a specific format. This allows for different formats within the same output file, a feature not previously possible.

• Split Configuration

The configuration settings have been divided into FrontendOptions and BackendOptions.

• Refactoring of backend classes

MacroMetadata and many backend classes have undergone refactoring, resulting in reduced memory requirements.

• Improved wide strings handling on Windows

The library now offers significant performance enhancements for handling wide strings on Windows platforms. It's important to note that only wide strings containing ASCII characters are supported. Previously, wide strings were converted to narrow strings at the frontend, impacting the critical path of the application. With this update, the underlying wide char buffer is copied and the conversion to UTF-8 encoding is deferred to the backend logging thread. Additionally, this update adds support for logging STL containers consisting of wide strings

• Default logger removal

The default logger, along with the configuration inheritance feature during logger creation, has been removed. Now, when creating a new logger instance, configurations such as the Sink and log pattern format must be explicitly specified each time. This simplifies the codebase.

• Global logger removal

The static global logger* variable that was initialised during quill::start() used to obtain the default logger has been removed. It is possible to add this on the user side. If you require a global logger you can have a look at this example

• Removal of printf style formatting support

The support for printf style formatting has been removed due to its limited usage and the increased complexity. Users requiring this feature should stay on v3.x.x versions to maintain compatibility.

• Removal of external libfmt usage

The option to build the library with external libfmt has been removed. It becomes difficult to maintain and backwards support previous versions of libfmt. Instead, libfmt is now an internal component of the library, accessible under the namespace fmtquill. You can use the bundled version of fmtquill by including the necessary headers from quill/bundled/fmt. Alternatively, you have the freedom to integrate your own version. Since libfmt is encapsulated within a distinct namespace, there are no conflicts even if you link your own libfmt alongside the logging library.

Migration Guidance

• Revise include files to accommodate the removal of Quill.h
• Update the code that starts the backend thread and the logger/sink creation. You can refer to any of the updated examples, such as this one
• When logging statements involving user-defined types, make sure these types are formatted into strings using your preferred method. Refer to this link for guidance. Alternatively, if you prefer delaying the conversion to strings until the backend thread and only passing a binary copy of the user-defined type on the hot path, you can provide the necessary class template specializations for each user-defined type. See an example here