The purpose of logging is to track the errors in any application, in a centralized manner. Logging should not be used only in big applications but also in smaller applications, especially if they play a crucial role with respect to the overall system. Log files can show any sort of event within an application or system, such as a failure, error, or a state transformation. When something inevitably goes wrong, such changes in state help indicate which transformation actually caused an error in the system.

Logging is mainly developed and used by system administrators on the operational level, for getting a high-level view of the complete system. A perfect log file should not be noisy and it should not contain distracting information. Log files should log only what is absolutely necessary and beneficial for error tracing.

Now, a common question that comes to every developer’s mind: “What exactly should I be logging?” 

This depends very much on the nature of the application which is being developed. Logging everything can be helpful as you have a huge amount of data to analyze when you have an error in the system. But it is not at all helpful if it needs to be inspected by a single person.

Instead, developers should be thinking about, what we can call as 3 R’s of logging: 

Logging the Right events in the Right Format for the Right user.

Some of the best practices for logging are:

1. Add a logging framework before you write the code.
2. The logging framework must support logging level modifications without restarting the application.
3. Logging should not be an afterthought.
4. We should avoid temporary log statements during development and also avoid ‘console.writeline’ statements.For example,

If we write logger function in such a way, then, in case if server or DB fails, then there will be an exception and logging will not be successful.

5. Write meaningful log messages and also add context to those messages.
6. Use timestamps and logtype for every logged event.

Common Logging Levels :

Here are some common logging levels, listed in order from least severe to most severe. When logging from your application, follow these levels:

1. TRACE

At this level, you are basically looking to capture every possible detail about the application’s behavior. This is likely to consume huge resources in production.

2. DEBUG

From this level, you’re getting into “noisy” territory and formatting more information than you’d want in normal production situations.

3. INFO

This level corresponds to normal application behavior. You need not care too much about these entries during normal operations, but they generally provide the skeleton of what happened.

4. WARN

This log level indicates that you might have a problem or an unusual situation is detected  

5. ERROR

An error is a serious issue and it represents the failure of something important in your application. In this, the application itself is not failing but, this will require attention probably sooner than later, but the application can still keep running.

6. FATAL

Fatal represents catastrophic situations in your application. Your application may stop to prevent some kind of serious problem, if possible.

There are many tools available in the market which any developer can use according to his/her requirement for a particular application. Some of these tools are:

1. Log4j ( Mainly used for Java )
2. Log4j2 ( Mainly used for Java )

Logging in Java requires using one of the available logging frameworks. These frameworks provide the methods and configuration necessary to create and send log messages. Java provides a built-in framework known as java.util.logging package. 

Abstraction layers such as SLF4J decouple the logging framework from your application, allowing us to change logging frameworks on demand. The abstraction layer mainly provides a generic API and determines which logging framework to bind to at runtime in an application.

The Java logging API mainly consists of 3 core components:

1. Loggers are responsible for capturing events and passing them to appropriate Appenders.
2. Appenders are responsible for recording log events to a destination file. 
3. Layouts are responsible for formatting the data in a log.

Logging frameworks are mainly configured through configuration files. These files are loaded by the logging framework at runtime. Although we can configure logging frameworks by code, using a configuration file is the preferred as it centralizes all configuration settings in a single location. 

Loggers are objects that are responsible to trigger log events. Loggers are created in the code of a Java application, and they generate events before passing them to an Appender. A class can have multiple independent Loggers responding to different events.Loggers provide several methods for recording log events. But, before you log an event, you need to assign a level to the event. Log levels determine the severity of the log.

Appenders forward logs from Loggers to an output directory or file. The log messages are formatted using a Layout before being sent to the destination.

Types of Appenders:

1. ConsoleAppender:

It is one of the most common Appenders, which simply displays log entries on the console. The ConsoleAppender is used as the default Appender and comes with basic settings. Sample Log4j2 configuration:

2. FileAppenders:

FileAppenders help to write log entries into a file. They are responsible for opening and closing files along with appending entries to files and also locking files to prevent data corruption. Sample Log4j2 configuration:

3. SyslogAppender:

SyslogAppenders send log entries to a Syslog server. It is a service that runs on a device and collects logs from the device’s operating system and various processes and services. Sample Log4j2 configuration:

If you are running an application in production, FileAppender or SyslogAppender will let you read old logs from a file.

Layouts convert the details of a log entry from one data type to another. Logging frameworks provide Layouts for plain text, HTML, syslog, XML, JSON and serialized.Layouts are generally configured using a configuration file, although starting with Java 7, SimpleFormatters can be configured using a system property.

To use a different Layout with java.util.logging,you can set the Appender’s formatter property to Layout of your choice. 

JSON format for logging:

JSON is quite portable and versatile, with most modern languages supporting it by default or through libraries. Its portability makes it ideal for log entries. Using JSON, a log can be read by many JSON interpreters. As the data is already structured, parsing a JSON log is easier than parsing a plain text log. This makes it easier to filter and analyze your logs.

By using Log4j, you can embed a JSON string into a JSON Layout by setting the JSONLayout’s ‘objectMessageAsJsonObject’ attribute to true in the XML file:

ThreadContext creates logs by adding unique data stamps to individual log entries. These stamps are managed on a per-thread basis and last till the thread is destroyed.This can be useful when you want to trace specific requests, transactions, or users through a very complex system.

NDC, or Nested Diagnostic Context, is based on the idea of  stack. Information can be pushed onto and popped from the stack. The values in the stack can then be accessed by a Logger without having to explicitly pass a value to the logging method.

The following example uses Log4j and NDC and to associate a userName with a log entry.

ThreadContext is a static class, so we can access methods without initializing an object.

We can store the current value of userName and sessionId of the user, in the stack using ThreadContext.push(userName) and ThreadContext.push(sessionId). 

ThreadContext.pop() removes individual items from the stack .

ThreadContext.removeStack() allows Java to reclaim the memory used by the stack, thus preventing memory leaks.

For example,

import org.apache.logging.Log4j.ThreadContext;

String userName = “ABC”;

String password = “12345”;

ThreadContext.push(userName);

ThreadContext.push(sessionId);

logger.info(“Login successful for user”);

ThreadContext.pop();

ThreadContext.remove();

Log4j can extract values from the NDC through the ’%x’ conversion character by using  PatternLayout class 

<PatternLayout pattern=”%x %-5p – %m%n” />

The code above gives the output as:

“ABC 12345 INFO – Login successful for user”

Markers allow us to mark individual log entries with unique data tokens. Markers can be used to group together entries and trigger actions. You can also combine Markers with ThreadContext in orderto improve the ability to search and filter log data.

For example, if we have a class that connects to a server.Suppose an exception occurs during communication to the server, we can log the exception as a fatal error.

import org.apache.logging.Log4j.Marker;

import org.apache.logging.Log4j.MarkerManager;

//Some Code

final static Marker SERVER_ERROR = MarkerManager.getMarker(“SERVER_ERROR”);

//Some Code

logger.fatal(SERVER_ERROR , “An exception occurred at server side.”);

To display marker in the log output, add the ‘%marker’ pattern to your PatternLayout method:

<PatternLayout pattern=”%p %marker: %m%n” />

The code above gives the output as:

[FATAL] DATABASE_ERROR: An exception occurred.

3. Serilog ( Mainly used for .NET )
4. NLog ( Mainly developed for .NET )

NLog is one of the best-performing and popular logging frameworks for .NET. Setting up NLog is quite simple. Developers can edit the ‘NLog.config’ file to set up intended targets and configure a target by updating the ‘NLog.config’ .When we modify ‘NLog.config’ file no code change or recompile is required

Here’s an example of how we can log using NLog:

.NET Core also supports a logging API that works with a variety of built-in and third-party logging providers known as ILogger. The responsibility of the ILogger interface is to display a log message in the specified log level. This interface mainly aggregates logging patterns to a single method.

The ILogger interface includes methods for logging to the file or DB. There are many extension methods that are available for logging. 

public interface ILogger 

{ 

void Log<TState>(LogLevel logLevel, EventId eventId, TState state); bool IsEnabled(LogLevel logLevel);

IDisposable BeginScope<TState>(TState state); 

}

The ‘ILoggerProvider’ manages and creates a logger, specified by the logging category.

public interface ILoggerProvider : IDisposable 

{ 

ILogger CreateLogger(string categoryName); 

}

With the help of ‘ILogger’ interface we can write a log to a file or DB. We can plug third-party tools like Serilog or Nlog for logging the logs to a file.

Conclusion:

Logging data for any application starts with choosing what to log and how to log the events in an application. The additional time spent in creating a good logging structure pays off when an error or unwanted event occurs at run-time in an application. Always keep in mind that inefficient and noisy logs can sometimes be accepted, but a good log structure and log data not only save time but also helps us to create better solutions for any problems in the application.