How To Improve Logging

at Mon Feb 11, 2019 in

tutorials

TL;DR

What: A few tips on how to take the most out of your logs

How: Log smart, give context, uniquely identify all things, add time.

Introduction

The alarm is sounding, the screen is blank and your only hope is… that’s right, the logs.

Surprisingly enough this is very often overlooked during the development process and this will invariably add a lot of difficulties in the future.

Logs give you visibility, letting you “see” what is happening under the hood. This at first doesn’t seem very important, but wait until when your code is in production, deployed into a multitude of servers, and you need to understand why the hell your system is not doing what it is supposed to.

A clear and meaningful message and a well-defined structure is a very good start but not enough, below are some insights that might help to extract more from your logs.

1. Understand The Levels

According to RFC 5424, there are 8 logging levels, they represent the severity level of an event.

• DEBUG: Detailed debug information.
• INFO: Interesting events. Examples: User logs in, SQL logs.
• NOTICE: Normal but significant events.
• WARNING: Exceptional occurrences that are not errors. Examples: Use of deprecated APIs, poor use of an API, undesirable things that are not necessarily wrong.
• ERROR: Runtime errors that do not require immediate action but should typically be logged and monitored.
• CRITICAL: Critical conditions. Example: Application component unavailable, unexpected exception.
• ALERT: Action must be taken immediately. Example: Entire website down, database unavailable, etc. This should trigger the SMS alerts and wake you up.
• EMERGENCY: Emergency: the system is unusable.

2. Define A Goal

Logs can serve a lot of purposes like debugging, user behavior insights, business intelligence, alerts, etc.

So set your goals and build your logging strategy around it. Depending on your goals it will directly affect the log structure and specially what you will log.

Some advocate for logging all the things, and usually I tend to agree with this but it can be problematic because it tends to produce a lot of noise and can hide important data if you are not careful.

Be specific, for example, if your main goal is business intelligence, you will probably not need exception stack traces, query parameters, etc.

3. Create a Standard And Stick To It

Develop a standardized way of logging and stick to it. Logs usually are massive amounts of data, saving them properly and consistently will facilitate to spot patterns and filter what you need.

4. Give Context

Be concise and human friendly, give context.

\\bad
$logger->error('Parse error on database query'); 
\\good
$logger->error('Parse error on database query', ['query_p arameters'=>$queryParameters]);
\\best
$logger->error('Parse error on database query to retrieve
users orders', ['query_parameters'=>$queryParameters]);

5. Uniquely Identify The Crap Out Of Your Logs

Use unique identifiers as much as possible, and I cannot stress this enough, notably in distributed systems without unique identifiers is almost impossible to debug.

\\bad
$logger->info('user logged in');
\\good
$logger->info("user {$userId} logger in ");

6. Time Is Key

Adding date and time to each event is crucial for navigating properly among the actions.

• Put the time at the beginning of the line, it usually is the guiding information.
• Use four-digit year.
• Use timezone - This is crucial for multi-region systems.
• Be as verbose as possible.

7. Make logging a non-blocking action

Your system should continue its course in case something prevents your logging from working, one strategy is save locally and sync remotely, this way if the network connection to the logging service is down it won’t affect your application.

Extra Tip: Store It Somewhere Else

Be prepared for the worst, have your logging data in a different region or availability zone. In the case of outages, you will still be able to retrieve it ;)

Conclusion

With a little bit of extra effort, you can be better prepared and respond quicker in case of incidents.

Give some extra love to your logging and you will thank yourself later.

---