6 Python Logging Best Practices You Should Be Aware Of

Python is one of the most successful programming languages. In this article, we’ll discuss the best practices for logging with Python. We’ll begin with some fundamentals and review the native Python logging facility, its standard logging levels, and how to configure it. Afterward, we’ll present our list of six logging best practices. These practices include Python-specific guidance and more general guidance you can be apply to other programming languages. Each of these best practices will improve your logging strategy.

Python Logging: Starting With the Basics

Unlike many other programming languages, Python comes with a built-in logging module. We’re going to cover the module and its logging levels along with basic configuration instructions in this section.

Python Default Logging Module

Python’s standard library includes a flexible built-in logging module, allowing you to create different configurations to fulfill your logging needs.

Python’s logging module consists of functions designed to allow developers to log to different destinations. They do this by defining different handlers and routing the log messages to the adequate handlers.

The following code exemplifies a simple use of the logging module:

import logging
logging.info('I'm an informational message.')
logging.debug('I'm a message for debugging purposes.')
logging.warning('I'm a warning. Beware!')

Logging Levels

Most logging tools provide different logging levels, and Python is no exception. In the previous example, you can see three of them: INFO, DEBUG, and WARNING. These are some of the logging levels available in the logging module. The complete list, in order of increasing severity, is as follows:

● NOTSET
● DEBUG
● INFO
● WARNING
● ERROR
● CRITICAL

But what are logging levels? Put shortly, they’re labels you add to your log entries. Afterward, you can use these labels to search and filter through your log entries.

Logging levels also help you manage the granularity of information in your logs. When you set a log level using the standard logging library, only events of that level or higher will be recorded.

Basic Logging Configuration

The main components of the logging module are loggers, handlers, and formatters. Loggers are the objects you call when you want to record a message in your application code. Handlers are also called targets, appenders, or writers on different platforms. They are the components effectively writing the messages to their destination. You could have a handler to log to one file, another to log to a different file, and another to log to syslog, for instance. Formatters, as their name implies, are responsible for formatting the layout of log messages.

Most configurations consist of a handler and a formatter. The API provides a method for logging, basicConfig(). Check out the following example:

import logging
logging.basicConfig(level=logging.INFO)
logging.info('This message will be logged')
logging.debug('This message will not be logged')

The example above uses the basicConfig() method to configure the logger with the INFO level, which means events with a level of INFO or higher will get logged and others won’t.
Now consider the following example:

logging.basicConfig(filename='myfirstlog.log', 
level=logging.DEBUG, 
format='%(asctime)s | %(name)s | %(levelname)s | %(message)s')

Here, we’re configuring more elaborate formatting. If we were to log the message “testing log formatting!” using the WARN level, the result would look like this:

2020-03-12 12:54:48,312 | root | WARN | Testing log formatting!

Python Logging Best Practices

Here are six Python logging best practices to help you on your way.

Don’t Reinvent the Wheel

Going with print statements or even writing to files might be tempting because it’s so easy, but it comes with a price. If you don’t use a proper logging solution, you can’t enjoy all the other benefits they have. Your primitive approach might suffice for a trivial application, but it’ll give you headaches when the app starts to grow. Your logs will be full of noise and void of signal because they don’t have levels to manage granularity. And what about the flexibility to easily change log destinations? Forget it. Without handlers, you’re out of luck.

If you’re trying to learn about wheels, sure, you can reinvent the wheel. Do you want to implement a mature logging solution? Go for it! You’ll learn a lot and develop a new appreciation for the challenges involved in logging. But if this isn’t the case for you, stick to tried-and-true solutions.

Use the Python Standard Logging Module

Speaking of tried-and-true solutions, the first “do” on our list should be unsurprising: stick to the standard logging module. It was designed to be flexible and easy to use. As you’ve already seen, the logging module allows you to easily define handlers and formatters and use them to create powerful combinations.

Use the Correct Levels When Logging

It might be difficult to decide which level to assign each event. Fortunately, the Python logging module presents fewer levels than other logging libraries. This makes things easier by eliminating some potential ambiguity. When it comes to Python levels, here are the general guidelines:

● DEBUG: You should use this level for debugging purposes in development.
● INFO: You should use this level when something interesting—but expected—happens (e.g., a user starts a new project in a project management application).
● WARNING: You should use this level when something unexpected or unusual happens. It’s not an error, but you should pay attention to it.
● ERROR: This level is for things that go wrong but are usually recoverable (e.g., internal exceptions you can handle or APIs returning error results).
● CRITICAL: You should use this level in a doomsday scenario. The application is unusable. At this level, someone should be woken up at 2 a.m.

Include a Timestamp for Each Log Entry

Knowing something happened without knowing when it happened is only marginally better than not knowing about the event at all. Make sure to add a timestamp to your log entries to make the lives of the people who use logs for troubleshooting easier. Doing so also allows developers to analyze the log entries to obtain insights/analytics about user behavior.

Adopt the ISO-8601 Format for Timestamps

Timestamps are essential in log entries. Unfortunately, people can’t agree on the best way to express instants in time, so we came up with several conflicting formats.

Using the format widely used in your country might look like it’s the right choice, especially if you don’t plan to offer your application overseas.

But this couldn’t be further from the truth. By simply adopting a standard format for your timestamps, you can prevent problems, as third-party libraries and tools will expect the standard format in the first place.

This standard format exists, and it’s called ISO-8601. It’s an international standard for the exchange of date- and time-related data. Here’s an example of a timestamp expressed in ISO-8601 format:

2020-03-14T15:00-03:00

This is a basic example of how to configure the formatting to allow ISO-8601 timestamps:

import logging
logging.basicConfig(format='%(asctime)s %(message)s')
logging.info('Example of logging with ISO-8601 timestamp')

Use the RotatingFileHandler Class

A general logging best practice—in any language—is to use log rotation. This is a mechanism designed to automatically archive, compress, or delete old log files to prevent full disks.

Fortunately, you don’t have to implement this by hand in Python. Instead, use the RotatingFileHandler class instead of the regular FileHandler one.

Next Steps

Python is one of the most popular languages. It offers simplicity, versatility, and a large ecosystem of third-party tools. Python is versatile enough to address a wide variety of use cases, from web applications to data science libraries, SysAdmin scripts, and many other types of programs.

Because logging is essential for most applications, reviewing Python logging and highlighting best practices is valuable for all tech pros, especially for those who are just getting familiar with it.
What should your next steps be? For starters, don’t stop learning. SolarWinds® Loggly® offers multiple resources to help, from the Ultimate Guide—which can walk you through setting up Python logging for your application—to topic blogs highlighting specific use cases in Python.

If you prefer a more hands-on approach, you should consider starting a free trial of Loggly.

Loggly allows you to centrally manage, search, and analyze your Python logs. Loggly is unlike any other Python logger; it automatically parses your logs as soon as it receives them. If you implement the log levels described above, this means your logs are automatically structured. This structure allows you to quickly browse and click on log data to search and view logs in context, which can shorten analysis and troubleshooting.

Getting started with Loggly is easy; there’s no need to install an agent or run through complicated configuration steps. You can send your Python logs to Loggly over syslog or over HTTP using a RESTful API.

This post was written by Carlos Schults. Carlos is a .NET software developer with experience in desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices designed to help improve app health, such as code review, automated testing, and continuous build.

Last updated: 2023