Перейти к содержанию

logging — Logging facility for Python

Documentation

What to log

  1. Handled and unhandled exceptions when they are not part of the application's normal workflow.
  2. Important events in the business process. For example, withdrawal of money; user ban.
  3. All requests to the service and their result: what request, with what method, to what URL, headers that are important to me, how the server responded, with what status, how long it took.
  4. Errors during validation of incoming data.
  5. Running and results of periodic tasks.
  6. The status of the steps in the start-stop service process.
  7. Security-related events: wrong tokens, passwords.
  8. Interactions with other services and their results: requests to and from them.
  9. Execution time for performance-critical code sections.

What not to write to the log

  1. Too frequent and insignificant events.
  2. Secret and private data: filter them out.
  3. Uninformative records from which it is not clear what happened.
  4. Don't use print.

Severity levels

Method Numeric value Description
DEBUG 10 Detailed information, typically of interest only when diagnosing problems.
INFO 20 Confirmation that things are working as expected.
WARNING (default) 30 An indication that something unexpected happened, or indicative of some problem in the near future (e.g. 'disk space low'). The software is still working as expected.
ERROR 40 Due to a more serious problem, the software has not been able to perform some function.
CRITICAL 50 A serious error, indicating that the program itself may be unable to continue running.

Categories of components

  1. Loggers expose the interface that application code directly uses.
  2. Handlers send the log records (created by loggers) to the appropriate destination.
  3. Filters provide a finer grained facility for determining which log records to output.
  4. Formatters specify the layout of log records in the final output.

Log event information is passed between loggers, handlers, filters and formatters in a LogRecord instance.

Logging is performed by calling methods on instances of the Logger class.

Destinations are served by handler classes. You can specify a destination (such as console or file) by using basicConfig().

Loggers

These are the most common configuration methods:

  1. Logger.setLevel().
  2. Logger.addHandler().
  3. Logger.removeHandler().
  4. Logger.addFilter().
  5. Logger.removeFilter().

With the logger object configured, the following methods create log messages:

  1. Logger.debug(), Logger.info(), Logger.warning(), Logger.error(), and Logger.critical().
  2. Logger.exception() creates a log message similar to Logger.error(). The difference is that Logger.exception() dumps a stack trace along with it. Call this method only from an exception handler.
  3. Logger.log() takes a log level as an explicit argument.

Note

Call Logger.exception() method only from an exception handler.

getLogger() returns a reference to a logger instance with the specified name if it is provided, or root if not. The names are period-separated hierarchical structures. Multiple calls to getLogger() with the same name will return a reference to the same logger object.

Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to False.)

Handlers

Handler objects are responsible for dispatching the appropriate log messages (based on the log messages’ severity) to the handler’s specified destination. Logger objects can add zero or more handler objects to themselves with an addHandler() method.

As an example scenario, an application may want to send:

  1. all log messages to a log file;
  2. all log messages of error or higher to stdout;
  3. all messages of critical to an email address.

This scenario requires three individual handlers where each handler is responsible for sending messages of a specific severity to a specific location.

These are the most common configuration methods:

  1. Handler.setLevel().
  2. Handler.setFormatter().
  3. Handler.addFilter().
  4. Handler.removeFilter().

The NullHandler, StreamHandler and FileHandler classes are defined in the core logging package. The other handlers are defined in a sub-module, logging.handlers. (There is also another sub-module, logging.config, for configuration functionality.)

Useful Handlers

In addition to the base Handler class, many useful subclasses are provided: Useful Handlers.

StreamHandler

Documentation

The StreamHandler class, sends logging output to streams such as sys.stdout, sys.stderr or any file-like object (or, more precisely, any object which supports write() and flush() methods).

FileHandler

Documentation

The FileHandler class, sends logging output to a disk file. It inherits the output functionality from StreamHandler.

Formatters

Formatter objects configure the final order, structure, and contents of the log message.

The constructor takes three optional arguments – a message format string, a date format string and a style indicator. 

logging.Formatter.__init__(fmt=None, datefmt=None, style='%')

If there is no message format string, the default is to use the raw message. If there is no date format string, the default date format is: 

%Y-%m-%d %H:%M:%S
with the milliseconds tacked on at the end. The style is one of %, { or $. If one of these is not specified, then % will be used.

If the style is %, the message format string uses %(<dictionary key>)s styled string substitution; the possible keys are documented in LogRecord attributes.

If the style is {, the message format string is assumed to be compatible with str.format() (using keyword arguments), while if the style is $ then the message format string should conform to what is expected by string.Template.substitute().

Formatters use a user-configurable function to convert the creation time of a record to a tuple. By default, time. localtime() is used; to change this for a particular formatter instance, set the converter attribute of the instance to a function with the same signature as time.localtime() or time.gmtime(). To change it for all formatters, for example if you want all logging times to be shown in GMT, set the converter attribute in the Formatter class (to time.gmtime for GMT display).

Configuring Logging for a Library

Attention

It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers.

This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers 'under the hood', you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

Attention

Defining your own levels is possible, but should not be necessary, as the existing levels have been chosen on the basis of practical experience. However, if you are convinced that you need custom levels, great care should be exercised when doing this, and it is possibly a very bad idea to define custom levels if you are developing a library.

That’s because if multiple library authors all define their own custom levels, there is a chance that the logging output from such multiple libraries used together will be difficult for the using developer to control and/or interpret, because a given numeric value might mean different things for different libraries.

Exceptions raised during logging

The logging package is designed to swallow exceptions which occur while logging in production. This is so that errors which occur while handling logging events — such as logging misconfiguration, network or other similar errors — do not cause the application using logging to terminate prematurely.

The default implementation of handleError() in Handler checks to see if a module-level variable, raiseExceptions, is set. If set, a traceback is printed to sys.stderr. If not set, the exception is swallowed.

Note

The default value of raiseExceptions is True. This is because during development, you typically want to be notified of any exceptions that occur. It’s advised that you set raiseExceptions to False for production usage.

Using arbitrary objects as messages

You can pass an arbitrary object as a message, and its __str__() method will be called when the logging system needs to convert it to a string representation.

Optimization

Formatting of message arguments is deferred until it cannot be avoided. However, computing the arguments passed to the logging method can also be expensive, and you may want to avoid doing it if the logger will just throw away your event. To decide what to do, you can call the isEnabledFor() method which takes a level argument and returns true if the event would be created by the Logger for that level of call. You can write code like this:

if logger.isEnabledFor(logging.DEBUG):
    logger.debug('Message with %s, %s', expensive_func1(),
                                        expensive_func2())

Note

In some cases, isEnabledFor() can itself be more expensive than you’d like (e.g. for deeply nested loggers where an explicit level is only set high up in the logger hierarchy). In such cases (or if you want to avoid calling a method in tight loops), you can cache the result of a call to isEnabledFor() in a local or instance variable, and use that instead of calling the method each time.

Such a cached value would only need to be recomputed when the logging configuration changes dynamically while the application is running (which is not all that common).