Lines Matching full:logging
4 Logging Cookbook
9 This page contains a number of recipes related to logging, which have been found
12 .. currentmodule:: logging
14 Using logging in multiple modules
17 Multiple calls to ``logging.getLogger('someLogger')`` return a reference to the
25 import logging
29 logger = logging.getLogger('spam_application')
30 logger.setLevel(logging.DEBUG)
32 fh = logging.FileHandler('spam.log')
33 fh.setLevel(logging.DEBUG)
35 ch = logging.StreamHandler()
36 ch.setLevel(logging.ERROR)
38 formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
57 import logging
60 module_logger = logging.getLogger('spam_application.auxiliary')
64 self.logger = logging.getLogger('spam_application.auxiliary.Auxiliary')
100 Logging from multiple threads
103 Logging from multiple threads requires no special effort. The following example
104 shows logging from the main (initial) thread and another thread::
106 import logging
112 logging.debug('Hi from myfunc')
116 … logging.basicConfig(level=logging.DEBUG, format='%(relativeCreated)6d %(threadName)s %(message)s')
122 logging.debug('Hello from main')
154 This shows the logging output interspersed as one might expect. This approach
163 text file while simultaneously logging errors or above to the console. To set
164 this up, simply configure the appropriate handlers. The logging calls in the
168 import logging
170 logger = logging.getLogger('simple_example')
171 logger.setLevel(logging.DEBUG)
173 fh = logging.FileHandler('spam.log')
174 fh.setLevel(logging.DEBUG)
176 ch = logging.StreamHandler()
177 ch.setLevel(logging.ERROR)
179 formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
206 Logging to multiple destinations
215 import logging
217 # set up logging to file - see previous section for more details
218 logging.basicConfig(level=logging.DEBUG,
224 console = logging.StreamHandler()
225 console.setLevel(logging.INFO)
227 formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
231 logging.getLogger('').addHandler(console)
234 logging.info('Jackdaws love my big sphinx of quartz.')
239 logger1 = logging.getLogger('myapp.area1')
240 logger2 = logging.getLogger('myapp.area2')
276 Here is an example of a module using the logging configuration server::
278 import logging
279 import logging.config
284 logging.config.fileConfig('logging.conf')
287 t = logging.config.listen(9999)
290 logger = logging.getLogger('simpleExample')
293 # loop through logging calls to see the difference
304 logging.config.stopListening()
308 properly preceded with the binary-encoded length, as the new logging
332 .. currentmodule:: logging.handlers
334 Sometimes you have to get your logging handlers to do their work without
335 blocking the thread you're logging from. This is common in Web applications,
374 handler = logging.StreamHandler()
376 root = logging.getLogger()
378 formatter = logging.Formatter('%(threadName)s: %(message)s')
404 .. _network-logging:
406 Sending and receiving logging events across a network
409 Let's say you want to send logging events across a network, and handle them at
413 import logging, logging.handlers
415 rootLogger = logging.getLogger('')
416 rootLogger.setLevel(logging.DEBUG)
417 socketHandler = logging.handlers.SocketHandler('localhost',
418 logging.handlers.DEFAULT_TCP_LOGGING_PORT)
424 logging.info('Jackdaws love my big sphinx of quartz.')
429 logger1 = logging.getLogger('myapp.area1')
430 logger2 = logging.getLogger('myapp.area2')
441 import logging
442 import logging.handlers
448 """Handler for a streaming logging request.
450 This basically logs the record using whatever logging policy is
469 record = logging.makeLogRecord(obj)
482 logger = logging.getLogger(name)
491 Simple TCP socket-based logging receiver suitable for testing.
497 port=logging.handlers.DEFAULT_TCP_LOGGING_PORT,
516 logging.basicConfig(
546 Adding contextual information to your logging output
549 Sometimes you want logging output to contain contextual information in
550 addition to the parameters passed to the logging call. For example, in a
558 level of granularity you want to use in logging an application, it could
567 with logging event information is to use the :class:`LoggerAdapter` class.
576 information. When you call one of the logging methods on an instance of
591 contextual information is added to the logging output. It's passed the message
592 and keyword arguments of the logging call, and it passes back (potentially)
607 class CustomAdapter(logging.LoggerAdapter):
617 logger = logging.getLogger(__name__)
628 that it looks like a dict to logging. This would be useful if you want to
651 import logging
654 class ContextFilter(logging.Filter):
672 levels = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL)
673 logging.basicConfig(level=logging.DEBUG,
675 a1 = logging.getLogger('a.b.c')
676 a2 = logging.getLogger('d.e.f')
685 lvlname = logging.getLevelName(lvl)
708 Logging to a single file from multiple processes
711 Although logging is thread-safe, and logging to a single file from multiple
712 threads in a single process *is* supported, logging to a single file from
720 :ref:`This section <network-logging>` documents this approach in more detail and
732 .. currentmodule:: logging.handlers
735 all logging events to one of the processes in your multi-process application.
738 them according to its own logging configuration. Although the example only
741 analogous) it does allow for completely different logging configurations for
746 import logging
747 import logging.handlers
755 # Because you'll want to define the logging configurations for listener and workers, the
757 # for configuring logging for that process. These functions are also passed the queue,
767 root = logging.getLogger()
768 h = logging.handlers.RotatingFileHandler('mptest.log', 'a', 300, 10)
769 f = logging.Formatter('%(asctime)s %(processName)-10s %(name)s %(levelname)-8s %(message)s')
773 # This is the listener process top-level loop: wait for logging events
783 logger = logging.getLogger(record.name)
792 LEVELS = [logging.DEBUG, logging.INFO, logging.WARNING,
793 logging.ERROR, logging.CRITICAL]
805 # will run the logging configuration code when it starts.
807 h = logging.handlers.QueueHandler(queue) # Just the one handler needed
808 root = logging.getLogger()
811 root.setLevel(logging.DEBUG)
822 logger = logging.getLogger(choice(LOGGERS))
850 A variant of the above script keeps the logging in the main process, in a
853 import logging
854 import logging.config
855 import logging.handlers
866 logger = logging.getLogger(record.name)
871 qh = logging.handlers.QueueHandler(q)
872 root = logging.getLogger()
873 root.setLevel(logging.DEBUG)
875 levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR,
876 logging.CRITICAL]
881 logger = logging.getLogger(random.choice(loggers))
890 'class': 'logging.Formatter',
896 'class': 'logging.StreamHandler',
900 'class': 'logging.FileHandler',
906 'class': 'logging.FileHandler',
912 'class': 'logging.FileHandler',
934 logging.config.dictConfig(d)
941 # And now tell the logging thread to finish up, too
947 ``foo`` subsystem in a file ``mplog-foo.log``. This will be used by the logging
948 machinery in the main process (even though the logging events are generated in
990 .. (see <https://pymotw.com/3/logging/>)
996 logging package provides a :class:`~handlers.RotatingFileHandler`::
999 import logging
1000 import logging.handlers
1005 my_logger = logging.getLogger('MyLogger')
1006 my_logger.setLevel(logging.DEBUG)
1009 handler = logging.handlers.RotatingFileHandler(
1049 When logging was added to the Python standard library, the only way of
1055 Logging (as of 3.2) provides improved support for these two additional
1067 >>> import logging
1068 >>> root = logging.getLogger()
1069 >>> root.setLevel(logging.DEBUG)
1070 >>> handler = logging.StreamHandler()
1071 >>> bf = logging.Formatter('{asctime} {name} {levelname:8s} {message}',
1075 >>> logger = logging.getLogger('foo.bar')
1080 >>> df = logging.Formatter('$asctime $name ${levelname} $message',
1089 Note that the formatting of logging messages for final output to logs is
1090 completely independent of how an individual logging message is constructed.
1097 Logging calls (``logger.debug()``, ``logger.info()`` etc.) only take
1098 positional parameters for the actual logging message itself, with keyword
1100 logging call (e.g. the ``exc_info`` keyword parameter to indicate that
1103 you cannot directly make logging calls using :meth:`str.format` or
1104 :class:`string.Template` syntax, because internally the logging package
1107 all logging calls which are out there in existing code will be using %-format
1112 arbitrary object as a message format string, and that the logging package will
1169 approach: the actual formatting happens not when you make the logging call, but
1179 import logging
1189 class StyleAdapter(logging.LoggerAdapter):
1198 logger = StyleAdapter(logging.getLogger(__name__))
1204 logging.basicConfig(level=logging.DEBUG)
1211 .. currentmodule:: logging
1218 Every logging event is represented by a :class:`LogRecord` instance.
1226 logging an event. This invoked :class:`LogRecord` directly to create an
1238 :meth:`Logger.makeRecord`, and set it using :func:`~logging.setLoggerClass`
1255 logger = logging.getLogger(__name__)
1258 could also add the filter to a :class:`~logging.NullHandler` attached to their
1263 In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a
1265 :func:`~logging.setLogRecordFactory`, and interrogate with
1266 :func:`~logging.getLogRecordFactory`. The factory is invoked with the same
1267 signature as the :class:`~logging.LogRecord` constructor, as :class:`LogRecord`
1274 old_factory = logging.getLogRecordFactory()
1281 logging.setLogRecordFactory(record_factory)
1287 overhead to all logging operations, and the technique should only be used when
1348 return logging.makeLogRecord(msg)
1353 Module :mod:`logging`
1354 API reference for the logging module.
1356 Module :mod:`logging.config`
1357 Configuration API for the logging module.
1359 Module :mod:`logging.handlers`
1360 Useful handlers included with the logging module.
1362 :ref:`A basic logging tutorial <logging-basic-tutorial>`
1364 :ref:`A more advanced logging tutorial <logging-advanced-tutorial>`
1370 Below is an example of a logging configuration dictionary - it's taken from
1371 …he Django project <https://docs.djangoproject.com/en/stable/topics/logging/#configuring-logging>`_.
1374 LOGGING = {
1387 '()': 'project.logging.SpecialFilter',
1398 'class':'logging.StreamHandler',
1427 section <https://docs.djangoproject.com/en/stable/topics/logging/#configuring-logging>`_
1449 rh = logging.handlers.RotatingFileHandler(...)
1460 The following working example shows how logging can be used with multiprocessing
1468 see logging in the main process, how the workers log to a QueueHandler and how
1469 the listener implements a QueueListener and a more complex logging
1478 import logging
1479 import logging.config
1480 import logging.handlers
1488 A simple handler for logging events. It runs in the listener process and
1490 which then get dispatched, by the logging system, to the handlers
1496 logger = logging.getLogger()
1498 logger = logging.getLogger(record.name)
1502 # doing the logging to files and console
1511 This initialises logging according to the specified configuration,
1515 logging.config.dictConfig(config)
1516 listener = logging.handlers.QueueListener(q, MyHandler())
1525 logger = logging.getLogger('setup')
1536 This initialises logging according to the specified configuration,
1544 logging.config.dictConfig(config)
1545 levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR,
1546 logging.CRITICAL]
1556 logger = logging.getLogger('setup')
1560 logger = logging.getLogger(random.choice(loggers))
1571 'class': 'logging.StreamHandler',
1590 'class': 'logging.handlers.QueueHandler',
1600 # logging configuration is available to dispatch events to handlers however
1610 'class': 'logging.Formatter',
1614 'class': 'logging.Formatter',
1620 'class': 'logging.StreamHandler',
1625 'class': 'logging.FileHandler',
1631 'class': 'logging.FileHandler',
1637 'class': 'logging.FileHandler',
1654 # Log some initial events, just to show that logging in the parent works
1656 logging.config.dictConfig(config_initial)
1657 logger = logging.getLogger('setup')
1676 # Logging in the parent still works normally.
1696 :class:`~logging.handlers.SysLogHandler` to insert a BOM into the message, but
1707 #. Attach a :class:`~logging.Formatter` instance to your
1708 :class:`~logging.handlers.SysLogHandler` instance, with a format string
1726 :rfc:`5424`-compliant messages. If you don't, logging may not complain, but your
1730 Implementing structured logging
1733 Although most logging messages are intended for reading by humans, and thus not
1737 straightforward to achieve using the logging package. There are a number of
1742 import logging
1754 logging.basicConfig(level=logging.INFO, format='%(message)s')
1755 logging.info(_('message 1', foo='bar', bar='baz', num=123, fnum=123.456))
1772 import logging
1800 logging.basicConfig(level=logging.INFO, format='%(message)s')
1801 logging.info(_('message 1', set_value={1, 2, 3}, snowman='\u2603'))
1818 .. currentmodule:: logging.config
1823 There are times when you want to customize logging handlers in particular ways,
1835 return logging.FileHandler(filename, mode, encoding)
1837 You can then specify, in a logging configuration passed to :func:`dictConfig`,
1838 that a logging handler be created by calling this function::
1840 LOGGING = {
1874 import logging, logging.config, os, shutil
1881 return logging.FileHandler(filename, mode, encoding)
1883 LOGGING = {
1913 logging.config.dictConfig(LOGGING)
1914 logger = logging.getLogger('mylogger')
1952 :class:`~logging.FileHandler` - for example, one of the rotating file handlers,
1956 .. currentmodule:: logging
1963 In Python 3.2, the :class:`~logging.Formatter` gained a ``style`` keyword
1967 governs the formatting of logging messages for final output to logs, and is
1968 completely orthogonal to how an individual logging message is constructed.
1970 Logging calls (:meth:`~Logger.debug`, :meth:`~Logger.info` etc.) only take
1971 positional parameters for the actual logging message itself, with keyword
1972 parameters used only for determining options for how to handle the logging call
1976 logging calls using :meth:`str.format` or :class:`string.Template` syntax,
1977 because internally the logging package uses %-formatting to merge the format
1979 backward compatibility, since all logging calls which are out there in existing
1986 For logging to work interoperably between any third-party libraries and your
1988 individual logging call. This opens up a couple of ways in which alternative
1995 In Python 3.2, along with the :class:`~logging.Formatter` changes mentioned
1996 above, the logging package gained the ability to allow users to set their own
2015 :ref:`arbitrary-object-messages`) that when logging you can use an arbitrary
2016 object as a message format string, and that the logging package will call
2068 approach: the actual formatting happens not when you make the logging call, but
2078 .. currentmodule:: logging.config
2083 You *can* configure filters using :func:`~logging.config.dictConfig`, though it
2085 :class:`~logging.Filter` is the only filter class included in the standard
2087 base class), you will typically need to define your own :class:`~logging.Filter`
2088 subclass with an overridden :meth:`~logging.Filter.filter` method. To do this,
2092 :class:`~logging.Filter` instance). Here is a complete example::
2094 import logging
2095 import logging.config
2098 class MyFilter(logging.Filter):
2111 LOGGING = {
2121 'class': 'logging.StreamHandler',
2132 logging.config.dictConfig(LOGGING)
2133 logging.debug('hello')
2134 logging.debug('hello - noshow')
2151 in :ref:`logging-config-dict-externalobj`. For example, you could have used
2156 handlers and formatters. See :ref:`logging-config-dict-userdef` for more
2157 information on how logging supports using user-defined objects in its
2171 import logging
2173 class OneLineExceptionFormatter(logging.Formatter):
2188 fh = logging.FileHandler('output.txt', 'w')
2192 root = logging.getLogger()
2193 root.setLevel(logging.DEBUG)
2198 logging.info('Sample message')
2202 logging.exception('ZeroDivisionError: %s', e)
2220 Speaking logging messages
2223 There might be situations when it is desirable to have logging messages rendered
2236 import logging
2240 class TTSHandler(logging.Handler):
2252 root = logging.getLogger()
2255 root.setLevel(logging.DEBUG)
2258 logging.info('Hello')
2259 logging.debug('Goodbye')
2272 .. _buffered-logging:
2274 Buffering logging messages and outputting them conditionally
2279 start logging debug events in a function, and if the function completes without
2285 functions where you want logging to behave this way. It makes use of the
2286 :class:`logging.handlers.MemoryHandler`, which allows buffering of logged events
2295 all the logging levels, writing to ``sys.stderr`` to say what level it's about
2296 to log at, and then actually logging a message at that level. You can pass a
2301 conditional logging that's required. The decorator takes a logger as a parameter
2305 records buffered). These default to a :class:`~logging.StreamHandler` which
2306 writes to ``sys.stderr``, ``logging.ERROR`` and ``100`` respectively.
2310 import logging
2311 from logging.handlers import MemoryHandler
2314 logger = logging.getLogger(__name__)
2315 logger.addHandler(logging.NullHandler())
2319 target_handler = logging.StreamHandler()
2321 flush_level = logging.ERROR
2361 logger.setLevel(logging.DEBUG)
2401 As you can see, actual logging output only occurs when an event is logged whose
2420 import logging
2423 class UTCFormatter(logging.Formatter):
2427 :class:`~logging.Formatter`. If you want to do that via configuration, you can
2428 use the :func:`~logging.config.dictConfig` API with an approach illustrated by
2431 import logging
2432 import logging.config
2435 class UTCFormatter(logging.Formatter):
2438 LOGGING = {
2452 'class': 'logging.StreamHandler',
2456 'class': 'logging.StreamHandler',
2466 logging.config.dictConfig(LOGGING)
2467 logging.warning('The local time is %s', time.asctime())
2482 Using a context manager for selective logging
2485 There are times when it would be useful to temporarily change the logging
2487 manager is the most obvious way of saving and restoring the logging context.
2489 optionally change the logging level and add a logging handler purely in the
2492 import logging
2528 logger = logging.getLogger('foo')
2529 logger.addHandler(logging.StreamHandler())
2530 logger.setLevel(logging.INFO)
2533 with LoggingContext(logger, level=logging.DEBUG):
2536 h = logging.StreamHandler(sys.stdout)
2537 with LoggingContext(logger, level=logging.DEBUG, handler=h, close=True):
2584 logging filters temporarily. Note that the above code works in Python 2 as well
2595 * Use a logging level based on command-line arguments
2596 * Dispatch to multiple subcommands in separate files, all logging at the same
2605 command-line argument, defaulting to ``logging.INFO``. Here's one way that
2610 import logging
2642 logging.basicConfig(level=options.log_level,
2653 import logging
2655 logger = logging.getLogger(__name__)
2665 import logging
2667 logger = logging.getLogger(__name__)
2686 import logging
2688 logger = logging.getLogger(__name__)
2717 The first word is the logging level, and the second word is the module or
2720 If we change the logging level, then we can change the information sent to the
2750 A Qt GUI for logging
2762 can log to the GUI from both the UI itself (via a button for manual logging)
2763 as well as a worker thread doing work in the background (here, just logging
2777 import logging
2793 logger = logging.getLogger(__name__)
2801 signal = Signal(str, logging.LogRecord)
2814 class QtHandler(logging.Handler):
2834 # Used to generate random levels for logging.
2836 LEVELS = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR,
2837 logging.CRITICAL)
2878 logging.DEBUG: 'black',
2879 logging.INFO: 'blue',
2880 logging.WARNING: 'orange',
2881 logging.ERROR: 'red',
2882 logging.CRITICAL: 'purple',
2901 formatter = logging.Formatter(fs)
2952 @Slot(str, logging.LogRecord)
2974 logging.getLogger().setLevel(logging.DEBUG)