正文
Python 日志模块的定制
提示：扫一扫查出行【扫一扫了解最新限行尾号】
复制提示
Python标准logging模块中主要分为四大块内容：
Logger: 定义应用程序使用的接口
Handler: 将Loggers产生的日志输出到目的地
Filter: 对Loggers产生的日志进行过滤
Formatter: 格式化Loggers产生的日志
其中常用的是Logger, Handler, Formatter。
目标
对logging模块进行封装，使其尽量达到几个目标：
- 易用性：尽量保持python原生标准的用法。
- 扩展性：避免将所有模块集中在一个文件中进行配置，而是对不同模块继承扩展。如此可以确保修改时尽量保持简洁性，而非混淆杂乱。
- 文字加色：同一时间段内出现大量日志输出时，可以更清晰地识别出问题所在。
- 输出多个目的地：在生产代码中，并非所有日志信息都输出到console或者file中，而是日志的重要级别，将那些比较重要的发送到console，将那些不太重要但以后分析问题时可能又会用到的发送到file中。
- 非阻塞发送器：记录日志时如果涉及到网络（如socket, smtp），那么这个记录过程可能会阻塞当前线程，在python 3.2+，我们可以非常容易地设计一个非阻塞的发送器（当然了，非阻塞机制的内部实现大多都是队列）。
目录结构
如下的定制日志主要包括两个包，log_config和tools，目录结构如下：
log_config 模块结构
默认配置
-
Logging Level:
logging.NOTSET is default.
You can change it by passing ‘level=’ to BTLogger instance in your application code.
- Sending Level:
Send logging messages to console with levels of logging.INFO and higher, and simultaneously to the disk file with levels of logging.DEBUG and higher.
You can change them by passing ‘level=’ to BTStreamHandler/BTFileHandler instance in log_config\__init__.py.
- Log Config:
LOG_PATH = r'C:\log'
LOG_NAME = r'mylogger.log'
You can change them in log_config\__init__.py.
- Console Text Color:
DEBUG
INFO
WARNING
ERROR
CRITICAL
You can change them by passing ‘fore_color=’
and ‘back_color=’ to decorator @text_attribute.set_textcolor(fore_color, back_color)
in log_config\__init__.py.
-
There
also exists a NonBlockingHandler class in log_config\handlers.py, which
will use features of version 3.2+. If you unfold it, you can get a
non_blocking_handler.
Demo
ts_log_config.py:
def log_test():
import log_config logger = log_config.BTLogger('mylogger') logger.debug('Debug message')
logger.info('Info message')
logger.warning('Warning message')
logger.error('Error message')
logger.critical('Critical message') def main():
log_test() if __name__ == '__main__':
main()
outputs:
代码实现
..\log_config\__init__.py:
"""
Wrap the classes of logging module so as to expose more flexible interfaces that application code directly uses. By default, when you do some log operations, it will log messages to console with levels of logging.INFO and higher,
and simultaneously to the disk file with levels of logging.DEBUG and higher. Usages:
To use log_config module, simply 'import log_config' in your code, and create a BTLogger instance, then, you can use this
instance to log messages:
import log_config logger = log_config.BTLogger('mylogger') logger.debug('Debug message')
logger.info('Info message')
logger.warning('Warning message')
logger.error('Error message')
logger.critical('Critical message')
""" import os
import sys
import logging
from log_config import formatters
from log_config import handlers
from log_config import text_attribute
from tools import os_tools __author__ = " "
__status__ = "debugging" # {debugging, production}
__version__ = "0.1.0" # major_version_number.minor_version_number.revision_number
__date__ = " " """
# Config root logger
basicConfig(
filename=__name__,
filemode='w',
format='%(asctime)s: %(levelname)s: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S',
level='logging.NOTSET'
)
""" # Config log path and file name.
LOG_PATH = r'C:\log'
LOG_NAME = r'mylogger.log'
filename = 'default.log' if not os_tools.makedir(LOG_PATH) else os.path.abspath(os.path.join(LOG_PATH, LOG_NAME)) class BTLogger(logging.Logger):
def __init__(self, name=None, level=logging.NOTSET):
"""
Initialize the BTLogger with basic settings. :param
name: Specify the logger's name. If name is None, default is root logger.
Note:
All BTLogger instances with a given name return the same logger
instance. This means that logger instances never need to be passed
between different parts of an application.
level: The threshold of logging records, messages which are less severe
than level will be ignored.
If the level is set to NOTSET, than:
1> If current logger is root logger, than all messages will
be logged.
2> Or, delegate to its parent logger, until an ancestor with
a level other than NOTSET is found, or the root is reached.
Note:
The root logger is created with level logging.WARNING.
:return:
None
"""
logging.Logger.__init__(self, name, level) # Create handlers
# non_blocking_ch = handlers.NonBlockingHandler(handlers.BTStreamHandler(sys.stdout, logging.DEBUG)).get_handler() # version 3.2+
console_handler = handlers.BTStreamHandler(stream=sys.stdout, level=logging.INFO)
file_handler = handlers.BTFileHandler(filename=filename, level=logging.DEBUG, mode='w') # Set formatters to handlers.
# non_blocking_ch.setFormatter(formatters.BTStreamFormatter()) # version 3.2+
console_handler.setFormatter(formatters.BTStreamFormatter())
file_handler.setFormatter(formatters.BTFileFormatter()) # Add the handlers to logger.
# self.addHandler(non_blocking_ch) # version 3.2+
self.addHandler(console_handler)
self.addHandler(file_handler) # Override methods in logging.Logger(debug, info, warning, error, critical).
# @handlers.NonBlockingHandler.non_blocking # version 3.2+
@text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_WHITE')
def debug(self, msg, *args, **kwargs):
self.log(logging.DEBUG, msg, *args, **kwargs) @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_GREEN')
def info(self, msg, *args, **kwargs):
self.log(logging.INFO, msg, *args, **kwargs) @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_YELLOW')
def warning(self, msg, *args, **kwargs):
self.log(logging.WARNING, msg, *args, **kwargs) warn = warning @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED')
def error(self, msg, *args, **kwargs):
self.log(logging.ERROR, msg, *args, **kwargs) @text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED')
def critical(self, msg, *args, **kwargs):
self.log(logging.CRITICAL, msg, *args, **kwargs) fatal = critical
..\log_config\formatters.py:
"""
Formatters specify the output format of logging messages.
""" from logging import Formatter # Changed in version 3.2+: The 'style' parameter was added.
'''
class BTStreamFormatter(Formatter):
def __init__(self,
fmt='[%(asctime)s] [%(levelname)s]: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S', style='%'):
Formatter.__init__(self, fmt, datefmt, style) class BTFileFormatter(Formatter):
def __init__(self,
fmt='%(asctime)s: %(levelname)s: %(message)s',
datefmt='%Y/%m/%d %H:%M:%S', style='%'):
Formatter.__init__(self, fmt, datefmt, style)
''' # Be suitable for version 3.2-.
class BTStreamFormatter(Formatter):
def __init__(self,
fmt='[%(asctime)s] [%(levelname)s]: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'):
Formatter.__init__(self, fmt, datefmt) class BTFileFormatter(Formatter):
def __init__(self,
fmt='%(asctime)s: %(levelname)s: %(message)s',
datefmt='%Y/%m/%d %H:%M:%S'):
Formatter.__init__(self, fmt, datefmt)
..\log_config\handlers.py:
"""
Handlers send logging messages(created by BTLogger) to the specific destination.
""" import sys
import logging
from logging import StreamHandler
from logging import FileHandler class BTStreamHandler(StreamHandler):
"""
Write logging records to a stream(e.g., console). Note that this class does not close the stream automatically,
as sys.stdout or sys.stderr may be used.
"""
def __init__(self, stream=sys.stderr, level=logging.INFO):
"""
Initialize the handler. :param
stream: If stream is not specified, sys.stderr is used.
:return
None
"""
StreamHandler.__init__(self, stream)
self.setLevel(level) # The threshold of handling records, default is logging.INFO. class BTFileHandler(FileHandler):
"""
Write logging records to disk files. Inherited from logging.FileHandler, only modify the default mode('a') to ('w').
"""
def __init__(self,
filename='default.log',
level=logging.DEBUG, # Handle messages with levels of logging.DEBUG and higher to file.
mode='w',
encoding=None,
delay=False):
FileHandler.__init__(self, filename, mode, encoding, delay)
self.setLevel(level) # New in version 3.2+: QueueHandler, QueueListener.
# Unfold the following block, and you will get a non_blocking_handler.
'''
from queue import Queue
from logging.handlers import QueueHandler
from logging.handlers import QueueListener
from functools import wraps class NonBlockingHandler(object):
"""
Let logging handlers do their work without blocking the thread you're logging from.
Especially those network-based handler can block, e.g., SocketHandler, SMTPHandler.
"""
def __init__(self, handler_instance):
self.queue = Queue(-1)
self.queue_handler = QueueHandler(self.queue)
self.handler_instance = handler_instance
global _queue_listener
_queue_listener = QueueListener(self.queue, self.handler_instance) def get_handler(self):
return self.handler_instance @classmethod
def non_blocking(cls, func):
@wraps(func)
def wrapper(*args, **kwargs):
_queue_listener.start()
res = func(*args, **kwargs)
_queue_listener.stop()
return res return wrapper
'''
..\log_config\text_attribute.py:
"""
Set the text attributes in console. Currently, you can only set font color. # -----------------------------------------
# Author:
# Created:
# Modified:
# Version: 0.1.0
# ----------------------------------------- Usages:
When you want to design a function which outputs messages to console with different colors, simply use '@text_attribute.set_textcolor(fore_color, back_color)' to decorate your function, e.g.:
@text_attribute.set_textcolor('FOREGROUND_BLACK', 'BACKGROUND_RED')
def func(*args, **kwargs):
# output some messages to console
# ... func(...)
""" import ctypes
from functools import wraps STD_INPUT_HANDLE = -10
STD_OUTPUT_HANDLE = -11
STD_ERROR_HANDLE = -12 _CONSOLE_TEXT_COLOR = {
'FOREGROUND_RED': 0x04,
'FOREGROUND_GREEN': 0x02,
'FOREGROUND_BLUE': 0x01,
# 'FOREGROUND_INTENSITY': 0x08,
'FOREGROUND_WHITE': 0x07,
'FOREGROUND_BLACK': 0x00,
'FOREGROUND_YELLOW': 0x06, 'BACKGROUND_RED': 0x40,
'BACKGROUND_GREEN': 0x20,
'BACKGROUND_BLUE': 0x10,
# 'BACKGROUND_INTENSITY': 0x80,
'BACKGROUND_WHITE': 0x70,
'BACKGROUND_BLACK': 0x00,
'BACKGROUND_YELLOW': 0x60
} class TextColor(object):
"""
Set console text color.
"""
std_output_handle = ctypes.windll.kernel32.GetStdHandle(STD_OUTPUT_HANDLE) @staticmethod
def text_color(fore_color, back_color):
fore_color = 'FOREGROUND_WHITE' if fore_color not in _CONSOLE_TEXT_COLOR else fore_color
back_color = 'BACKGROUND_BLACK' if back_color not in _CONSOLE_TEXT_COLOR else back_color
return _CONSOLE_TEXT_COLOR[fore_color] | _CONSOLE_TEXT_COLOR[back_color] @staticmethod
def set(fore_color='FOREGROUND_WHITE', back_color='BACKGROUND_BLACK', handle=std_output_handle):
is_set = ctypes.windll.kernel32.SetConsoleTextAttribute(handle, TextColor.text_color(fore_color, back_color))
return is_set @staticmethod
def unset():
return TextColor.set() def set_textcolor(fore_color='FOREGROUND_WHITE', back_color='BACKGROUND_BLACK'): def decorate(func):
"""
A decorator which can set the color of console message, include foreground and background.
:param
func: The original function which will be wrapped by a wrapper.
:return:
wrapper
"""
@wraps(func)
def wrapper(*args, **kwargs):
TextColor.set(fore_color, back_color)
res = func(*args, **kwargs)
TextColor.unset()
return res return wrapper return decorate
..\tools\__init__.py: None
..\tools\os_tools.py:
"""
Originated from os module. # -----------------------------------------
# Author:
# Created:
# Modified:
# Version: 0.1.0
# -----------------------------------------
""" import os def makedir(path):
"""
Recursive directory creation, include all intermediate-level directories needed to contain the leaf directory. If specific path already exists, do nothing. Or path is not a directory, return False.
"""
try:
os.makedirs(path, 0o777)
# os.makedirs(path, 0o777, False) # Version 3.2+
except OSError:
if not os.path.isdir(path): # Path is not a valid directory.
return False # Path already exists.
return True
else: # Path does not exist.
return True
