Python Requests library throws exceptions in logging -

Python Requests library throws exceptions in logging -

the python requests library appears have rather unusual quirks when comes logging behaviour. using latest python 2.7.8, have next code:

import requests import logging logging.basicconfig( filename='mylog.txt', format='%(asctime)-19.19s|%(task)-36s|%(levelname)s:%(name)s: %(message)s', level=eval('logging.%s' % 'debug')) logger = logging.getlogger(__name__) logger.info('myprogram starting up...', extra={'task': ''}) # far ... (ommited code) ... payload = {'id': 'abc', 'status': 'ok'} # @ point programme continues throws exception. requests.get('http://localhost:9100/notify', params=payload) print 'task complete! notifyurl hit! - exiting'

my programme seems exit normally, within log file creates (mylog.txt) find next exception:

keyerror: 'task' logged file connectionpool.py, line 362

if remove this: requests.get('http://localhost:9100/notify', params=payload) exception gone.

what doing wrong here , how may prepare this? using requests v2.4.3.

as indicated in t-8ch's answer, logger beingness used internally requests library , library doesn't know parameters. possible solution implant custom filter in library's logger (in case, 1 of modules):

class taskaddingfilter(logging.filter): def __init__(self): logging.filter.__init__(self) def filter(self, record): record.args = record.args + ('task', '') # ... requestslogger = logging.getlogger('requests.packages.urllib3.connectionpool') requestslogger.addfilter(taskaddingfilter())

potentially, need add together such filtering loggers requests, are:

requests.packages.urllib3.util requests.packages.urllib3.connectionpool requests.packages requests.packages.urllib3 requests.packages.urllib3.util.retry requests requests.packages.urllib3.poolmanager

in version, can find them using logging.logger.manager.loggerdict attribute. so, this:

for name,logger in logging.logger.manager.loggerdict.iteritems(): logger = logging.getlogger(name) # because of lazy initialization if name.startswith('requests.'): logger.addfilter(taskaddingfilter())

the taskaddingfilter can bit smarter of course, e.g. adding particular task entry depending on in code. i've added simplest solution code you've provided - exception doesn't occur anymore - wide range of possibilities seems obvious ;)

python python-2.7 logging python-requests urllib3