Friday, 30 April 2021

How to redirect logs from secondary threads in Azure Functions using Python

I am using Azure functions to run a Python script that launches multiple threads (for performance reasons). Everything is working as expected, except for the fact that only the info logs from the main() thread appear on the Azure Functions log. All the logs that I am using in the "secondary" threads that I start in main() do not appear in the Azure Functions logs.

Is there a way to ensure that the logs from the secondary threads show on the Azure Functions log?

The modules that I am using are "logging" and "threading".

I am using Python 3.6; I have already tried to lower the logging level in the secondary threads, but this did not help unfortunately. The various secondary thread functions are in different modules.

My function has a structure similar to the following pseudo-code:

def main()->None:
  logging.basicConfig(level=logging.INFO)
  logging.info("Starting the process...")
  thread1 = threading.Thread(target=foo,args=("one arg",))
  thread2 = threading.Thread(target=foo,args=("another arg",))
  thread3 = threading.Thread(target=foo,args=("yet another arg",))
  thread1.start()
  thread2.start()
  thread3.start()
  logging.info("All threads started successfully!")
  return

# in another module

def foo(st:str)->None:
  logging.basicConfig(level=logging.INFO)
  logging.info(f"Starting thread for arg {st}")

The current Azure log output is:

INFO: Starting the process...
INFO: "All threads started successfully!"

I would like it to be something like:

INFO: Starting the process...
INFO: "All threads started successfully!"
INFO: Starting thread for arg one arg
INFO:Starting thread for arg  another arg
INFO:Starting thread for arg  yet another arg

(of course the order of the secondary threads could be anything)



from How to redirect logs from secondary threads in Azure Functions using Python

No comments:

Post a Comment