Wednesday, 3 February 2021

Multiprocessing hangs only in virtualenv

I recently tried adding multiprocessing functionality to one of my projects, but I soon discovered that for some reason the call to Pool.map was hanging indefinitely.

I tried to make a separate script to test if Pool.map would work in a different context:

from multiprocessing import Pool


def hello(x):
    print(f"Hello there {x}!")


def main():
    p = Pool(5)
    p.map(hello, range(5))
    p.close()


if __name__ == '__main__':
    main()

This also didn't work (nothing would be printed, process was hanging). I then tried to run the above script without activating the virtual environment I had been using so far, and and sure enough, I did get the expected output:

Hello there 0!
Hello there 1!
Hello there 2!
Hello there 3!
Hello there 4!

The virtual environment was created using virtualenv .venv. virtualenv was installed using pip install virtualenv. I used python mp_test.py to run the above script. The .venv environment was activated from PowerShell using the .venv\Scripts\Activate.ps1.



from Multiprocessing hangs only in virtualenv

No comments:

Post a Comment