Tuesday, 7 May 2019

Exception thrown on pool.close() while debugging, but not while running

I don't think I encountered this problem working on this in Python 2.7, but while debugging in 3.7, Python throws an exception when pool.close() is called. This is the relevant part of the function:

pool = multiprocessing.Pool(6)
iterator = pool.imap_unordered(worker_func, worker_input)

while True:
    try:
        t0, t1 = next(iterator)
    except multiprocessing.TimeoutError:
        continue
    except StopIteration:
        break
    else:
        dbinserts1(t0)
        dbinserts2(t1)            

pool.close()
pool.join()

The only change made by 2to3 was rewriting iterator.next() as next(iterator). The function only fails while debugging, otherwise it runs successfully. This is (probably) the most relevant part of the stack trace:

Error in atexit._run_exitfuncs: Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/util.py", line 322, in _exit_function p.join() File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 138, in join assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process



from Exception thrown on pool.close() while debugging, but not while running

No comments:

Post a Comment