I don't think I encountered this problem working on this in Python 2.7, but while debugging in 3.7, Python throws an exception when pool.close() is called. This is the relevant part of the function:
pool = multiprocessing.Pool(6)
iterator = pool.imap_unordered(worker_func, worker_input)
while True:
try:
t0, t1 = next(iterator)
except multiprocessing.TimeoutError:
continue
except StopIteration:
break
else:
dbinserts1(t0)
dbinserts2(t1)
pool.close()
pool.join()
The only change made by 2to3 was rewriting iterator.next() as next(iterator). The function only fails while debugging, otherwise it runs successfully. This is (probably) the most relevant part of the stack trace:
Error in atexit._run_exitfuncs: Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/util.py", line 322, in _exit_function p.join() File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 138, in join assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
from Exception thrown on pool.close() while debugging, but not while running
No comments:
Post a Comment