I am working on migrating a python2 project to python3. The program uses trained scikit models. The models where trained with version scikit-learn=0.20.3. After fixing the code so that it can run with python3, I have started re-training the models with the latest scikit-learn version ,0.24.1 .
After changing the import of joblib (no longer using sklearn.externals.joblib) and as said updated the version of scikit , the program hangs without doing anything(cpu usage drops down). The line where it stops is related to multiprocessing:
for j in jobs:
j.start()
for j in jobs:
j.join()
where jobs is defined as
jobs.append(Process(target=LJ_parallel, args=(a series of arguments)
- Can I try and find some obvious issue here on why, after retraining and upgrading, the program freezes? Is this simply some issue with multiprocessing related to the migration, to the update or simply to non-optimal coding?
Finally, this is the message I get after ctrl+c and stop the hang:
TRAPP: multi-core mode for LJ pocket simulations starts
^CTraceback (most recent call last):
File "../../..//scripts/trapp-pocket.py", line 2813, in <module>
j.join()
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/multiprocessing/process.py", line 140, in joi
n
res = self._popen.wait(timeout)
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/multiprocessing/popen_fork.py", line 48, in w
ait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/multiprocessing/popen_fork.py", line 28, in p
oll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/multiprocessing/popen_fork.py", line 28, in p
oll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
./visual_drugscoreexample.sh: line 12: cd: OUT: No such file or directory
^CTraceback (most recent call last):
File "../../..//scripts/visual_drugscore.py", line 2, in <module>
from mpl_toolkits.axes_grid1.colorbar import colorbar
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/mpl_toolkits/axes_grid1/__init_
_.py", line 1, in <module>
from . import axes_size as Size
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/mpl_toolkits/axes_grid1/axes_si
ze.py", line 14, in <module>
from matplotlib import cbook
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/matplotlib/__init__.py", line 1
07, in <module>
from . import cbook, rcsetup
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/matplotlib/cbook/__init__.py",
line 28, in <module>
import numpy as np
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/numpy/__init__.py", line 156, i
n <module>
from . import random
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/numpy/random/__init__.py", line
179, in <module>
from . import _pickle
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/site-packages/numpy/random/_pickle.py", line
1, in <module>
from .mtrand import RandomState
File "mtrand.pyx", line 1, in init numpy.random.mtrand
File "bit_generator.pyx", line 40, in init numpy.random.bit_generator
File "/home/mavridks/miniconda/miniconda3.7/envs/trappenv3/lib/python3.7/secrets.py", line 15, in <module>
import base64
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 724, in exec_module
File "<frozen importlib._bootstrap_external>", line 818, in get_code
File "<frozen importlib._bootstrap_external>", line 916, in get_data
KeyboardInterrupt
Thanks.
from .join hangs after updating scikit-learn=0.20.3 to latest version
No comments:
Post a Comment