On my MacBook (version 10.14.6) I am succesfully running a Django application including TensorFlow and tf-sentencepiece (in particular to use the universal sentence encoder model). When I perform a pipenv lock -r > requirements.txt I get the following required packages:
-i https://pypi.org/simple
absl-py==0.8.1
aniso8601==7.0.0
astor==0.8.0
bert-embedding==1.0.1
boto3==1.10.34
boto==2.49.0
botocore==1.13.34
certifi==2019.11.28
chardet==3.0.4
coreapi==2.3.3
coreschema==0.0.4
cycler==0.10.0
defusedxml==0.6.0
diff-match-patch==20181111
django-cors-headers==3.2.0
django-elasticsearch-dsl-drf==0.18
django-elasticsearch-dsl==0.5.1
django-filter==2.2.0
django-import-export==2.0
django-neomodel==0.0.4
django-nine==0.2.2
django-pandas==0.6.1
django-rest-framework==0.1.0
django-rest-swagger==2.2.0
django==2.2.8
djangorestframework==3.10.3
docutils==0.15.2
elasticsearch-dsl==6.1.0
elasticsearch==6.4.0
et-xmlfile==1.0.1
gast==0.3.2
gensim==3.8.1
gluonnlp==0.6.0
graphene-neo4j==2.2.3.9
graphene==2.1.8
graphql-core==2.2.1
graphql-relay==2.0.1
graphviz==0.8.4
grpcio==1.25.0
h5py==2.10.0
idna==2.8
ipaddress==1.0.23
itypes==1.1.0
jdcal==1.4.1
jinja2==2.10.3
jmespath==0.9.4
joblib==0.14.0
keras-applications==1.0.8
keras-preprocessing==1.1.0
kiwisolver==1.1.0
lazy-import==0.2.2
lxml==4.4.2
markdown==3.1.1
markuppy==1.14
markupsafe==1.1.1
matplotlib==3.1.2
mock==3.0.5
mxnet==1.4.0
neo4j-driver==1.6.3
neomodel-next==3.5.2.5
neomodel==3.3.1
neotime==1.7.4
nltk==3.4.5
numpy==1.14.6
odfpy==1.4.0
openapi-codec==1.3.2
openpyxl==3.0.2
pandas==0.25.3
promise==2.2.1
protobuf==3.11.1
pycld2==0.41
pyparsing==2.4.5
python-dateutil==2.8.0
pytorch-pretrained-bert==0.6.2
pytz==2019.3
pyyaml==5.2
regex==2019.11.1
requests==2.22.0
risparser==0.4.3
rx==1.6.1
s3transfer==0.2.1
scikit-learn==0.22
scipy==1.3.3
simplejson==3.17.0
singledispatch==3.4.0.3
six==1.13.0
sklearn==0.0
smart-open==1.9.0
sqlalchemy==1.3.11
sqlparse==0.3.0
tablib==0.14.0
tensorboard==1.13.1
tensorflow-estimator==1.13.0
tensorflow-hub==0.7.0
tensorflow==1.13.1
termcolor==1.1.0
tf-sentencepiece==0.1.82
torch==1.3.1
tqdm==4.40.1
typing==3.6.6
uritemplate==3.0.0
urllib3==1.25.7
werkzeug==0.16.0
wheel==0.33.6 ; python_version >= '3'
xlrd==1.2.0
xlwt==1.3.0
The above list is also generated in my Dockerfile:
# The first instruction is what image we want to base our container on
# We Use an official Python runtime as a parent image
FROM python:3.7-stretch
# The enviroment variable ensures that the python output is set straight
# to the terminal with out buffering it first
ENV PYTHONUNBUFFERED 1
# create root directory for our project in the container
RUN mkdir /var/lib/django
# Set the working directory to /var/lib/django
WORKDIR /var/lib/django
# Copy the current directory contents into the container at /var/lib/django
ADD . /var/lib/django/
# Install any needed packages specified in pipfile using piping
RUN pip install --upgrade pip
RUN pip install pipenv
# Below are packages that need to be added to allow C tools
RUN apt update
RUN apt-get -y install libc-dev
RUN apt-get -y install build-essential
# Install requirements
RUN pipenv lock -r > requirements.txt
RUN pip install -r requirements.txt
# Run onDockerStartup file (executes python required functions for startup)
RUN python3 onDockerStartup.py
EXPOSE 8000
CMD ["python3", "manage.py", "runserver", "0.0.0.0:8000", "--noreload"]
However, when I start my application with docker-compose I get the following error:
django_1 | Traceback (most recent call last):
django_1 | File "manage.py", line 21, in <module>
django_1 | main()
django_1 | File "manage.py", line 17, in main
django_1 | execute_from_command_line(sys.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
django_1 | utility.execute()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
django_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
django_1 | self.execute(*args, **cmd_options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 60, in execute
django_1 | super().execute(*args, **options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
django_1 | output = self.handle(*args, **options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 95, in handle
django_1 | self.run(**options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 104, in run
django_1 | self.inner_run(None, **options)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
django_1 | self.check(display_num_errors=True)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 390, in check
django_1 | include_deployment_checks=include_deployment_checks,
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 377, in _run_checks
django_1 | return checks.run_checks(**kwargs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
django_1 | new_errors = check(app_configs=app_configs)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
django_1 | all_namespaces = _load_all_namespaces(resolver)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
django_1 | url_patterns = getattr(resolver, 'url_patterns', [])
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 584, in url_patterns
django_1 | patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
django_1 | res = instance.__dict__[self.name] = self.func(instance)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/resolvers.py", line 577, in urlconf_module
django_1 | return import_module(self.urlconf_name)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/var/lib/django/backend/urls.py", line 29, in <module>
django_1 | path(r'search/', include('apps.search.urls')),
django_1 | File "/usr/local/lib/python3.7/site-packages/django/urls/conf.py", line 34, in include
django_1 | urlconf_module = import_module(urlconf_module)
django_1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
django_1 | return _bootstrap._gcd_import(name[level:], package, level)
django_1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
django_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
django_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
django_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
django_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
django_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
django_1 | File "/var/lib/django/apps/search/urls.py", line 4, in <module>
django_1 | from . import views
django_1 | File "/var/lib/django/apps/search/views.py", line 24, in <module>
django_1 | from .servicelayer.question_answer import find_answer_to_question
django_1 | File "/var/lib/django/apps/search/servicelayer/question_answer.py", line 1, in <module>
django_1 | from ml.use import embed_use
django_1 | File "/var/lib/django/ml/use.py", line 9, in <module>
django_1 | import tf_sentencepiece
django_1 | File "/usr/local/lib/python3.7/site-packages/tf_sentencepiece/__init__.py", line 5, in <module>
django_1 | from tf_sentencepiece.sentencepiece_processor_ops import *
django_1 | File "/usr/local/lib/python3.7/site-packages/tf_sentencepiece/sentencepiece_processor_ops.py", line 43, in <module>
django_1 | _gen_sentencepiece_processor_op = tf.load_op_library(so_file)
django_1 | File "/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library
django_1 | lib_handle = py_tf.TF_LoadLibrary(library_filename)
django_1 | tensorflow.python.framework.errors_impl.NotFoundError: /usr/local/lib/python3.7/site-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.13.1: undefined symbol: _ZN10tensorflow12OpDefBuilder4AttrESs
How should I solve this error to get sentencepiece to start successfully?
from Undefined symbol when importing tf-sentencepiece
No comments:
Post a Comment