Summary
I am working on a series of add-ons for Anki, an open-source flashcard program. Anki add-ons are shipped as Python packages, with the basic folder structure looking as follows:
anki_addons/
addon_name_1/
__init__.py
addon_name_2/
__init__.py
anki_addons
is appended to sys.path
by the base app, which then imports each add_on with import <addon_name>
.
The problem I have been trying to solve is to find a reliable way to ship packages and their dependencies with my add-ons while not polluting global state or falling back to manual edits of the vendored packages.
Specifics
Specifically, given an add-on structure like this...
addon_name_1/
__init__.py
_vendor/
__init__.py
library1
library2
dependency_of_library2
...
...I would like to be able to import any arbitrary package that is included in the _vendor
directory, e.g.:
from ._vendor import library1
The main difficulty with relative imports like this is that they do not work for packages that also depend on other packages imported through absolute references (e.g. import dependency_of_library2
in the source code of library2
)
Solution attempts
So far I have explored the following options:
- Manually updating the third-party packages, so that their import statements point to the fully qualified module path within my python package (e.g.
import addon_name_1._vendor.dependency_of_library2
). But this is tedious work that is not scalable to larger dependency trees and not portable to other packages. - Adding
_vendor
tosys.path
viasys.path.insert(1, <path_to_vendor_dir>)
in my package init file. This works, but it introduces a global change to the module look-up path which will affect other add-ons and even the base app itself. It just seems like a hack that could result in a pandora's box of issues later down the line (e.g. conflicts between different versions of the same package, etc.). - Temporarily modifying sys.path for my imports; but this fails to work for third-party modules with method-level imports.
- Writing a PEP302-style custom importer based off an example I found in setuptools, but I just couldn't make head nor tail of that.
I've been stuck on this for quite a few hours now and I'm beginning to think that I'm either completely missing an easy way to do this, or that there is something fundamentally wrong with my entire approach.
Is there no way I can ship a dependency tree of third-party packages with my code, without having to resort to sys.path
hacks or modifying the packages in question?
from Import vendored dependencies in Python package without modifying sys.path or 3rd party packages
No comments:
Post a Comment