Thursday, 27 August 2020

How to replace usages of deprecated imp.load_dynamic?

The imp module is deprecated (since version 3.4), however some parts of the infrastructure (e.g. pyximport) still use imp.load_dynamic, which leads to deprecation-warnings with newer Python versions.

Internally, imp.load_dynamic uses importlib-machinery:

from importlib._bootstrap import _load
def load_dynamic(name, path, file=None):
    """**DEPRECATED**
    Load an extension module.
    """
    import importlib.machinery
    loader = importlib.machinery.ExtensionFileLoader(name, path)

    # Issue #24748: Skip the sys.modules check in _load_module_shim;
    # always load new extension
    spec = importlib.machinery.ModuleSpec(
        name=name, loader=loader, origin=path)
    return _load(spec)

But it feels silly to duplicate this code (which at least once had to be improved) for all kind of projects.

importlib's documentation proposes the following:

import importlib.util
import sys
def alternative_load_dynamic(name, path, file=None):
    spec = importlib.util.spec_from_file_location(name, path)
    module = importlib.util.module_from_spec(spec)
    sys.modules[name] = module
    spec.loader.exec_module(module)
    return sys.modules[name]

While it has the advantage of not using the private API compared to the first implementation (i.e. _load) and gets the behavior at the core of issue #24748 right, it is not a drop-in replacement for imp.load_dynamic:

  • this alternative can more: it can also load pure python modules (*.py or *.pyc).
  • this alternative can less: it can only load dynamic modules with ending *.so (or *.pyd on Windows), but not for example foo.my_ending.

What are other options to replace imp.load_dynamic?



from How to replace usages of deprecated imp.load_dynamic?

No comments:

Post a Comment