Monday 31 October 2022

Importing Numpy fails after building from source against amd blis

I'm trying to build a local version of Numpy from source against BLIS (for BLAS and CBLAS) and against OpenBLAS for LAPACK.

I started with building BLIS locally for zen3 with CBLAS enabled, like so:

./configure --enable-threading=openmp --enable-cblas --prefix=$HOME/blis zen3

then ran the tests (which all passed) and ran make install. I made sure all relevant files are in the $HOME/blis library (see attached screenshot).

I also built openBLAS locally, no special configs there.

Afterwards, I modified numpy's site.cfg to configure openBLAS and blis folders' accordingly:

[blis]
libraries = blis
library_dirs = /home/or/blis/lib/
include_dirs = /home/or/blis/include/blis
runtime_library_dirs = /home/or/blis/lib/

[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib

I continued by building and installing numpy with:

NPY_BLAS_ORDER=blis NPY_LAPACK_ORDER=openblas NPY_CBLAS_LIBS= python ./numpy/setup.py build -j 32

Note that NPY_CBLAS_LIBS is empty as numpy's build docs say to do so if CBLAS is included in the BLIS library, which it is.

Then, importing numpy resulting in: Original error was: /home/or/.pyenv/versions/3.9.6/lib/python3.9/site-packages/numpy-1.24.0.dev0+998.g6a5086c9b-py3.9-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm

I'm clueless at this point as I couldn't find anything online about this specific case.

Installing numpy from pip (which comes built with openblas) can be imported successfully.

Update 1: While reading make install logs, I found out that it couldn't find my BLIS library files at the location, even though the files are in the specified path. I also tried to recompile and install BLIS in various paths and reconfigure numpy before compiling it, but got the same result.

When I downloaded a pre-compiled version of BLIS from AMD's website, numpy seems to get it, but this isn't the recommended way to go because I'm missing optimizations for Zen3.



from Importing Numpy fails after building from source against amd blis

No comments:

Post a Comment