Linker Symbols get Lost after shared library is moved to site-packages.

[FabianSchuetze]

Thanks for this wonderful project.

For a Python package that I created, the linked dependencies of the shared library are not defined after the shared library is copied into the Python package. The ldd output for the shared library is:

ldd ./venv/lib/python3.11/site-packages/package/bindings.cpython-311-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007fff5a1cd000)
	libkernels.so => not found
	libc10.so => not found
	libtorch_python.so => not found
	...

That is in stark contrast to the shared library inside the original build folder:

ldd ./build/package/cpp/bindings/bindings.cpython-311-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffcd55fa000)
	libkernels.so => /tmp/pacakge/build/package/cpp/kernels/libkernels.so (0x0000711b619d6000)
	libc10.so => /tmp/pacakge/build/_deps/torch-src/lib/libc10.so (0x0000711b618be000)
	...

The CMakeLists file organizing the build is as follows:

add_library(bindings MODULE bindings.cpp)
target_link_libraries(bindings PRIVATE kernels ${TORCH_LIBRARIES})
...
install(TARGETS bindings LIBRARY DESTINATION package)

Torch is obtained via FetchContent, and Kernels is another target of my package. The pyproject.toml file is:

[build-system]
requires = ["scikit-build-core"]
build-backend = "scikit_build_core.build"

...

[tool.scikit-build]
wheel.packages = ["LOCATION"]
build-dir = "build" 

How can I ensure that the linker resolves these symbols? The docs mention issues with dynamic linking, but I don't fully understand the description and am not sure if it is applicable here.

[LecrisUT]

You are probably lacking RPATHs in the library auditwheel can do the bundling and rpath fixing or you could do that manually with some effort.

Basically if the libraries arr not in the system paths /usr/lib, they should have appropriate hints from where to load them like LD_LIBRARY_PATH or RPATH. The latter is the only packageable solution you can provide.

We do have #1009 that aims to simplify this