Skip to content

Set FORCE_RPATH for ROCm - #34

Merged
jithunnair-amd merged 1 commit into
mainfrom
force_rpath_for_rocm
Aug 4, 2023
Merged

jithunnair-amd merged 1 commit into
mainfrom
force_rpath_for_rocm

Conversation

@jithunnair-amd

Copy link
Copy Markdown
Collaborator

Tested successfully using wheels generated from: http://rocmhead.amd.com:8080/job/pytorch/job/dev/job/manylinux_rocm_wheels/241/

Successfully set RPATH for libs in wheel:

jenkins@f789cee5d847:/data$ pip install --force-reinstall torch-2.1.0.dev20230803+rocm5.6-cp38-cp38-linux_x86_64.whl
Processing ./torch-2.1.0.dev20230803+rocm5.6-cp38-cp38-linux_x86_64.whl
...
Successfully installed MarkupSafe-2.1.3 filelock-3.12.2 fsspec-2023.6.0 jinja2-3.1.2 mpmath-1.3.0 networkx-3.1 sympy-1.12 torch-2.1.0.dev20230803+rocm5.6 typing-extensions-4.7.1
jenkins@f789cee5d847:/data$ echo $LD_LIBRARY_PATH
...
jenkins@f789cee5d847:/data$ cd /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib
jenkins@f789cee5d847:/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib$ readelf -a libtorch_hip.so | grep PATH    
0x000000000000000f (RPATH)              Library rpath: [$ORIGIN]

Setting LD_LIBRARY_PATH has no impact, still picking up all libs from /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib

jenkins@f789cee5d847:/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib$ export LD_LIBRARY_PATH=/opt/rocm/lib
jenkins@f789cee5d847:/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib$ ldd libtorch_hip.so
        linux-vdso.so.1 (0x00007ffd1fdbf000)
        libc10_hip.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libc10_hip.so (0x00007f81b7896000)
        libamdhip64.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libamdhip64.so (0x00007f81b606a000)
        libMIOpen.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libMIOpen.so (0x00007f8199630000)
        libroctx64.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libroctx64.so (0x00007f819942e000)
        libhipblas.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhipblas.so (0x00007f81991ac000)
        libhipfft.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhipfft.so (0x00007f819919b000)
        libhiprand.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhiprand.so (0x00007f8199193000)
        libhipsparse.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhipsparse.so (0x00007f8198f4c000)
        libhipsolver.so.0 => /opt/rocm/lib/libhipsolver.so.0 (0x00007f8198f0b000)
        librccl.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librccl.so (0x00007f818836f000)
        libc10.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libc10.so (0x00007f8188295000)
        libtorch_cpu.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libtorch_cpu.so (0x00007f8170548000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8170512000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f81703c3000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f81703b9000)
        libmagma.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libmagma.so (0x00007f8150538000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8150356000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8150339000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8150147000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f81e379e000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8150141000)
        libamd_comgr.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libamd_comgr.so (0x00007f8147a08000)
        libhsa-runtime64.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhsa-runtime64.so (0x00007f814754d000)
        libnuma.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libnuma.so (0x00007f8147340000)
        librocblas.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocblas.so (0x00007f8130265000)
        librocsolver.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocsolver.so (0x00007f80e376f000)
        librocfft.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocfft.so (0x00007f80e303a000)
        librocrand.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocrand.so (0x00007f80e0490000)
        librocsparse.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocsparse.so (0x00007f80ae85c000)
        librocm_smi64.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./librocm_smi64.so (0x00007f80ae5b0000)
        libgomp.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libgomp.so (0x00007f80ae386000)
        libroctracer64.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libroctracer64.so (0x00007f80ae129000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f80ae10d000)
        libtinfo.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libtinfo.so (0x00007f80adee1000)
        libelf.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libelf.so (0x00007f80adcc6000)
        libdrm.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libdrm.so (0x00007f80adcb0000)
        libdrm_amdgpu.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libdrm_amdgpu.so (0x00007f80adca3000)
        libhiprtc.so => /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/./libhiprtc.so (0x00007f80ad9db000)

@jithunnair-amd
jithunnair-amd merged commit 3fe2e34 into main Aug 4, 2023
jithunnair-amd added a commit that referenced this pull request Aug 4, 2023
(cherry picked from commit 3fe2e34)
jithunnair-amd added a commit that referenced this pull request Aug 4, 2023
(cherry picked from commit 3fe2e34)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant