nvidia-hpc-sdk toolchain extra libs support

[Tomcat-42]

Is your feature request related to a problem? Please describe.

I'm using xmake to build a nvidia-hpc-sdk application. Basically I followed this issue to get It working.

Take a look at the module file of the nvhpc sdk: https://gist.github.com/Tomcat-42/f2256ee49fda99f16e56f2ac44a3674a .

It defines the include dirs at the CPATH env var:

/opt/nvidia/hpc_sdk/Linux_x86_64/2024/math_libs/include
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/mpi/include
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/nccl/include
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/nvshmem/include
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/compilers/extras/qd/include/qd

and the corresponding libs paths at LD_LIBRARY_PATH:

/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda/lib64
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda/extras/CUPTI/lib64
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/compilers/extras/qd/lib
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/compilers/lib
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/math_libs/lib64
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/mpi/lib
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/nccl/lib
/opt/nvidia/hpc_sdk/Linux_x86_64/2024/comm_libs/nvshmem/lib

The issue is: using the toolchain config cuda, I'm not able to include/link these extra libs in my code.

Describe the solution you'd like

When using the nvidia-hpc-sdk toolchain, I would like to add all those libraries automatically to the lib and include path.

Describe alternatives you've considered

I see to clearly distincts possible approaches:

• Modify the cuda toolchain to handle these extra libs.
• Add another toolchain, nvhpc, that would handle this.

Additional context

I'm opened this issue to discuss with you @waruqi the state of the nvidia-hpc-sdk support in xmake, and the possible improvements that could be done.

Later, when we decide the best approach, I will open a PR to implement it.

[waruqi]

you can open a pr to add a nvidia-hpc-sdk package to xmake-repo. and implement on_fetch to find these library files and links/linkdirs.

like this https://github.com/xmake-io/xmake-repo/blob/efaf379de2b94981a33ffe947e30ea5310257b8d/packages/c/cuda/xmake.lua#L16

then you can use it in your project.

add_requires("nvidia-hpc-sdk")

target("test")
    add_files("src/*.cpp")
    add_packages("nvidia-hpc-sdk")

[Tomcat-42]

Thanks for your help @waruqi

Actually, the sdk provides a Cmake config file (It's not a CMakeLists.txt, is just a .cmake file) there is a way to import this file in the "nvidia-hpc-package"?

[waruqi]

see https://xmake.io/#/package/local_package?id=find-packages-from-cmake

But it's not always reliable.

[Tomcat-42]

Like this?

But it's not finding the configs:

Maybe it's beacause the sdk is in the /opt dir?

[waruqi]

So it is unreliable and you need to debug and configure various parameters yourself. I don't have hpc-sdk environment.

you can open a pr to add a nvidia-hpc-sdk package to xmake-repo. and implement on_fetch to find these library files and links/linkdirs.

like this https://github.com/xmake-io/xmake-repo/blob/efaf379de2b94981a33ffe947e30ea5310257b8d/packages/c/cuda/xmake.lua#L16

then you can use it in your project.

add_requires("nvidia-hpc-sdk")

target("test")
    add_files("src/*.cpp")
    add_packages("nvidia-hpc-sdk")

I would recommend this option even more.