Enable Sparse Krylov for non-Nvidia GPUs

[pengtu]

I have a question on how to enable sparse Krylov for non-Nvidia GPUs, specifically for Intel GPU using oneAPI.jl. The document states that:

1. Sparse matrices with GPU specific storage are supported. I guess that they can be converted to dense GPU matrices during the Krylov operations.

2. Preconditions, especially incomplete Cholesky or incomplete LU factorizations that involve triangular solves can be applied directly on GPU to use CUSPARSE routines.

Is there a list of CUSPASE routines that are needed in order to make sparse Krylov.jl to work on Intel GPUs?

Thanks!

[amontoison]

Hi @pengtu!

1. We need the support of sparse matrix - dense vector products in oneAPI.jl to enable sparse Krylov on Intel GPUs.
   We need to add a oneSparseMatrixCSR format for example in oneAPI.jl and a dispatch for mul!(y::oneVector{T}, A::oneSparseMatrixCSR{T}, x::oneVector{T}) / mul(y, A', x).

2. CUSPARSE has routines to compute IC(0) and ILU(0) preconditioners of CuSparseMatrixCSC / CuSparseMatrixCSR on the device. If Intel proposes similar preconditioners we should interface these functions. If we don't have any GPU accelerated preconditioners on Intel GPUs, we are still able to compute them on the host and after preform the sparse triangular solves on the device (spsv).
   We need to interface ldiv!(y::oneVector{T}, T::Triangular{T, oneSparseMatrixCSR{T}, x::oneVector{T}) in all cases.

[pengtu]

Thank you very much @amontoison for the clear instructions! We will prototyping the required functions and get back to you.

[amontoison]

@pengtu
Do you have some members of your Intel team that participate at ICIAM 2023 next week?
It could be easier to prototype it together.

[pengtu]

@amontoison Do you mean International Congress on Industrial and Applied Mathematics in Tokyo? Unfortunately, we don't have anyone attending this year. It is a good idea to prototype together though. We can set up a collaboration once I have the internal matter settled. I will give you an update after next week.

[amontoison]

Hi @pengtu! Do you have any update?
I'm working at the Argonne National Laboratory this Fall so it will be to great to test Krylov.jl on Intel GPUs.