pinned memory

[nam2934]

Hi,
I'm curious to know whether HeavyDB employs pinned memory.
From my perspective, utilizing pinned memory appears to offer advantages in terms of H2D data transfer.
If HeavyDB doesn't use pinned memory, could you kindly share any specific reasons behind this decision?

Thank you.

[cdessanti]

Hi,
In the past, we had a specific option for using pinned memory. However, it was dropped due to issues like memory fragmentation and OOM killer intervention. C
urrently, there is negligible performance difference between pinned and regular memory when using a single GPU. Some serialization issues in the software also render the use of this memory type impractical, particularly when employing multiple GPUs.

Despite this, we are planning to reintroduce the usage of this memory for specific narrow scopes in the near future.

Regards,
Candido

[cdessanti]

@nam2934

I recently re-enabled the Pinned memory usage (along with the Managed memory) while doing some experimentation on GH-200, and I got a performance advantage while accessing the system memory directly from the GPU (that's unexpected on this class of machine), taking advantage of the page fault capability of the hardware.

The throughput I'm achieving when accessing to system memory this way is around 430 GB / sec, out of the theoretical 500 GB of CPU Dram speed. The GPU has a Chip2Chip connection with the CPU rated at 800 GB /sec.