[mvapich-discuss] Overlap of communication and computation on the GPU

[Chu, Ching-Hsiang]

Hi, Yussuf,

CUDA kernels work asynchronously from CPU's perspective. MVAPICH2-GDR library does not block the CUDA kernels in any way. So the execution of your kernel should overlap with Isend/Irecv in most cases. If you observe otherwise, please feel free to let us know.

Thanks,

Ching-Hsiang Chu

________________________________
From: mvapich-discuss <mvapich-discuss-bounces at cse.ohio-state.edu> on behalf of Yussuf Ali <yussuf.ali at jaea.go.jp>
Sent: Thursday, May 16, 2019 4:13 AM
To: mvapich-discuss at cse.ohio-state.edu
Subject: [mvapich-discuss] Overlap of communication and computation on the GPU

Dear MVAPICH developers and users,

I have a question regarding communication and computation overlap with
MVAPICH2-GDR.
If we write the following code in our program:

MPI_Irecv(cuBufferA.......);
MPI_Isend(cuBufferB.......)
my_kernel<<<...>>>(cuBufferC,cuBufferD);

All buffers are device buffers and my_kernel can be calculated
independently of cuBufferA and cuBufferB.
Will the GPU automatically overlap Irecv/Isend with the my_kernel
execution?
Or will my_kernel be executed after Irecv/Isend are finished?

Thank you for your help,
Yussuf Ali

_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20190520/68fd9699/attachment.html>