[Mvapich-discuss] RDMA memory registration for AllGather

[Subramoni, Hari]

Hi, Niyaz.

One buffer can be registered with multiple HCAs. There is no problem with this. You just to call ibv_reg_mr multiple times.

Thx,
Hari.

From: Mvapich-discuss <mvapich-discuss-bounces at lists.osu.edu> On Behalf Of Niyaz Murshed via Mvapich-discuss
Sent: Tuesday, August 20, 2024 3:18 PM
To: Niyaz Murshed via Mvapich-discuss <mvapich-discuss at lists.osu.edu>
Cc: nd <nd at arm.com>
Subject: [Mvapich-discuss] RDMA memory registration for AllGather

Hello all , I am trying to understand how a single memory buffer is registered to multiple rdma verbvs context in case there are data coming in from different interfaces. As I understand, we can only register RDMA memory to a single protection
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
    Report Suspicious  <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vwQdMSZND6YBZRdwP87EmWtyyOA7umUZGthn5X3C5R7Es3iK6sJuxVQdaB3IuKM_j2DQQOzGjjDQTj8jA8JgzpaGTj7LSjRc22xjUk3eC03zlVBUOtjbKeVvaDgXqDki5RyQ-5X2tuEHcGA32bjPcQ$>   ‌
ZjQcmQRYFpfptBannerEnd
Hello all ,

I am trying to understand how a single memory buffer is registered to multiple rdma verbvs context in case there are data coming in from different interfaces.
As I understand, we can only register RDMA memory to a single protection domain which is linked to one interface.

Assuming we receive data from 3 ranks on interface1 and 2 ranks from interface 2, how do these interfaces write to same common buffer ?
How is the same buffer registered to 2 interfaces?

Regards,
Niyaz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/mvapich-discuss/attachments/20240821/0709cfcc/attachment-0002.html>