[Mvapich-discuss] Mpich over RDMA sample

[Subramoni, Hari]

Hi, Niyaz.

Are you using MPICH or MVAPICH? This is the MVAPICH mailing list. Not the MPICH list. For issues related to MPICH, please contact the MPICH developers.

Best,
Hari.

From: Mvapich-discuss <mvapich-discuss-bounces at lists.osu.edu> On Behalf Of Niyaz Murshed via Mvapich-discuss
Sent: Tuesday, June 11, 2024 6:54 PM
To: mvapich-discuss at lists.osu.edu
Cc: nd <nd at arm.com>
Subject: [Mvapich-discuss] Mpich over RDMA sample

Hello, I am trying to learn about MPICH and its performance over RDMA. I am using libfabric and installed mpich using the below configure. ./configure --prefix=/opt/mpich/ --with-ofi=/opt/libfabric/ When I run any applications between 2 directly
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
    Report Suspicious  <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vwQdMSZND6YBZRdwP87EmcuTKOH7uspXetcIRVNMBVDirJQ_WGsXmRvHQlK-rHP4x-ctNgz6bAsf0R9wB8SOHaGARJWi_y-3WMl0J4AGU4E4sIZZkShZhsYgHQNj3XycR27EKK0AZZOwIVSKDskkUg$>   ‌
ZjQcmQRYFpfptBannerEnd
Hello,

I am trying to learn about MPICH and its performance over RDMA.
I am using libfabric and installed mpich using the below configure.


./configure --prefix=/opt/mpich/  --with-ofi=/opt/libfabric/

When I run any applications between 2 directly connected servers having Mellanox NICs, I see that communication is happening over tcp and not over RoCE.
Is there any way to test commination over RoCE ?

For eg. I was able to test it for libfabric using the below sample that comes along with libfabric to test RMA.
Is there something similar for MPICH ? or use the current sample to use RoCE by some parameter?

Server :
fi_rma_bw -s   192.168.1.100  -e msg   -d mlx5_1 -S 1024 -I 1

Client :
fi_rma_bw -s   192.168.1.200  -e msg   -d mlx5_3  192.168.1.100  -S 1024 -I 1


Regards,
Niyaz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/mvapich-discuss/attachments/20240612/b005b68f/attachment-0002.html>