[mvapich-discuss] Regarding transport interfaces of MVAPICH2

[Hari Subramoni]

Hi,

I apologize for the delay in response.

Both OFA-IB-Nemesis & OFA-IB-CH3 are based on libibverbs layer in OFED and
depend on Subnet manager/Subnet administration components in OFED. In
particular, both these interaces need OpenSM to configure the LID's on all
interfaces as well as the many switch forwarding tables etc. In short,
these interfaces won't work without OpenSM.

RDMA_CM is one of the many ways which the OFA-IB-CH3 interface uses to
establish conections. We are working on integrating this support
into the OFA-IB-Nemesis interface and will be available in one of the
future releases.

The OFA-IB-Nemesis & OFA-IB-CH3 relies in TCP/IP based transport
mechanisms to exchange remote QP/LID/GID information.

I hope I answered all your questions. Please let us know if you need more
information on any of these points.

Thx,
Hari.

On Wed, 1 Jun 2011, Bhargava Ramu Kavati wrote:

> Hi,
> I am trying to run MPI applications on an OFED cluster (over IB) using
> MVAPICH2 with OFA-IB-Nemesis as underlying transport interface.  I have
> below queries regarding OFA-IB-Nemesis/OFA-IB-CH3, please clarify.
>
> 1) *OFA-IB-Nemesis:* It is based on libibverbs layer in OFED and does not
> use connection manager in OFED.
>     *OFA-IB-CH3:* It supports librdmacm in OFED and hence relies on
> connection manager.
>     Is my understanding correct in both the above cases?
>
> 2) Does OFA-IB-Nemesis depend on Subnet manager/Subnet administration
> components in OFED ? I mean, does OFA-IB-Nemesis require any services from
> Subnet manager/Subnet administration ?
>
> 3) How a MPI app which is using MVAPICH2 with OFA-IB-Nemesis gets the remote
> node details such as LID/GID, QP number etc. to connect  to remote node? Is
> it via out of band communication?
>
> Thank you in advance.
>
> Thanks & Regards,
> Ramu
>