[mvapich-discuss] Regarding transport interfaces of MVAPICH2

[Bhargava Ramu Kavati]

Hi Hari,
Thank you for your response.  I have below queries related your response.

i) Subnet manager/Subnet administration are anyway needed for OFED Fabric to
become up and running.  In other words, subnet manager/subnet administration
are required irrespective of application running in the app layer.  Apart
from this, as an application does MPI app (using OFA-IB-Nemesis) call any
functions defined by subnet manager/subnet administration components? Plz
correct me if I am wrong.
ii) Considering the fact that OFA-IB-Nemesis uses TCP/IP to exchange remote
node details, how does a MPI app (using OFA-IB-Nemesis) establishes
connection to remote node, is it by using modify QP verb (like pingpong app
examples of libibverbs do) ?

Thanks & Regards,
Ramu

On Sun, Jun 5, 2011 at 8:58 PM, Hari Subramoni
<subramon at cse.ohio-state.edu>wrote:

> Hi,
>
> I apologize for the delay in response.
>
> Both OFA-IB-Nemesis & OFA-IB-CH3 are based on libibverbs layer in OFED and
> depend on Subnet manager/Subnet administration components in OFED. In
> particular, both these interaces need OpenSM to configure the LID's on all
> interfaces as well as the many switch forwarding tables etc. In short,
> these interfaces won't work without OpenSM.
>
> RDMA_CM is one of the many ways which the OFA-IB-CH3 interface uses to
> establish conections. We are working on integrating this support
> into the OFA-IB-Nemesis interface and will be available in one of the
> future releases.
>
> The OFA-IB-Nemesis & OFA-IB-CH3 relies in TCP/IP based transport
> mechanisms to exchange remote QP/LID/GID information.
>
> I hope I answered all your questions. Please let us know if you need more
> information on any of these points.
>
> Thx,
> Hari.
>
> On Wed, 1 Jun 2011, Bhargava Ramu Kavati wrote:
>
> > Hi,
> > I am trying to run MPI applications on an OFED cluster (over IB) using
> > MVAPICH2 with OFA-IB-Nemesis as underlying transport interface.  I have
> > below queries regarding OFA-IB-Nemesis/OFA-IB-CH3, please clarify.
> >
> > 1) *OFA-IB-Nemesis:* It is based on libibverbs layer in OFED and does not
> > use connection manager in OFED.
> >     *OFA-IB-CH3:* It supports librdmacm in OFED and hence relies on
> > connection manager.
> >     Is my understanding correct in both the above cases?
> >
> > 2) Does OFA-IB-Nemesis depend on Subnet manager/Subnet administration
> > components in OFED ? I mean, does OFA-IB-Nemesis require any services
> from
> > Subnet manager/Subnet administration ?
> >
> > 3) How a MPI app which is using MVAPICH2 with OFA-IB-Nemesis gets the
> remote
> > node details such as LID/GID, QP number etc. to connect  to remote node?
> Is
> > it via out of band communication?
> >
> > Thank you in advance.
> >
> > Thanks & Regards,
> > Ramu
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20110606/3d283cc6/attachment.html