[mvapich-discuss] MPI-2 over ibverbs/rdma

[Vivek Satpute]

Hi Jonathan,



On Wed, Nov 25, 2009 at 8:21 PM, Jonathan Perkins <
perkinjo at cse.ohio-state.edu> wrote:

> On Wed, Nov 25, 2009 at 02:04:13PM +0530, Vivek Satpute wrote:
> > Hi,
> >
> > First of all thanks for the instant reply.
> >
> > I am bit confused with IPoIB and MPI-2. Please find inline
> reply/questions .
> >
> > On Wed, Nov 25, 2009 at 1:38 PM, Dhabaleswar Panda wrote:
> > > You can use the OpenFabrics-Gen2/IB interface of MVAPICH2 to run it
> > > directly over ibverbs.
> >
> > If  I configured above then for executing MPI application from node1
> > to node2, node3 and node4 (e.g mpirun_rsh -np 4 n0 n1 n2 n3 ./cpi), I
> > will have to mention a IB interfaces of each n0, n1, n3, n4. So
> > ultimately MPI will use IPoIB module. Am i right ?
>
> You shouldn't need to specify any additional information.  The
> mpirun_rsh command only needs to know which nodes you're trying to
> launch mpi program on.  It appears that you're basing your questions on
> some additional assumptions.  Are you asking this question based on some
> problem that you're run into?
>
>  Yes, you are right. :-) I think I should explain my problem and
assumptions here :-)

I am working on OFED-1.4, it has many stuffs related to Infiniband. I have
installed OFED
on Infiniband hardwares, and IB devices are active and they are working
similar to normal
ethernet devices. Means I can ping/ssh to remote machine through IB devices.

Whenever I send ping packets or do ssh to remote machine, all traffic goes
through
IPoIB module, mean it uses IPoIB code path.

My understanding/assumption is, Infiniband also work without using IPoIB.
So, I want to find out application/utility which work without IPoIB module.
All the traffic generated by such xyz application should go to remote
machine without IPoIB so
for achieving this I can use LID/GID of IB devices.


Is there any such technology/utility/application exists which work without
IPoIB ?
Is there any packet path which goes to remote node without passing through
IPoIB ?

My this assumption can be wrong so please correct me if I am wrong ..



[  As per my understanding libibverbs are below the IPoIB. So applications
(examples of
   libibvers in OFED-1.4) should work on LID/GID, but those applications
also uses IP address
   means ultimately packets of those application also passes through IPoIB
module. ]

Thanks,
Vivek

MPI is an API that you write your software against.  You don't need to
> worry about what the underlying transport is (thus it doesn't matter if
> IPoIB is used or not).  When you build mvapich2 with its default options
> it will know to use IB communications and thus will try to use the first
> working IB interface on each of the nodes that you run the MPI
> application on.  The only reason you should need to specify the
> interface you want to use is when you have a special configuration where
> the default behaviour is not desirable.
>
> >
> > Please refer to MVAPICH2 1.4 user guide (available from the following
> > > URL). This indicates how to build, configure and run MVAPICH2 on Gen2
> > > interface.
> > >
> > > http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.4.html
> > >
> > > Hope this helps.
> > >
> > > DK
> > >
>
> --
> Jonathan Perkins
> http://www.cse.ohio-state.edu/~perkinjo<http://www.cse.ohio-state.edu/%7Eperkinjo>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20091125/cc9ca76c/attachment.html