[mvapich-discuss] Verify the application is really running over IB

[Jeff Squyres]

In addition to what Dr. Panda said, Cisco recommends that all HPC  
customers upgrade to the OFED IB driver stack if possible (some  
customers cannot upgrade for various reasons).  FWIW: all new HPC/MPI  
work is occurring in the OFED arena.

I bring this up because you specifically mention Topspin Infiniband,  
which I'm *assuming* is the Cisco IB stack (not the OFED IB stack),  
and is therefore shipping with a somewhat older version of MVAPICH  
that was derived from the OSU MVAPICH.  The Cisco MVAPICH should only  
be compiled with IB support enabled; a simple latency test should  
prove that you're running over IB and not ethernet.

Much more recent versions of MPI implementations are included with  
the OFED stack (Cisco provides binary distributions of OFED on  
www.cisco.com).


On Aug 29, 2007, at 11:44 AM, Dhabaleswar Panda wrote:

>
>
> On Wed, 29 Aug 2007 wgy at altair.com.cn wrote:
>
>> Hello, list:
>> It might be a silly questions but I wonder how to verify run with  
>> mvapich
>> (come with Topspin Infiniband) over Infiniband, NOT Gigabite network.
>> Is there an option to force mvapich to use IB network otherwise  
>> just exits?
>
> MVAPICH has several underlying interfaces: Gen2, uDAPL, VAPI, TCP/ 
> IP and
> shared memory. Please take a look at the user guide (available from
> mvapich project page) to see the differences and capabilities of these
> interfaces. Gen2 interface (corresponding to OFED) will give you
> the best performance and scalability. If you have OFED stack  
> installed,
> you should be able to configure mvapich to run over Gen2 interface
> (as per the instructions indicated in the user guide). During OFED
> installation, you can also select mvapich from the package.
>
> On your existing installation, you can also run OSU benchmarks (such
> as OSU latency). If you get latency number in the range of 2~4  
> microsec
> for short messages (say 4 bytes), it is already running over the  
> native
> IB.
>
> Hope this helps.
>
> DK
>
>> Thanks for your suggestion.
>> Rdgs.
>> Henry, Wu
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss


-- 
Jeff Squyres
Cisco Systems