[mvapich-discuss]Infiniband - how do I know that it is working?

[teng ma]

2 processes on 2 nodes (2*1) maybe too little to reach the communication
upbound. If your nodes are multicore nodes,  you can spawn 16nodes * some
number/node  MPI processes. These traffic maybe generate some performance
difference between ib and ethernet.

Teng
On Thu, May 31, 2012 at 12:16 PM, Ju JiaJia <jujj603 at gmail.com> wrote:

> I have run osu_latency and osu_bw with both mvapich2 and mpich2 on two
> nodes, here is my test result
>
>  MVAPICH2(Infiniband）MPICH2（Ethernet）0 1.4515.16 11.4915.25 21.49 15.154
> 1.49 15.158 1.515.1716 1.5315.17 321.57 16.4641.66 16.88128 1.8317.8256
> 2.920.39 5123.14 26.061K3.8 47.52K 4.89193.974K 5.99228.97 8K7.85 233.12
> 16K12.51 241.3332K 18.08432.31 64K28.7675.25 128K50.02 1429.21256K 92.57
> 2582.83 512K177.654693.17 1M347.75 9110.472M688.12 17942.074M 1373.9
> 35609.21
>
> Here  is HPL results on two nodes:
> HPL.out.ethernet.211:   WR00L2C2       20000   128     2     1
> *1686.40*              3.163e+00
> HPL.out.ib.211:            WR00L2C2       20000   128     2     1
>   * 1639.99 *             3.252e+00
>
> run script :
> export MV2_ENABLE_AFFINITY=0
> PATH=/gos4/user39/jujj/program-files/mvapich2/bin:$PATH
>
> #
> mpiexec -f mfile -n 2 taskset -c 0 ./xhpl > HPL.out.ib.211 2>&1
> mpiexec -f mfile -n 2 taskset -c 0 ./xhpl_ethernet > HPL.out.ethernet.211
> 2>&1
>
> As you can see, no big difference. I tested NAMD also, no big difference.
> Is there any tools like netstat over infiniband ? So i can see whether
> connections is built or which network is being used. Or MVAPICH2 support
> some ways to do this? Log or something?
>
> On Thu, May 31, 2012 at 11:43 AM, Dhabaleswar Panda <
> panda at cse.ohio-state.edu> wrote:
>
>> After installing MVAPICH2, you can run OSU MPI Micro-Benchmarks to verify
>> that your installtion is correct. You can verify your performance with the
>> performance numbers/graphs available at the MVAPICH site. After that you
>> can carry out your applications-level study.
>>
>> DK
>>
>> On Thu, 31 May 2012, Ju JiaJia wrote:
>>
>> > Hi all:
>> >
>> > I am currently running HPL built using mvapich2 which use infiniband,
>> > but it shows no apparently difference, compares in performance to a
>> > build using mpich2 which uses Ethenet.
>> >
>> > There should be some improvement in performance. I doubt whether
>> > infiniband is working. Anyone knows how to check whether infiniband is
>> > working ?
>> >
>> >
>> > Ju JiaJia
>> >
>>
>>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20120601/c19071ba/attachment.html