[mvapich-discuss] mvapich2 problems on node with active mlx4_0 and nes0 hcas

[Devendar Bureddy]

Good to know that things are working fine.

For everybody's information, we had offline discussion and this issue is
resolved by re-configuring mvapich2 with "--disable-rdma-cm". With this
flag, mvapich2 ignores all IWARP devices completely.

-Devendar


On Fri, Sep 13, 2013 at 12:29 AM, Ron Rechenmacher <ron at fnal.gov> wrote:

> I've rebuilt both 1.9 and 2.0a with the config option (--disable-rdma-cm).
> Once I recompiled my test executable and used the MV2_IBA_HCA=mlx4_0
> variable setting, they both worked!
> Thanks.
> Lynn can now disable Devendar's access.
>
> Thanks again Devendar!
> --Ron
>
> Ron Rechenmacher wrote, On 09/12/13 22:16:
>
>  OK, I'll give it a try.
>> Thanks!
>>
>> Devendar Bureddy wrote, On 09/12/13 22:14:
>>
>>> This issue is causing because of conflicting requirement of start-up
>>> mechanisms in MVAPICH2 for NetEffect and Mellanox adapters. The auto
>>> detection of NetEffect can be completely disabled with "--disable-rdma-cm"
>>> configuration flag. Please note that,
>>> this flag do not turn off any of the MVAPICH2 optimizations. It is safe
>>> to use it.
>>>
>>> [mvapich at dsfr1 mvapich2-2.0a-normdacm]$ install/bin/mpirun_rsh -rsh -np
>>> 2 dsag dseb5 MV2_IBA_HCA=mlx4_0 osu_benchmarks/.libs/osu_bw
>>> # OSU MPI Bandwidth Test
>>> # Size Bandwidth (MB/s)
>>> 1 0.75
>>> 2 1.52
>>> 4 2.92
>>> 8 5.64
>>> 16 11.32
>>> 32 22.52
>>> 64 45.64
>>> 128 85.00
>>> 256 202.77
>>> 512 377.03
>>> 1024 653.54
>>> 2048 998.65
>>> 4096 1424.24
>>> 8192 1604.75
>>> 16384 2588.76
>>> 32768 2726.58
>>> 65536 2698.27
>>> 131072 2849.58
>>> 262144 2873.89
>>> 524288 2888.68
>>> 1048576 2896.58
>>> 2097152 2899.66
>>> 4194304 2908.59
>>>
>>>
>>> -Devendar
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20130913/1a06b0fe/attachment.html