[mvapich-discuss] Dual port HCA back-to-back woes

[Dr. Dmitry Nolde]

Dear Dr. Abhinav

Thank you for very quick and detailed answer. You are right
my configuration was "cross-over". Today I made some tests using 
straight connection. Some tests now work. For example,  osu_bibw 
osu_bw, osu_latency, osu_mbw_mr from perf_test directory run normally.
However I'm not sure that these tests use two ports, because these tests 
give similar result both with VIADEV_USE_MULTIPORT=0 and 
VIADEV_USE_MULTIPORT=1.

But the main problem is that osu_bcast test does not work in dual-port 
mode.
Command "mpirun_rsh -np 4 kewa3 kewa3 kewa4 kewa4 VIADEV_USE_MULTIPORT=1 
osu_bcast" produce the following output:
# OSU MPI_Bcast Latency Test (Version 1.2)
# Size          Latency (us)
[0:kewa3] Abort: [kewa3:0] Got completion with error 
IBV_WC_RETRY_EXC_ERR, code=12
  at line 2374 in file viacheck.c
mpirun_rsh: Abort signaled from [0]
done.

I tried use flag DISABLE_HARDWARE_MCAST but without success.
Two opensmd daemons was started on first node to configure corresponding 
port 1 and 2.

I am using make.mvapich.gen2 configuration. Is this correct or I need to 
use gen2_multirail config.

Thanks in advance


Sincerely yours
Dmitry E. Nolde
Shemyakin-Ovchinnikov Institute of Bioorganic Chemistry,
Moscow Russia