[mvapich-discuss] Install error for mvapich2-0.9.5

[Tang, Changqing]

HI, Here is another question about mvapich-0.9.8 multi-rail on OpenIB.

I have system with two PCI-Express SDR cards, firmware 4.7.0, for the
first card, port 1 is active, port 2 is down, for the second card, port
1 is down, port 2 is active.

I installed OpenFabric 1.0, and installed mvapich-0.9.8 with gen2
multi-rail support.

$ cat hostfile
xcg13
xcg14
$
$ export MPI_ROOT=/scratch/ctang/mvapich-0.9.8
$ $MPI_ROOT/bin/mpicc -g -o pp.x ping_pong.c
$ $MPI_ROOT/bin/mpirun_rsh \
-np 2 -hostfile hostfile \
NUM_HCAS=2 \
NUM_PORTS=1 \
NUM_QP_PER_PORT=1 \
LD_LIBRARY_PATH=/scratch/ctang/mvapich-0.9.8/lib/shared \
./pp.x 1048576
ping-pong 1048576 bytes ...
[0] Abort: [xcg13:0] Got completion with error code 12
 at line 1277 in file viacheck.c
Timeout alarm signaled
Cleaning up all processes ...done.
$ 

What's wrong with me, and can you tell me how to solve this problem ?
Thanks.

--CQ Tang

 

>-----Original Message-----
>From: Sayantan Sur [mailto:surs at cse.ohio-state.edu] 
>Sent: Saturday, September 09, 2006 6:13 PM
>To: Tang, Changqing
>Cc: mvapich-discuss at cse.ohio-state.edu
>Subject: Re: [mvapich-discuss] Install error for mvapich2-0.9.5
>
>Hello,
>
>* On Sep,1 Tang, Changqing<changquing.tang at hp.com> wrote :
>> 
>> HI, I am a HP engineer trying to use your mvapich-0.9.5. I 
>compile the 
>> code on a Opteron system with two PCI-X IB cards, But got the 
>> following error:
>> 
>> 29:2: #error SRQ is not suppported for Mellanox PCI-X cards
>
>Thanks for trying out our latest MVAPICH2-0.9.5 release.
>
>Based on earlier feedback from Christian about the scripts 
>having this error on PCI-X platforms, we have updated our 
>build scripts. You may update (or obtain) an SVN copy either 
>from the MVAPICH2 trunk (main
>development) or the 0.9.5 branch (all major bug fixes to 0.9.5 
>release).
>You may visit the MVAPICH2 download page for more information 
>on trunk and branches `policies'.
>
>> 
>> Based on what information you make this statement ? We can install 
>> both VAPI and OpenIB with SRQ support on the system
>
>We made this statement based on the degraded performance of 
>the SRQ feature on the PCI-X HCAs. You can follow an existing 
>thread discussing the performance issues.
>
>http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2006-S
eptember/000339.html
>
>For the time being Send/Receive mode offers better performance 
>on PCI-X HCAs. Note that for PCI-Express HCAs, by default the 
>SRQ support is used.
>
>Thanks,
>Sayantan.
>
>--
>http://www.cse.ohio-state.edu/~surs
>