[mvapich-discuss] mvapich 1.1 and ofed 1.3.1

[Pawel Dziekonski]

hello,

I've just compiled mvapich 1.1 against ofed 1.3.1 and intel compilers on intel
em64t. IB net is based on mthca0's and Voltaire. script and logs are here:
https://cefeid.wcss.wroc.pl/d/tmp/mvapich-1.1/

my changes to make.mvapich.gen2 script:

diff mvapich-1.1/make.mvapich.gen2 MVAPICH-1.1/make.mvapich.gen2
18,24c18,24
< IBHOME=${IBHOME:-/usr/local/ofed}
< IBHOME_LIB=${IBHOME_LIB:-/usr/local/ofed/lib64}
< PREFIX=${PREFIX:-/usr/local/mvapich}
< export CC=${CC:-gcc}
< export CXX=${CXX:-g++}
< export F77=${F77:-g77}
< export F90=${F90:-}
---
> IBHOME=${IBHOME:-/usr}
> IBHOME_LIB=${IBHOME_LIB:-/usr/lib64}
> PREFIX=${PREFIX:-/usr/local/MVAPICH-1.1}
> export CC=icc
> export CXX=icc
> export F77=ifort
> export F90=ifort
59c59
<     ROMIO="--with-romio"
---
>     ROMIO="--with-romio --with-file-system=lustre+nfs"
84c84
< export LIBS=${LIBS:--L${IBHOME_LIB} -Wl,-rpath=${IBHOME_LIB} -libverbs -libumad -lpthread}
---
> export LIBS="${IBHOME_LIB}/libibverbs.a ${IBHOME_LIB}/libibumad.a ${IBHOME_LIB}/libibcommon.a -lpthread"

as you see, I wanted to have IB and MPI libs statically linked in.

however hello fails with a non-obvious error (for me ;)

/usr/local/MVAPICH-1.1/bin/mpirun -np 2 -machinefile hello.mpi.hosts ./hello.mpi.x.mv11
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
libibverbs: Warning: no userspace device-specific driver found for /sys/class/infiniband_verbs/uverbs0
forrtl: severe (174): SIGSEGV, segmentation fault occurred
MPI process terminated unexpectedly
Exit code -5 signaled from wn001
Killing remote processes...forrtl: severe (174): SIGSEGV, segmentation fault occurred
MPI process terminated unexpectedly
DONE


any hints?
thanks in advance, Pawel
-- 
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND
phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl