[mvapich-discuss] mvapich 1.1 and ofed 1.3.1

[Pawel Dziekonski]

On Mon, 09 Mar 2009 at 12:37:50PM -0400, Jonathan Perkins wrote:
> Pawel:
> My reply is inline below.
> 
> On Mon, Mar 09, 2009 at 04:02:13PM +0100, Pawel Dziekonski wrote:
> > 
> > hello,
> > 
> > I've just compiled mvapich 1.1 against ofed 1.3.1 and intel compilers on intel
> > em64t. IB net is based on mthca0's and Voltaire. script and logs are here:
> > https://cefeid.wcss.wroc.pl/d/tmp/mvapich-1.1/
> > 
> > my changes to make.mvapich.gen2 script:
> > 
> > diff mvapich-1.1/make.mvapich.gen2 MVAPICH-1.1/make.mvapich.gen2
> > 84c84
> > < export LIBS=${LIBS:--L${IBHOME_LIB} -Wl,-rpath=${IBHOME_LIB} -libverbs -libumad -lpthread}
> > ---
> > > export LIBS="${IBHOME_LIB}/libibverbs.a ${IBHOME_LIB}/libibumad.a ${IBHOME_LIB}/libibcommon.a -lpthread"
> > 
> > as you see, I wanted to have IB and MPI libs statically linked in.
> 
> Instead of providing the paths to the static libraries in this manner
> can you try just replacing the -Wl,rpath=${IBHOME_LIB} portion with
> -static?  Let us know if this solves your problem.

In this way it will link everything static, including libc, and fail. Instead
I used:

export LIBS="-L/usr/lib64 -Wl,-Bstatic -libverbs -libumad -libcommon -Wl,-Bdynamic -lpthread"

and it works, however test job still fails with same error. Maybe this is an OFED design?

regards, Pawel

-- 
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND
phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl