[mvapich-discuss] osu_benchmarks linking troubles using pgi/14.4 on machines with cuda disabled

[Burward-Hoy, Jane M]

Hi all,

Some more information.

All configure settings and flags work except --enable-shared:

>./configure --enable-shared
...
> make V=1 > make_output 2>& 1

Making all in osu_benchmarks
make[2]: Entering directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9/osu_benchmarks'
source='osu_alltoallv.c' object='osu_alltoallv.o' libtool=no \
	DEPDIR=.deps depmode=pgcc /bin/sh ../confdb/depcomp \
	pgcc -DHAVE_CONFIG_H -I. -I../src/include   -D_ENABLE_OPENACC_
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpl/include
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpl/include
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/openpa/src
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/openpa/src
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpi/romio/include -I/include -I/include -I/include
-I/include -acc -DNDEBUG -DNVALGRIND -O2 -c -o osu_alltoallv.o
osu_alltoallv.c
/bin/sh ../libtool  --tag=CC   --mode=link pgcc -acc -DNDEBUG -DNVALGRIND
-O2 ../lib/libmpich.la -L/lib -L/lib   -L/lib -Wl,-rpath,/lib -L/lib
-Wl,-rpath,/lib -L/lib -L/lib -o osu_alltoallv osu_alltoallv.o  -libmad
-lrdmacm -libumad -libverbs -lrt -lhwloc -lpthread -lhwloc
libtool: link: pgcc -acc -DNDEBUG -DNVALGRIND -O2 -Wl,-rpath -Wl,/lib
-Wl,-rpath -Wl,/lib -o .libs/osu_alltoallv osu_alltoallv.o
../lib/.libs/libmpich.so -L/lib -L./src/mpl -L./src/openpa/src
/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-conejo
/mvapich2-1.9/src/mpl/.libs/libmpl.so
/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-conejo
/mvapich2-1.9/src/openpa/src/.libs/libopa.so -libmad -lrdmacm -libumad
-libverbs -lrt -lpthread -lhwloc -Wl,-rpath -Wl,/usr/local/lib
nvlink fatal   : Unsupported file type '../lib/.libs/libmpich.so'
pgacclnk: child process exit status 2:
/yellow/usr/projects/hpcsoft/toss2.1/common/pgi/14.4/linux86-64/14.4/bin/pg
nvd
make[2]: *** [osu_alltoallv] Error 2
make[2]: Leaving directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9/osu_benchmarks'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9'
make: *** [all] Error 2


Any ideas?

Jane





On 6/19/14 9:31 AM, "Burward-Hoy, Jane M" <janebh at lanl.gov> wrote:

>Hi all,
>
>The osu_benchmarks builds are failing with a fatal error using pgi/14.4:
>
>make[2]: Entering directory
>`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14
>.
>4-conejo/mvapich2-1.9/osu_benchmarks'
>  CC       osu_alltoallv.o
>  CCLD     osu_alltoallv
>nvlink fatal   : Unsupported file type '../lib/.libs/libmpich.so'
>pgacclnk: child process exit status 2:
>/usr/projects/hpcsoft/toss2.1/common/pgi/14.4/linux86-64/14.4/bin/pgnvd
>
>when it tries to load the built version of either mvapich2 1.9 and 2.0a.
>
>This is only for the mvapich2 builds on some of our older machines where
>cuda is not installed. Everything works as expected for our machines with
>cuda installed.
>
>For the machines without cuda, the default configuration of mvapich2
>installs 
>with no problems (1.9 and 2.0a).  When I add the following compiler flags,
>however, I encounter the above fatal error when trying to compile the
>osu_benchmarks:
>
>./configure --prefix=$prefix --enable-shared --enable-sharedlibs=gcc
>--enable-romio --with-file-system=ufs+nfs+panfs+lustre --enable-g=dbg
>--enable-debuginfo --with-mpe --enable-f77 --enable-fc --enable-cxx
>--with-pm=no --with-pmi=slurm --with-device=ch3:mrail --with-rdma=gen2
>--with-ch3-rank-bits=32 --without-cuda
>
>
>Has anyone else encountered this when trying to build mvapich2 using the
>pgi 
>14.4 compiler (disabling cuda)?
>
>Most appreciative of any help,
>
>Jane
>
>Jane M. Burward-Hoy, PhD
>Programming Environment and Runtime Team
>HPC-3:  High Performance Computing Systems
>Los Alamos National Laboratory
>
>Email:  janebh at lanl.gov
>Office:  (505) 664-0485
>FAX:  (505) 665-6333
>MS:  T080
>Location:  TA-03, Bldg. 2327, Rm. 1134
>
>
>
>
>
>_______________________________________________
>mvapich-discuss mailing list
>mvapich-discuss at cse.ohio-state.edu
>http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>