[mvapich-discuss] osu_benchmarks linking troubles using pgi/14.4 on machines with cuda disabled

Mon Jun 23 12:52:54 EDT 2014

Hi Jonathan,

Yes – this worked!

Many thanks,

Jane
--

Jane M. Burward-Hoy, PhD
Programming Environment and Runtime Team
HPC-3:  High Performance Computing Systems
Los Alamos National Laboratory

Email:  janebh at lanl.gov
Office:  (505) 664-0485
FAX:  (505) 665-6333
MS:  T080
Location:  TA-03, Bldg. 2327, Rm. 1134

From: Jonathan Perkins <perkinjo at cse.ohio-state.edu<mailto:perkinjo at cse.ohio-state.edu>>
Date: Friday, June 20, 2014 7:13 PM
To: Jane Burward-Hoy <janebh at lanl.gov<mailto:janebh at lanl.gov>>
Cc: "mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>" <mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>>
Subject: Re: [mvapich-discuss] osu_benchmarks linking troubles using pgi/14.4 on machines with cuda disabled

Hi.  Sorry for the delayed response.  This may be able to be solved by adding ``-ta=tesla:nordc'' to your CFLAGS.  Can you try this out and see if it helps?

On Fri, Jun 20, 2014 at 9:04 PM, Burward-Hoy, Jane M <janebh at lanl.gov<mailto:janebh at lanl.gov>> wrote:
Hi all,

Some more information.

All configure settings and flags work except --enable-shared:

>./configure --enable-shared
...
> make V=1 > make_output 2>& 1

Making all in osu_benchmarks
make[2]: Entering directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9/osu_benchmarks'
source='osu_alltoallv.c' object='osu_alltoallv.o' libtool=no \
        DEPDIR=.deps depmode=pgcc /bin/sh ../confdb/depcomp \
        pgcc -DHAVE_CONFIG_H -I. -I../src/include   -D_ENABLE_OPENACC_
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpl/include
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpl/include
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/openpa/src
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/openpa/src
-I/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-cone
jo/mvapich2-1.9/src/mpi/romio/include -I/include -I/include -I/include
-I/include -acc -DNDEBUG -DNVALGRIND -O2 -c -o osu_alltoallv.o
osu_alltoallv.c
/bin/sh ../libtool  --tag=CC   --mode=link pgcc -acc -DNDEBUG -DNVALGRIND
-O2 ../lib/libmpich.la<http://libmpich.la> -L/lib -L/lib   -L/lib -Wl,-rpath,/lib -L/lib
-Wl,-rpath,/lib -L/lib -L/lib -o osu_alltoallv osu_alltoallv.o  -libmad
-lrdmacm -libumad -libverbs -lrt -lhwloc -lpthread -lhwloc
libtool: link: pgcc -acc -DNDEBUG -DNVALGRIND -O2 -Wl,-rpath -Wl,/lib
-Wl,-rpath -Wl,/lib -o .libs/osu_alltoallv osu_alltoallv.o
../lib/.libs/libmpich.so -L/lib -L./src/mpl -L./src/openpa/src
/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-conejo
/mvapich2-1.9/src/mpl/.libs/libmpl.so
/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.4-conejo
/mvapich2-1.9/src/openpa/src/.libs/libopa.so -libmad -lrdmacm -libumad
-libverbs -lrt -lpthread -lhwloc -Wl,-rpath -Wl,/usr/local/lib
nvlink fatal   : Unsupported file type '../lib/.libs/libmpich.so'
pgacclnk: child process exit status 2:
/yellow/usr/projects/hpcsoft/toss2.1/common/pgi/14.4/linux86-64/14.4/bin/pg
nvd
make[2]: *** [osu_alltoallv] Error 2
make[2]: Leaving directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9/osu_benchmarks'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14.
4-conejo/mvapich2-1.9'
make: *** [all] Error 2

Any ideas?

Jane

On 6/19/14 9:31 AM, "Burward-Hoy, Jane M" <janebh at lanl.gov<mailto:janebh at lanl.gov>> wrote:

>Hi all,
>
>The osu_benchmarks builds are failing with a fatal error using pgi/14.4:
>
>make[2]: Entering directory
>`/yellow/usr/projects/hpctools/janebh/hpcsoft/mvapich2/mvapich2-1.9-pgi-14
>.
>4-conejo/mvapich2-1.9/osu_benchmarks'
>  CC       osu_alltoallv.o
>  CCLD     osu_alltoallv
>nvlink fatal   : Unsupported file type '../lib/.libs/libmpich.so'
>pgacclnk: child process exit status 2:
>/usr/projects/hpcsoft/toss2.1/common/pgi/14.4/linux86-64/14.4/bin/pgnvd
>
>when it tries to load the built version of either mvapich2 1.9 and 2.0a.
>
>This is only for the mvapich2 builds on some of our older machines where
>cuda is not installed. Everything works as expected for our machines with
>cuda installed.
>
>For the machines without cuda, the default configuration of mvapich2
>installs
>with no problems (1.9 and 2.0a).  When I add the following compiler flags,
>however, I encounter the above fatal error when trying to compile the
>osu_benchmarks:
>
>./configure --prefix=$prefix --enable-shared --enable-sharedlibs=gcc
>--enable-romio --with-file-system=ufs+nfs+panfs+lustre --enable-g=dbg
>--enable-debuginfo --with-mpe --enable-f77 --enable-fc --enable-cxx
>--with-pm=no --with-pmi=slurm --with-device=ch3:mrail --with-rdma=gen2
>--with-ch3-rank-bits=32 --without-cuda
>
>
>Has anyone else encountered this when trying to build mvapich2 using the
>pgi
>14.4 compiler (disabling cuda)?
>
>Most appreciative of any help,
>
>Jane
>
>Jane M. Burward-Hoy, PhD
>Programming Environment and Runtime Team
>HPC-3:  High Performance Computing Systems
>Los Alamos National Laboratory
>
>Email:  janebh at lanl.gov<mailto:janebh at lanl.gov>
>Office:  (505) 664-0485<tel:%28505%29%20664-0485>
>FAX:  (505) 665-6333<tel:%28505%29%20665-6333>
>MS:  T080
>Location:  TA-03, Bldg. 2327, Rm. 1134
>
>
>
>
>
>_______________________________________________
>mvapich-discuss mailing list
>mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>
>http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>

_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss

--
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20140623/7e28b17f/attachment.html>