[mvapich-discuss] problem with mvapich2-1.6 and PGI 11.3

[Blaise Bourdin]

Hi,

I am having difficulties with mvapich2-1.6 compiled with PGI 11.3-0 (most recent release) on a RHEL 5.4 (Rocks 5.3) linux64 box.

I configured mvapich2 using the following options:
./configure --prefix=$MPICH_HOME --with-rdma=gen2 --with-pm=hydra:gforker --enable-fast=O3 --enable-romio --enable-sharedlibs=gcc CC=pgcc F77=pgf77 F90=ifort CXX=pgCC
compiled and install without any problem.

I can compile and run C example, but am not able to link C++ examples. Here is what I get when I try to compile a simple C++ helloworld test:

[bourdin at head ~]$ mpicxx -V

pgCC 11.3-0 64-bit target on x86-64 Linux -tp nehalem 

[bourdin at head ~]$ mpicxx -c hello.c
[bourdin at head ~]$ mpicxx -v -o hello hello.o
mpicxx for MPICH2 version 1.6

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /share/apps/pgi/linux86-64/11.3/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtbegin.o /share/apps/pgi/linux86-64/11.3/lib/initmp.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /share/apps/pgi/linux86-64/11.3/lib/pgi.ld -L/share/apps/mvapich2-1.6-ib-pgi11.3/lib -L/share/apps/mvapich2-1.6-ib-pgi11.3/lib -L/share/apps/pgi/linux86-64/11.3/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 hello.o -rpath /share/apps/mvapich2-1.6-ib-pgi11.3/lib -lmpichcxx -lmpich -lopa -lpthread -lrdmacm -libverbs -libumad -ldl -lrt -rpath /share/apps/pgi/linux86-64/11.3/lib -o hello -lzceh -lgcc_eh --eh-frame-hdr -lstdz -lCz /share/apps/pgi/linux86-64/11.3/lib/nonuma.o -lpgmp -lpthread -lnspgc -lpgc -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtend.o /usr/lib64/crtn.o
/usr/bin/ld: hello: hidden symbol `_Unwind_Resume' in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgcc_eh.a(unwind-dw2.o) is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
pgCC-Fatal-linker completed with exit code 1

For reference, I have no problems with 10.3:
[bourdin at head ~]$ mpicxx -V

pgCC 10.3-0 64-bit target on x86-64 Linux -tp nehalem-64 

[bourdin at head ~]$ mpicxx -c hello.c
[bourdin at head ~]$ mpicxx -v -o hello hello.o
mpicxx for MPICH2 version 1.6

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /share/apps/pgi/linux86-64/10.3/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtbegin.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /share/apps/pgi/linux86-64/10.3/lib/pgi.ld -L/usr/lib64 -L/share/apps/mvapich2-1.6-ib-pgi10.3/lib -L/share/apps/mvapich2-1.6-ib-pgi10.3/lib -L/share/apps/pgi/linux86-64/10.3/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 hello.o -rpath /share/apps/mvapich2-1.6-ib-pgi10.3/lib -lmpichcxx -lmpich -lopa -llimic2 -lpthread -lrdmacm -libverbs -libumad -ldl -lrt -rpath /share/apps/pgi/linux86-64/10.3/lib -o hello -lstd -lC -lnspgc -lpgc -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtend.o /usr/lib64/crtn.o

[bourdin at head ~]$ ldd hello
	libmpichcxx.so.1.2 => /share/apps/mvapich2-1.6-ib-pgi10.3/lib/libmpichcxx.so.1.2 (0x00002ba9b4dfa000)
	libmpich.so.1.2 => /share/apps/mvapich2-1.6-ib-pgi10.3/lib/libmpich.so.1.2 (0x00002ba9b502d000)
	liblimic2.so.0 => /usr/lib64/liblimic2.so.0 (0x00002ba9b54a7000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00000035bda00000)
	librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00000035bde00000)
	libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00000035bd200000)
	libibumad.so.2 => /usr/lib64/libibumad.so.2 (0x00002ba9b56aa000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00000035bd600000)
	librt.so.1 => /lib64/librt.so.1 (0x00000035be200000)
	libm.so.6 => /lib64/libm.so.6 (0x00002ba9b58b1000)
	libc.so.6 => /lib64/libc.so.6 (0x00000035bce00000)
	libpgc.so => /share/apps/pgi/linux86-64/10.3/libso/libpgc.so (0x00002ba9b5b34000)
	/lib64/ld-linux-x86-64.so.2 (0x00000035bca00000)

Any idea?

Blaise


-- 
Department of Mathematics and Center for Computation & Technology
Louisiana State University, Baton Rouge, LA 70803, USA
Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin