[mvapich-discuss] MVAPICH2 1.9b build is broken with Clang

Wed Apr 10 18:15:13 EDT 2013

On Wed, Apr 10, 2013 at 8:24 PM, Jonathan Perkins
<perkinjo at cse.ohio-state.edu> wrote:
> Hello Dmitri, thanks for the report.  I haven't been able to try with
> version 3.2 of Clang yet but I've seen that there is an issue with an
> older version when building with --enable-shared.  I believe that this
> is the same issue you're facing.
>
> Basically with clang it appears that a utility library (mpl) is not
> being built with -fPIC.  The error actually shows up before reaching cpi
> or osu_benchmarks.  I am still trying to determine the exact issue with
> building mpl and will try to prepare a proper.  In the meantime you can
> try removing the --enable-shared and --enable-sharedlibs=gcc options if
> you are not particular about using a static or shared build of MVAPICH2.

Hello Jonathan,

Thank you for your reply.

Unfortunately, removing --enable-shared and --enable-sharedlibs=gcc
does not fix the issue.  Configuring with:

./configure --prefix=/opt/eb/software/MVAPICH2/1.9b-ClangGCC-1.1.2
--with-rdma=gen2 --with-thread-package=pthreads --enable-fast
--enable-f77 --enable-fc --enable-cxx

I get:

[easybuild at cluster-build examples]$ make V=1
/bin/sh ../libtool  --tag=CC   --mode=link clang  -O2 -march=native
-L../lib -L../src/mpl -L../src/openpa/src -L/lib -L/lib
-L/opt/eb/software/GCC/4.7.2/lib64
-L/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib  -L/lib
-Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib -static -L../lib
-L../src/mpl -L../src/openpa/src -L/lib -L/lib
-L/opt/eb/software/GCC/4.7.2/lib64
-L/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib  -L/lib
-Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib -o cpi cpi.o -lm
-libmad -libumad -libverbs -lrt -lhwloc -lhwloc -lm -lpthread
-lpmpich -lmpich -lmpl -lopa -lopa -lmpl  -libmad -libumad -libverbs
-lrt -lhwloc -lhwloc -lm -lpthread
libtool: link: clang -O2 -march=native -Wl,-rpath -Wl,/lib -Wl,-rpath
-Wl,/lib -Wl,-rpath -Wl,/lib -Wl,-rpath -Wl,/lib -o cpi cpi.o
-L../lib -L../src/mpl -L../src/openpa/src -L/lib
-L/opt/eb/software/GCC/4.7.2/lib64
-L/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libpmpich.a
-L./src/mpl -L./src/openpa/src ../src/mpl/.libs/libmpl.a
../src/openpa/src/.libs/libopa.a
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/src/openpa/src/.libs/libopa.a
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/src/mpl/.libs/libmpl.a
-libmad -libumad -libverbs -lrt
/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib/libhwloc.so -ldl -lm
-lpthread -Wl,-rpath
-Wl,/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib -Wl,-rpath
-Wl,/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libpmpich.a(lib_libpmpich_la-bcast.o):
In function `MPI_Bcast':
src/mpi/coll/bcast.c:(.text+0xc9): undefined reference to
`mv2_increment_shmem_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libpmpich.a(lib_libpmpich_la-reduce.o):
In function `MPI_Reduce':
src/mpi/coll/reduce.c:(.text+0xf2): undefined reference to
`mv2_increment_shmem_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(allreduce.o):
In function `PMPI_Allreduce':
src/mpi/coll/allreduce.c:(.text+0x1104): undefined reference to
`mv2_increment_shmem_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(barrier.o):
In function `PMPI_Barrier':
src/mpi/coll/barrier.c:(.text+0x48d): undefined reference to
`mv2_increment_shmem_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(bcast.o):
In function `PMPI_Bcast':
src/mpi/coll/bcast.c:(.text+0x17c7): undefined reference to
`mv2_increment_shmem_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(reduce.o):src/mpi/coll/reduce.c:(.text+0x1c66):
more undefined references to `mv2_increment_shmem_coll_counter' follow
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(allgather.o):
In function `PMPI_Allgather':
src/mpi/coll/allgather.c:(.text+0xfa2): undefined reference to
`mv2_increment_allgather_coll_counter'
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a(scatter.o):
In function `PMPI_Scatter':
src/mpi/coll/scatter.c:(.text+0xe20): undefined reference to
`mv2_increment_shmem_coll_counter'
clang-3: error: linker command failed with exit code 1 (use -v to see
invocation)
make: *** [cpi] Error 1

This can be fixed by applying the attached patch.  The error is caused
by Clang implementing C99 inline semantics correctly.  This is
explained in, for example,
http://www.greenend.org.uk/rjk/tech/inline.html

After fixing this, I see:

make[2]: Entering directory
`/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/osu_benchmarks'
/bin/sh ../libtool  --tag=CC   --mode=link clang  -O2 -march=native
-DNDEBUG -DNVALGRIND ../lib/libpmpich.la ../lib/libmpich.la -L/lib
-L/lib -L/opt/eb/software/GCC/4.7.2/lib64
-L/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib  -L/lib
-Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib -o osu_alltoallv
osu_alltoallv.o  -libmad -libumad -libverbs -lrt -lhwloc -lhwloc -lm
-lpthread
libtool: link: clang -O2 -march=native -DNDEBUG -DNVALGRIND -Wl,-rpath
-Wl,/lib -Wl,-rpath -Wl,/lib -o osu_alltoallv osu_alltoallv.o
../lib/.libs/libpmpich.a -L/lib -L/opt/eb/software/GCC/4.7.2/lib64
-L/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib -L./src/mpl
-L./src/openpa/src -lmpl -lopa
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/lib/.libs/libmpich.a
../lib/.libs/libmpich.a
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/src/mpl/.libs/libmpl.a
/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/src/openpa/src/.libs/libopa.a
-libmad -libumad -libverbs -lrt
/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib/libhwloc.so -ldl -lm
-lpthread -Wl,-rpath
-Wl,/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib -Wl,-rpath
-Wl,/opt/eb/software/hwloc/1.6.2-ClangGCC-1.1.2/lib
/usr/bin/ld: cannot find -lmpl
clang-3: error: linker command failed with exit code 1 (use -v to see
invocation)
make[2]: *** [osu_alltoallv] Error 1
make[2]: Leaving directory
`/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b/osu_benchmarks'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/home/easybuild/.local/easybuild/build/MVAPICH2/1.9b/ClangGCC-1.1.2/mvapich2-1.9b'
make: *** [all] Error 2

(The same error as in the first message.)

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: c99-inline-semantics.patch
Type: application/octet-stream
Size: 751 bytes
Desc: not available
Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20130411/e7dbb092/c99-inline-semantics.obj