[mvapich-discuss] error compiling mvapich2-1.8 with CUDA support on ubuntu

[Jonathan Perkins]

On Fri, Jul 27, 2012 at 04:03:30PM +0200, Zbigniew Koza wrote:
> Hi,
> 
> I've just tried to compile mvapich2-1.8 (official release version
> from a tarball) with CUDA support on ubuntu.
> 
> > ./configure -prefix=/usr/local/mvapich2-1.8 --enable-cuda
> --with-cuda=/usr/local/cuda
> --with-cuda-include=/usr/local/cuda/include
> --with-cuda-libpath=/usr/lib

You should not need to specify /usr/lib as a libpath as this should be
part of the system search path already.  I'm not very familiar with the
ubuntu distro but its possible that adding this option is changing which
version of libraries are being linked (eg /usr/lib instead of
/usr/lib64).

What happens when you try the same configuration minus the
--with-cuda-include and --with-cuda-libpath options?

> > make
> [...]
> make all-postamble
> make[1]: Entering `/home/zkoza/mvapich2/mvapich2-1.8'
> ( cd examples && make )
> make[2]: Entering `/home/zkoza/mvapich2/mvapich2-1.8/examples'
>   CC              cpi.c
> ../bin/mpicc   -o cpi cpi.o  -lm
> /usr/bin/ld: /home/zkoza/mvapich2/mvapich2-1.8/lib/libmpich.a(alltoall_cuda_osu.o):
> undefined reference to symbol 'ceil@@GLIBC_2.2.5'
> /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libm.so
> so try adding it to the linker command line
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libm.so:
> could not read symbols: Invalid operation
> collect2: ld returned 1 exit status
> 
> 
> This error does not appear when compiling the library without CUDA support.
> Changing gcc or nvcc version does not help.
> However, I've noticed the problem can be circumvented by manually
> editing Makefile to remove -lm flag.
> 
> Is this an acceptable solution?
> (the resulting library does not seem to work with CUDA pointers
> anyway, but before I proceed to that point, I would like to
> be sure that the compilation process proceeds correctly).

This is not ideal as we'd like to have configure create the Makefiles
and avoid manual changes.  Also as you've mentioned, the resulting
binaries are not working properly.

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo