[mvapich-discuss] mvapich2-1.0.3 + charm++ error

[Sangamesh B]

Hi,

    I've installed mvapich2-1.0.3 (make.mvapich2.ofa stack) with Intel
compilers on Cent OS on a Rocks 4.3 cluster.

   Installed charm5.9 (a parallel programming OO language library)

  The charm build is successful with following options:

./build charm++ mpi-linux iccstatic pthreads --no-build-shared
--incdir=/opt/MPI/mvapich2_intel/include
--libdir=/opt/MPI/mvapich2_intel/lib --libdir=/usr/lib

 But when I try to execute tests/charm++/simplearrayhello program:

[root at local simplearrayhello]# make
../../../bin/charmc   hello.ci
../../../bin/charmc  -c hello.C
../../../bin/charmc  -language charm++ -o hello hello.o
icpc: command line remark #10010: option '-static-libcxa' is deprecated and
will be removed in a future release. See '-help deprecated'
ld: skipping incompatible /usr/lib/libpthread.so when searching for
-lpthread
ld: skipping incompatible /usr/lib/libpthread.a when searching for -lpthread
ld: skipping incompatible /usr/lib/libpthread.so when searching for
-lpthread

..
..
ld: skipping incompatible /usr/lib/libc.so when searching for -lc
ld: skipping incompatible /usr/lib/libc.a when searching for -lc
/opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0xa7): In
function `MPIDI_CH3I_CM_Finalize':
: undefined reference to `ibv_dereg_mr'
/opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0xb9): In
function `MPIDI_CH3I_CM_Finalize':
: undefined reference to `ibv_dereg_mr'
/opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0x112): In
function `MPIDI_CH3I_CM_Finalize':
: undefined reference to `ibv_destroy_qp'


...
Fatal Error by charmc in directory
/opt/apps/namd_intel/charm-5.9/tests/charm++/simplearrayhello
   Command icpc -static-libcxa -L/opt/MPI/mvapich2_intel/lib -L/usr/lib
-L/opt/MPI/mvapich2_intel/lib -L/usr/lib -o hello -L../../../bin/../lib
-I../../../bin/../include ../../../bin/../lib/libldb-rand.o hello.o
moduleinit3173.o ../../../bin/../lib/libmemory-default.o
../../../bin/../lib/libthreads-default.o -lck -lconv-cplus-y -lconv-core
-lconv-util -L/opt/MPI/mvapich2_intel/lib -L/usr/lib -lckqt -lmpich
-lpthread -lpthread -ldl -lpthread -lm returned error code 1
charmc exiting...
make: *** [hello] Error 1

It looks, charmc is not able to link libibverbs.

With MPICH2, there is no problem charm++ execution.

The src/arch/mpi-linux/conv-mach.sh is:

CMK_CPP_CHARM='/lib/cpp -P'
CMK_CPP_C='mpicc -E'
CMK_CC='mpicc '
CMK_CXX='mpicxx '
CMK_CXXPP='mpicxx -E '
CMK_CF77='ifort'
CMK_CF90='ifort'
CMK_RANLIB='ranlib'
CMK_LIBS='-L/opt/MPI/mvapich2_intel/lib -L/usr/lib -lckqt -lmpich -lpthread'
CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"
CMK_NATIVE_LIBS=''
CMK_NATIVE_CC='icc '
CMK_NATIVE_LD='icpc'
CMK_NATIVE_CXX='icpc '
CMK_NATIVE_LDXX='icpc'
CMK_NATIVE_CC='icc '
CMK_NATIVE_CXX='icpc '
CMK_F90LIBS='-L/usr/absoft/lib -L/opt/absoft/lib -lf90math -lfio -lU77
-lf77math '
CMK_MOD_NAME_ALLCAPS=1
CMK_MOD_EXT="mod"
CMK_F90_USE_MODDIR=1
CMK_F90_MODINC="-p

Is this setting is ok or I need to add some more library paths (i.e. OFED
libs)?

Thank you,
Sangamesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080804/c6da1818/attachment.html