[mvapich-discuss] mvapich2-1.0.3 + charm++ error

[Sangamesh B]

Hi Matthew,

My answers are in-line.

On Mon, Aug 4, 2008 at 7:35 PM, Matthew Koop <koop at cse.ohio-state.edu>wrote:

> Sangamesh,
>
> In the past I have given these instructions for MVAPICH:
>
> http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2006-June/000189.html
>
> MVAPICH2 should be the same with the exception of mpiCC for mpicxx. The
> settings you have below look fine to me. If you have the proper wrapper
> scripts (mpicc, etc) there should be no need to specify the OFED libraries
> there.
>
> Can you verify that 'which mpicc' and 'which mpicxx' do point to the
> working MVAPICH2 installation?
>
Yes. It is mvapich2's mpicc only.

One more point is: After building  charm++, the charmc script in the bin dir
doesn't contain reference to "-lmpich".  So it gave errors " undefined
reference to MPI_CH3_ .." .

Then I put it manually "-lmpich". The above errors disappeared but " > :
undefined reference to `ibv_dereg_mr' " errors started appearing.

Shall I try with mvapich2-1.0.2rc1?

Thank you,
Sangamesh

>
> Thanks,
>
> Matt
>
> On Mon, 4 Aug 2008, Sangamesh B wrote:
>
> > Hi,
> >
> >     I've installed mvapich2-1.0.3 (make.mvapich2.ofa stack) with Intel
> > compilers on Cent OS on a Rocks 4.3 cluster.
> >
> >    Installed charm5.9 (a parallel programming OO language library)
> >
> >   The charm build is successful with following options:
> >
> > ./build charm++ mpi-linux iccstatic pthreads --no-build-shared
> > --incdir=/opt/MPI/mvapich2_intel/include
> > --libdir=/opt/MPI/mvapich2_intel/lib --libdir=/usr/lib
> >
> >  But when I try to execute tests/charm++/simplearrayhello program:
> >
> > [root at local simplearrayhello]# make
> > ../../../bin/charmc   hello.ci
> > ../../../bin/charmc  -c hello.C
> > ../../../bin/charmc  -language charm++ -o hello hello.o
> > icpc: command line remark #10010: option '-static-libcxa' is deprecated
> and
> > will be removed in a future release. See '-help deprecated'
> > ld: skipping incompatible /usr/lib/libpthread.so when searching for
> > -lpthread
> > ld: skipping incompatible /usr/lib/libpthread.a when searching for
> -lpthread
> > ld: skipping incompatible /usr/lib/libpthread.so when searching for
> > -lpthread
> >
> > ..
> > ..
> > ld: skipping incompatible /usr/lib/libc.so when searching for -lc
> > ld: skipping incompatible /usr/lib/libc.a when searching for -lc
> > /opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0xa7): In
> > function `MPIDI_CH3I_CM_Finalize':
> > : undefined reference to `ibv_dereg_mr'
> > /opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0xb9): In
> > function `MPIDI_CH3I_CM_Finalize':
> > : undefined reference to `ibv_dereg_mr'
> > /opt/MPI/mvapich2_intel/lib/libmpich.a(rdma_iba_init.o)(.text+0x112): In
> > function `MPIDI_CH3I_CM_Finalize':
> > : undefined reference to `ibv_destroy_qp'
> >
> >
> > ...
> > Fatal Error by charmc in directory
> > /opt/apps/namd_intel/charm-5.9/tests/charm++/simplearrayhello
> >    Command icpc -static-libcxa -L/opt/MPI/mvapich2_intel/lib -L/usr/lib
> > -L/opt/MPI/mvapich2_intel/lib -L/usr/lib -o hello -L../../../bin/../lib
> > -I../../../bin/../include ../../../bin/../lib/libldb-rand.o hello.o
> > moduleinit3173.o ../../../bin/../lib/libmemory-default.o
> > ../../../bin/../lib/libthreads-default.o -lck -lconv-cplus-y -lconv-core
> > -lconv-util -L/opt/MPI/mvapich2_intel/lib -L/usr/lib -lckqt -lmpich
> > -lpthread -lpthread -ldl -lpthread -lm returned error code 1
> > charmc exiting...
> > make: *** [hello] Error 1
> >
> > It looks, charmc is not able to link libibverbs.
> >
> > With MPICH2, there is no problem charm++ execution.
> >
> > The src/arch/mpi-linux/conv-mach.sh is:
> >
> > CMK_CPP_CHARM='/lib/cpp -P'
> > CMK_CPP_C='mpicc -E'
> > CMK_CC='mpicc '
> > CMK_CXX='mpicxx '
> > CMK_CXXPP='mpicxx -E '
> > CMK_CF77='ifort'
> > CMK_CF90='ifort'
> > CMK_RANLIB='ranlib'
> > CMK_LIBS='-L/opt/MPI/mvapich2_intel/lib -L/usr/lib -lckqt -lmpich
> -lpthread'
> > CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"
> > CMK_NATIVE_LIBS=''
> > CMK_NATIVE_CC='icc '
> > CMK_NATIVE_LD='icpc'
> > CMK_NATIVE_CXX='icpc '
> > CMK_NATIVE_LDXX='icpc'
> > CMK_NATIVE_CC='icc '
> > CMK_NATIVE_CXX='icpc '
> > CMK_F90LIBS='-L/usr/absoft/lib -L/opt/absoft/lib -lf90math -lfio -lU77
> > -lf77math '
> > CMK_MOD_NAME_ALLCAPS=1
> > CMK_MOD_EXT="mod"
> > CMK_F90_USE_MODDIR=1
> > CMK_F90_MODINC="-p
> >
> > Is this setting is ok or I need to add some more library paths (i.e. OFED
> > libs)?
> >
> > Thank you,
> > Sangamesh
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080806/aff98df1/attachment.html