[mvapich-discuss] Error in make of mvapich2-1.8rc1

Fri May 25 10:43:35 EDT 2012

Thanks for your note.  Hopefully I can help resolve your issues.

I noticed that you are build the NEMESIS TCP device
(--with-device=ch3:nemesis).  The other features that you're trying to
enable such as CUDA and LiMIC2 are not compatible with this device.
Please use the default device (ch3:mrail) to use the Infiniband network
with the other advanced features.  Based on your original configuration
you should only need the CUDA and LiMIC2 options.

Also you should use the official mvapich2-1.8 release as opposed to
mvapich2-1.8rc1.  You can download this from 
http://mvapich.cse.ohio-state.edu/download/mvapich2/mvapich2-1.8.tgz.

For more information on how to configure and build MVAPICH2 please take
a look at
http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.8.html.

If you still have trouble with hwloc please send the config.log from
src/pm/hydra/ subdirectory.  I suspect that your local installation of
hwloc may be interacting in a non-ideal way with the internal version
used by the hydra process manager.  We can debug this further if needed.

On Fri, 25 May 2012, Jonathan Perkins wrote:

> Regarding HWLOC & PLPA questions:
> Since he has an installation of HWLOC on his system MVAPICH2 will try to
> use that.  However if it is not in a standard location then he will need
> to use LD_LIBRARY_PATH or add -Wl,-rpath=... into his LDFLAGS.  If he
> does the LD_LIBRARY_PATH he will also need to do this at runtime as this
> will not be propogated for him (I suggest he use -Wl,-rpath).
>
> The HWLOC and PLPA checks he is referring to are specific to the hydra
> launcher which uses there own version of hwloc/plpa and not the system
> supplied one.  I cannot tell for sure why this is failing without seeing
> the config.log from the hydra directory (src/pm/hydra/config.log).  I
> suspect it is an interaction with the system installed library and some
> LD_LIBRARY_PATH issue.  If HWLOC was able to work PLPA would never be
> tried.  We can just let him know that this is a fallback attempt.
>
> Regarding the other stuff:
> Before he continues too far I think he'll need to know that CUDA and
> LiMIC2 do not work with Nemesis.  Even the HWLOC stuff works differently
> with Nemesis as you'll actually need to have hydra do the binding and
> not the library.
>
> I'm not sure of the status of dynamiclibs but this may not work and as
> we have seen before, --enable-smpcoll is a MPICH2 specific flag.  He can
> keep --enable-smpcoll but I suggest removing --enable-dynamiclibs.
>

On Fri, May 25, 2012 at 01:58:31PM +0800, ashish pancholi wrote:
> Hello ,
> 
> I am trying to install mvapich2-1.81rc2 on my system:
> 
> Processor:              2 x Intel 5520
> Motherboard:           Dual socket, Intel 5520sc
> GPU:                      4 x Tesla c1060, 1 Quadro NVS 290 (for display)
> OS :                       Centos 6.2 x86_64 (kernel 2.6.32-220.el6.x86_64)
> CUDA Toolkit:         5.0 with Nsight for Eclipse
> Hwloc                     1.4.1
> Java                       Sun JDK  1.6.31 (jre 1.6.31) ,Open-JDK(jre-1.6.0-openjdk.x86_64)
> 
> Config command is:
> 
> ./configure --prefix=/opt/mvapich2 --with-device=ch3:nemesis --enable-dynamiclibs --enable-cxx --enable-debuginfo --enable-smpcoll --enable-mpe --enable-shared --enable-cuda --with-mpe --with-cuda=/usr/local/cuda --with-cuda-include=/usr/local/cuda/include --with-cuda-libpath=/usr/local/cuda/lib64 --with-cuda-libpath=/usr/local/cuda/lib --with-hwloc --with-limic2-include=/usr/local/include --with-limic2-libpath=/usr/local/lib --with-thread-package=posix 2>&1|tee buildlog.log
> 
> Configure completes with following erros. 
> 
> "
> checking for hwloc version... 1.2.1rc2-1
> checking for gcc... gcc
> checking whether the C compiler works... no
> configure: error: in `/root/Downloads/mvapich2-1.8rc1/builddir/src/pm/hydra/tools/topo/hwloc/hwloc':
> configure: error: C compiler cannot create executables
> See `config.log' for more details
> "
> 
> Q1. I have hwloc 1.4.1 on my system (lstopo --version returns 1.4.1).
>  I want to use the NUMA capability in mvapich2. When I give --with-hwloc option, does this use the 1.4.1 libs (path to lib,include is exported to $PATH and $LD_LIBRARY_PATH at system startup) or does it try to install 1.2.1rc2-1 ?
> 
> checking for PLPA version... 1.3.2
> checking for gcc... gcc
> checking whether the C compiler works... no
> configure: error: in `/root/Downloads/mvapich2-1.8rc1/builddir/src/pm/hydra/tools/topo/plpa/plpa':
> configure: error: C compiler cannot create executables
> See `config.log' for more details
> 
> Q2. Is PLPA not deprecated by hwloc (as mentioned on http://www.open-mpi.org/projects/plpa/) ? So if I have hwloc 1.4.1 this is not needed.
> 
> I am attaching the buildlog.log (output to the console during the config) and config.log. 
> 
> When I make I get the following error:
> 
> make[8]: Entering directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp'
>   CC              /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_finalize.c
>   CC              /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_init.c
>   CC              /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c: In function ‘MPID_nem_tcp_iStartContigMsg’:
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c:317: error: incompatible types when assigning to type ‘void *’ from type ‘MPIDI_CH3_PktGeneric_t’
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c: In function ‘MPID_nem_tcp_iStartContigMsg_paused’:
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c:456: error: incompatible types when assigning to type ‘void *’ from type ‘MPIDI_CH3_PktGeneric_t’
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c: In function ‘MPID_nem_tcp_iSendContig’:
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c:604: error: incompatible types when assigning to type ‘void *’ from type ‘MPIDI_CH3_PktGeneric_t’
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c: In function ‘MPID_nem_tcp_SendNoncontig’:
> /root/Downloads/mvapich2-1.8rc1/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp/tcp_send.c:732: error: incompatible types when assigning to type ‘void *’ from type ‘MPIDI_CH3_PktGeneric_t’
> make[8]: *** [tcp_send.o] Error 1
> make[8]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels/nemesis/nemesis/netmod/tcp'
> make[7]: *** [all-redirect] Error 1
> make[7]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels/nemesis/nemesis/netmod'
> make[6]: *** [all-redirect] Error 2
> make[6]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels/nemesis/nemesis'
> make[5]: *** [all-redirect] Error 2
> make[5]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels/nemesis'
> make[4]: *** [all-redirect] Error 1
> make[4]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3/channels'
> make[3]: *** [all-redirect] Error 2
> make[3]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid/ch3'
> make[2]: *** [all-redirect] Error 1
> make[2]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src/mpid'
> make[1]: *** [all-redirect] Error 2
> make[1]: Leaving directory `/root/Downloads/mvapich2-1.8rc1/builddir/src'
> make: *** [all-redirect] Error 2
> 
> Attached is the output on console during the make (make.log). Can u please help in figuring out what is it that I am doing wrong or how can this error be resolved.
> 
> TIA
> ashish .
> 
> 
> 
> 
> 
> 
> 

> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo