[mvapich-discuss] Problem with MPI_Init

[Schwind Michael]

Hello Matthew, 

On Thursday 07 December 2006 02:07, Matthew Koop wrote:
> Michael,
>
> I'm a bit unclear on what issue you are facing here. I'm assuming this is
> with MVAPICH2 (not openmpi as you note later)? If so, what version are you
> using and with what compile and configure options? 

I am using openib version 1.0 and gcc-4.1.2.
My kernel is 2.6.18 from kernel.org patched 
with perfctr 2.6.25. 


Openib was configured with the these userlevel-options:
WITH_IBVERBS=yes
WITH_MTHCA=yes
WITH_IPATHVERBS=yes
WITH_EHCA=no
WITH_CM=yes
WITH_SDP=yes
WITH_DAPL=yes
WITH_RDMACM=yes
WITH_MANAGEMENT_LIBS=yes
WITH_OSM=yes
WITH_DIAGS=yes
WITH_MPI=no
WITH_PERFTEST=yes
WITH_SRPTOOLS=yes
WITH_TVFLASH=yes
WITH_MSTFLINT=yes

The kernel is compiled with the following options:
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_INFINIBAND_USER_ACCESS=m
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_MTHCA=m
# CONFIG_INFINIBAND_MTHCA_DEBUG is not set
CONFIG_INFINIBAND_IPOIB=m
# CONFIG_INFINIBAND_IPOIB_DEBUG is not set
CONFIG_INFINIBAND_SRP=m
CONFIG_INFINIBAND_ISER=m
CONFIG_INFINIBAND_SDP=m
# CONFIG_INFINIBAND_SDP_DEBUG is not set
# CONFIG_INFINIBAND_MADEYE is not set


> How many processes is 
> this seen with?
It's seen with only 2 processes runing on different nodes.


> The MPID_CS_ENTER macro is only defined when --enable-threads is used in
> the configure stage, otherwise it is not defined. Even with
> multithreading, we have not seen any issues.


I tried now different configurations with and without 
multithreading-support.

Values set in make.mvapch2.gen2:
OPT_FLAG=-O2
ENABLE_CKPT=no
PTMALLOC=yes
MULTI_THREAD=yes
RDMA_CM_SUPPORT=no
ROMIO=yes

The result is  that my program
and osu_latency  hang in MPI_Init as
mentioned in my original posting.

When I change MULTI_THREAD=yes to
MULTI_THREAD=no, the result is that 
the programs work.

Thanks Michael

>
> Please let me know a little more about what options you are using so we
> can better assist.
>
> Thanks,
> Matt



> On Tue, 5 Dec 2006, Schwind Michael wrote:
> > Hello,
> >
> > first lets say thank you for your great work
> > on mvapich.
> >
> > Now my problem:
> >
> > My program hangs  in MPI_Init, when I start it.
> >
> >
> > The problem seems to be that the thread
> > waits to lock a mutex.
> >
> > The mutex was set in MPI_Init through
> > MPID_CS_ENTER() on line 89 in init.c.
> >
> > The thread calls the macro  MPID_CS_ENTER()
> > a second time in the function MPI_Comm_rank
> > some time later in the function MPI_Init:
> >
> > if (split_comm == 1){
> >        int my_id, size;
> >        MPI_Comm_rank(MPI_COMM_WORLD, &my_id);
> >
> > My distribution is debian testing on amd64.
> > I use openmpi 1.0 compiled with gcc 4.1.2.
> >
> >
> > Whats wrong with my setup.
> >
> > Thanks
> >
> > Michael
> >
> >
> >
> >
> > _______________________________________________
> > mvapich-discuss mailing list
> > mvapich-discuss at cse.ohio-state.edu
> > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss