[mvapich-discuss] Completion of non-blocking collective MPI_Iallgather

[Kumbhar Pramod Shivaji]

Hello All,

I am looking into mpi non-blocking collectives (specifically mpi_iallgather & mpi_iallgatherv)
and wondering about the internal completion / progress of these routines. To explain, I have following
pseudo example:

MPI_Iallgather(…&req)          // this called by all ranks at the same time
...
while( i < 4) {

          compute(…);   // compute for few seconds
          MPI_Test(…&flag, &req);                                 // Can I expect this flag to be true? i.e. operation is completed here at some point?
          i++;
}
….
MPI_Wait(&req);    // I see that MPI_Iallgather is getting finished when everyone calls MPI_Wait at this point, why?


I was running this test example on single node with 8 mpi ranks. I see that MPI_Test is always returning false. With
compute() function of two seconds, I expect MPI_Iallgather to finish quickly. But instead mpi_iallgather is completed
at MPI_Wait(). See attached sample trace of the program.

Could anyone point out possible cause of this behaviour? It will be great help.


Regards,
Pramod

MVAPICH 2 version: 2.0
Configure command:  /configure --with-slurm  --with-slurm-include=slurm/default/include --with-slurm-lib=slurm/default/lib  --with-pm=none --with-pmi=slurm --enable-threads=multiple --enable-shared --enable-sharedlibs=gcc --enable-fc --enable-cxx --with-mpe --enable-rdma-cm --enable-fast --enable-smpcoll --with-hwloc --enable-xrc --with-device=ch3:mrail --with-rdma=gen2 --enable-cuda --with-cuda=/usr/local/cuda-5.0 --with-cuda-include=/usr/local/cuda-5.0/include --with-cuda-lib=/usr/local/cuda-5.0/lib64 --enable-g=dbg --enable-debuginfo CC=gcc CXX=g++ FC=gfortran F77=gfortran


[cid:C1753CB8-7077-4BC7-ACC6-C4998A458254 at epfl.ch]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150126/5a1e6e53/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpi_iallgather_trace.png
Type: image/png
Size: 79473 bytes
Desc: mpi_iallgather_trace.png
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150126/5a1e6e53/attachment-0001.png>