[mvapich-discuss] Old error popping up again - message truncated

[Gardiner, Judith]

We're using mvapich2-2.1 and are encountering an error that I thought was solved a couple of versions ago.  The code runs correctly on 240 processors with OpenMPI, although with inconsistent performance, but it fails with mvapich2.  I haven't tried to debug it to be sure it's not an application error.  Do you have an environment variable I can set to quickly figure that out?  The code is Fortran 90, if that makes any difference.

Fatal error in MPI_Allreduce:
Message truncated, error stack:
MPI_Allreduce(937)........................: MPI_Allreduce(sbuf=0x7fff6a048a7c, rbuf=0x7fff6a048a78, count=1, MPI_INT, MPI_MAX, comm=0x84000004) failed
MPI_Allreduce(919)........................:
MPIDI_CH3I_SHMEM_COLL_Barrier_bcast(1496).:
create_2level_comm(708)...................:
MPIR_Allreduce_impl(777)..................:
MPIR_Allreduce_index_tuned_intra_MV2(2486):
FUNCNAME(357).............................:
MPIDI_CH3U_Receive_data_found(282)........: Message from rank 1 and tag 14 truncated; 260 bytes received but buffer size is 4


Thanks for your help.

Judy

--
Judith D. Gardiner, Ph.D.
Ohio Supercomputer Center
614-292-9623
judithg at osc.edu<mailto:judithg at osc.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20160105/c8080a34/attachment.html>