[mvapich-discuss] Change rndv_window_size

[Dhabaleswar Panda]

Hi Satoshi,

Thanks for your report. Could you provide some more details on the error
you are seeing.  Otherwise, it is very hard to know what is happening here
and extend help.

1. Which MPI program you are running?

2. On how many nodes/cores you are running this program? Are you running
with single rail or multiple rails? What is the smallest system size you
see this error?

3. When do you see this error (in the begining, middle or end)? Is this
error consistent (happens always at the same time) or random?

4. Which MVAPICH2 version you are using?

Is this program publicly available. This may help us to reproduce the
error.

Best Regards,

DK



On Tue, 16 Mar 2010, Satoshi Isono wrote:

> Hello everyone,
>
> I couldn't run MPI code with below errors. I think that I have to set larger number than default one as rndv_window_size. How can I set larger buffer size? Could you advise me about it?
>
> I guess I may be able to use these below environment values to control this number.
>
>    MV2_IBA_EAGER_THRESHOLD
>    MV2_VBUF_TOTAL_SIZE
>
> The following is error messages:
>
> Warning! Rndv Receiver is receiving (269136 < 442152) less than as expected
> MPI process terminated unexpectedly
> Exit code -5 signaled from com-0559 Killing remote processes...
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> MPI process terminated unexpectedly
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 422928) less than as expected
> MPI process terminated unexpectedly
> Warning! Rndv Receiver is receiving (269136 < 365256) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 307584) less than as expected
> Warning! Rndv Receiver is receiving (269136 < 288360) less than as expected
> Fatal error in MPI_Bcast:
> Message truncated, error stack:
> MPI_Bcast(1144)........................:
> MPI_Bcast(buf=0x6ca4bf0, count=33642, MPI_DOUBLE_PRECISION, root=0, comm=0x84000002) failed
> MPIR_Bcast(228)........................:
> MPIDI_CH3U_Post_data_receive_found(439): Message from rank 0 and tag 2 truncated; 288360 bytes received but buffer size is 269136
>
> Regards,
> Satoshi Isono
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>