[mvapich-discuss] MVAPICH2 (2.1rc1) on PSM error

[Wasko, Wojciech]

>  I occasionally get the following error when I issue a large number

> (in this case, the large number is >10 million) of `MPI_Put`: 

> `b11.10016Exhausted 1048576 MQ irecv request descriptors,

> which usually indicates a user program error or insufficient request descriptors

> (PSM_MQ_RECVREQS_MAX=1048576)`


Sayan,

 

Your program seems to be exhausting the number of entries in the Matched Queue that are available. To increase it, you can change the memory model in PSM to “large” by passing the environment variable PSM_MEMORY=large to the processes.

 

However, it doesn’t solve the fundamental problem, it just pushes the boundary further out. Bottomline, I think any transport will quit when queuing up billions of puts on it… To deal with the problem correctly, I think the workload should insert a barrier every certain number of puts. That’d make sure you don’t overflow the transport.

 

Wojciech Waśko

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20161003/95a3dbd1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6315 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20161003/95a3dbd1/attachment.p7s>
-------------- next part --------------
--------------------------------------------------------------------

Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek
przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by
others is strictly prohibited.