[mvapich-discuss] [SPAM] Maximum number of pending MPI_Isend calls? What signal is thrown by INTERNAL ERRORs?

[Tim Hartley]

Hi,

I'm curious if there is a hard or soft limit on the number of pending
MPI_Isend calls, either from a single MPI process, or to a single MPI
process.

I am running a multithreaded test application on 32 nodes of the OSU
BMI department. We use middleware called DataCutter which is MPI-aware
(so only one thread - the "runtime" thread - does any MPI calls). I'm
fiddling around quite a bit and I am able to get the following fail
messages:


Fatal error in MPI_Recv: Other MPI error
[cli_32]: aborting job:
Fatal error in MPI_Recv: Other MPI error
INTERNAL ERROR: Invalid error class (66) encountered while returning from
MPI_Test.  Please file a bug report.
Fatal error in MPI_Test: Unknown error.  Please file a bug report., error stack:
(unknown)(): connection failure
[cli_31]: aborting job:
Fatal error in MPI_Test: Unknown error.  Please file a bug report., error stack:
(unknown)(): connection failure

etc.

However, if I simply throttle the number of pending Isends, this
behavior goes away. This absolutely may be a bug in my application
code, but I haven't been able to get a corefile or catch the signal
with a signal handler or an already-attached gdb.

What signal will this kind of error throw?

Thanks a lot,
Tim