[mvapich-discuss] MVAPICH error question

[Martin Schatz]

Hello,
	My name is Martin Schatz and I am currently working on a C++ project that uses the mvapich2 library.  I have been running into some problems while running my code and cannot for the life of me figure out what is going on.  I was hoping that maybe someone on this list could help out.  If this is the wrong mailing list, please refer me to the correct one.

	For background, the project I am working on requires a thread-safe implementation of MPI as multiple threads (pthreads are used) are making MPI calls at the same time.  Currently, I have the level of thread safety set to MPI_THREAD_SERIALIZED, though I have tried the same code with the higher MPI_THREAD_MULTIPLE safety level (but I do not think the MULTIPLE level is necessary as I am ok with the calls occurring in some serial fashion).  When running my program using mpirun with debugging turned on, I see no indication that something is wrong, however when I run the program in an large parallel environment (TACC machines), the following error occurs:

Exit code -5 signaled from i182-111.ranger.tacc.utexas.edu
Killing remote processes...current bytes -56, total bytes 0, remote id 11
Assertion failed in file ch3_smp_progress.c at line 2328: s_current_bytes[vc->smp.local_nodes] == 0
internal ABORT - process 12MPI process terminated unexpectedly
DONE

This error does not occur in a specific place during the execution (everything seems to be working fine when it occurs).  I was wondering if anyone on this list could explain what this error means and perhaps some tips on tracking down the bug.  Also, I am not using explicit locks in the program around the MPI calls.  I thought that by specifying the level of thread safety in MPI_thread_init, it would be taken care of for me.  Is this in error?  Thanks for any guidance you can provide.


Martin Schatz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20101206/434ad597/attachment-0001.html