[mvapich-discuss] Error (PMPI_Init_thread) when running MPI

[Hari Subramoni]

Hello,

It looks like /dev/shm folder on your system is full. This is possibly
leading to this issue. Ideally, this should not happen... especially if
your application is exiting cleanly. Can you please let us know if there
have been any failures at the application level previously?

As a workaround, please try to clear the files in /dev/shm and retry.

On a different note, I see that you maybe running in container environment.
We have a version of mvapich2 specifically optimized for delivering bare
metal virtualized environments called MVAPICH2-Virt. I'd recommend that you
use it.

Regards,
Hari.


On Mar 15, 2017 5:25 PM, "Junjie Qian" <junjieqian at outlook.com> wrote:

Hi List,


Recently I have a problem with MVAPICH2 running, which did not occur
before. The error log is as:


[cli_0]: aborting job:
Fatal error in PMPI_Init_thread:
Other MPI error, error stack:
MPIR_Init_thread(514)..........:
MPID_Init(365).................: channel initialization failed
MPIDI_CH3_Init(404)............:
MPIDI_CH3I_SHMEM_Helper_fn(929): write: No space left on device
[:mpispawn_0][readline] Unexpected End-Of-File on file descriptor 8. MPI
process died?
[mpispawn_0][mtpmi_processops] Error while reading PMI socket. MPI process
died?

My command line is as: mpirun_rsh -np 2 container_1 container_2
MV2_NUM_PORTS=1 MV2_IBA_HCA=mlx4_0 MV2_DEFAULT_PORT=1 MV2_SMP_USE_CMA=0
MV2_ENABLE_AFFINITY=0 ./exec

Can you give me some suggestion on how to solve this issue? I tried
different combinations of command options in the user guide, but none help.

Thank you
Junjie Qian


_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20170315/99d21e97/attachment-0001.html>