[mvapich-discuss] Problems with MVAPICH2
=?gb2312?B?wO641Q==?=
li.larry at hotmail.com
Thu Jul 7 11:06:27 EDT 2011
Hi,
I got tow error while running jobs with Mvapich217 a2 / Mvapich216, It got error message like this: Thanks. error1: mpiexec -f ./hosts -np 64 ./vasp
running on 64 nodes
distr: one band on 1 nodes, 64 groups
vasp.5.2.11 18Jan11 complex
POSCAR found : 3 types and 36 ions
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
WARNING: small aliasing (wrap around) errors must be expected
FFT: planning ...( 1 )
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)=====================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= EXIT CODE: 11
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
=====================================================================================
[proxy:0:0 at node08] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed
[proxy:0:0 at node08] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:0 at node08] main (./pm/pmiserv/pmip.c:222): demux engine error waiting for event
[proxy:0:1 at node09] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed
[proxy:0:1 at node09] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:1 at node09] main (./pm/pmiserv/pmip.c:222): demux engine error waiting for event
[proxy:0:2 at node10] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed
[proxy:0:2 at node10] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:2 at node10] main (./pm/pmiserv/pmip.c:222): demux engine error waiting for event
[mpiexec at hpcserver] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:70): one of the processes terminated badly; aborting
[mpiexec at hpcserver] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion
[mpiexec at hpcserver] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:179): launcher returned error waiting for completion
[mpiexec at hpcserver] main (./ui/mpich/mpiexec.c:397): process manager error waiting for completion error2:[huochunfang at hpcserver test]$ mpirun_rsh -rsh -np 64 -hostfile ./hosts ~/bin/vasp
running on 64 nodes
distr: one band on 64 nodes, 1 groups
vasp.5.2.11 18Jan11 complex
POSCAR found : 2 types and 514 ions
LDA part: xc-table for Ceperly-Alder, standard interpolation
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ...( 16 )
WAVECAR not readMPI process (rank: 32) terminated unexpectedly on node02
Exit code -5 signaled from node02
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFA80 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF99F Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF8C0 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF97E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF96E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF7A2 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFAA2 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF997 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF862 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF9AF Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF93A Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF7EF Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000031634306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000316341D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000031634306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000316341D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000031634306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000316341D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
MPI process (rank: 43) terminated unexpectedly on node11
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF867 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF889 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFA39 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF82E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF99F Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1D339AC Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF9F3 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFA57 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF8CB Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF9F3 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF97E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF99F Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF83E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFAAF Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF80F Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF95C Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB8BF87B4 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF857 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039110306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000391101D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039DFA306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 00000039DFA1D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF97E Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF78D Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF8C0 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF84F Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECF8C0 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFAC2 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1ECFCB5 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libmkl_mc3.so 00002AAAB1D339A7 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000039110306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000391101D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
forrtl: error (69): process interrupted (SIGINT)
Image PC Routine Line Source
libc.so.6 00000034270306F7 Unknown Unknown Unknown
vasp 0000000000EACF22 Unknown Unknown Unknown
vasp 000000000057F100 Unknown Unknown Unknown
vasp 0000000000428490 Unknown Unknown Unknown
vasp 000000000040870C Unknown Unknown Unknown
libc.so.6 000000342701D994 Unknown Unknown Unknown
vasp 0000000000408619 Unknown Unknown Unknown
larry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20110707/65cb7a9c/attachment-0001.html
More information about the mvapich-discuss
mailing list