[mvapich-discuss] MVAPICH
Mouhamad Al-Sayed-Ali
Mouhamad.Al-Sayed-Ali at u-bourgogne.fr
Thu Apr 14 04:01:19 EDT 2011
Hello all,
I have checked the backtrace, but I did not understand where is the problem ?
I have obtanied the following error:
------
OASIS3 environment variable :
OASIS3DEBUGLEVEL environment variable :
LOASIS3,IOASIS3LVL,IOASIS3DEBUGLVL = F 0 0
LOASIS3,IOASIS3LVL,IOASIS3DEBUGLVL = F 0 0
MPL_INIT : LMPLUSERCOMM not used
Communicator : ********
MPL_INIT : LMPLUSERCOMM not used
Communicator : ********
MPL_INIT : LMPLUSERCOMM not used
MPL_INIT : LMPLUSERCOMM not used
Communicator : ********
Communicator : ********
signal_drhook(SIGABRT=6): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGBUS=7): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGSEGV=11): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGILL=4): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGSTKFLT=16): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGFPE=8): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGTRAP=5): New handler installed at 0x171d73f; old
preserved at 0x0
MPL_GROUPS_CREATE: MPI_CART_CREATE 12 FROM PROCESSOR 2
signal_drhook(SIGINT=2): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGQUIT=3): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGTERM=15): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGXCPU=24): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGSYS=31): New handler installed at 0x171d73f; old
preserved at 0x0
Invalid argument in process 2
MPL_GROUPS_CREATE: MPI_CART_CREATE 12 FROM PROCESSOR 1
Invalid argument in process 1
[myproc#2,tid#1,pid#-1,signal#11(SIGSEGV)]: Received signal :: 0MB
(heap), 0MB (rss), 0MB (stack), 0 (paging), nsigs 1, time 0.00
[myproc#1,tid#1,pid#-1,signal#11(SIGSEGV)]: Received signal :: 0MB
(heap), 0MB (rss), 0MB (stack), 0 (paging), nsigs 1, time 0.00
Activating SIGALRM=14 and calling alarm(10), time = 0.00
JSETSIG: sl->active = 0
Activating SIGALRM=14 and calling alarm(10), time = 0.00
JSETSIG: sl->active = 0
signal_drhook(SIGALRM=14): New handler installed at 0x171d73f; old
preserved at 0x0
signal_drhook(SIGALRM=14): New handler installed at 0x171d73f; old
preserved at 0x0
tid#1 starting drhook traceback, time = 0.00
tid#1 starting drhook traceback, time = 0.00
tid#1 starting sigdump traceback, time = 0.00
[gdb__sigdump] : Received signal#11(SIGSEGV), pid=-1
tid#1 starting sigdump traceback, time = 0.00
[gdb__sigdump] : Received signal#11(SIGSEGV), pid=-1
[LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
[LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:92 :
ARPCLIM-mvapich-gcc450-debug3 [0x1782106]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:826 :
ARPCLIM-mvapich-gcc450-debug3 [0x171d724]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:1003 :
ARPCLIM-mvapich-gcc450-debug3 [0x171dc2f]
<Unknown> :
libpthread.so.0 [0x30e320de70]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x19e7bf2]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x19be54b]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_groups.F90:65 :
ARPCLIM-mvapich-gcc450-debug3 [0x1753684]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sumpini.F90:274 :
ARPCLIM-mvapich-gcc450-debug3 [0x52325f]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/su0yoma.F90:163 :
ARPCLIM-mvapich-gcc450-debug3 [0x41419a]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/cnt0.F90:143 :
ARPCLIM-mvapich-gcc450-debug3 [0x407fb8]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/master.F90:35 :
ARPCLIM-mvapich-gcc450-debug3 [0x407d41]
<Unknown> :
libc.so.6(__libc_start_main+0xf4) [0x30e261d8b4]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x407c09]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:92 :
ARPCLIM-mvapich-gcc450-debug3 [0x1782106]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:826 :
ARPCLIM-mvapich-gcc450-debug3 [0x171d724]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:1003 :
ARPCLIM-mvapich-gcc450-debug3 [0x171dc2f]
<Unknown> :
libpthread.so.0 [0x30e320de70]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x19e7bf2]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x19be54b]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_groups.F90:65 :
ARPCLIM-mvapich-gcc450-debug3 [0x1753684]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sumpini.F90:274 :
ARPCLIM-mvapich-gcc450-debug3 [0x52325f]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/su0yoma.F90:163 :
ARPCLIM-mvapich-gcc450-debug3 [0x41419a]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/cnt0.F90:143 :
ARPCLIM-mvapich-gcc450-debug3 [0x407fb8]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/master.F90:35 :
ARPCLIM-mvapich-gcc450-debug3 [0x407d41]
<Unknown> :
libc.so.6(__libc_start_main+0xf4) [0x30e261d8b4]
<Unknown> :
ARPCLIM-mvapich-gcc450-debug3 [0x407c09]
[LinuxTraceBack] : End of backtrace(s)
Done tracebacks, calling exit with sig=11, time = 0.10
ABOR1 CALLED
Dr.Hook calls ABOR1 ...
ABORT! 2 Dr.Hook calls ABOR1 ...
[LinuxTraceBack] : End of backtrace(s)
Done tracebacks, calling exit with sig=11, time = 0.10
ABOR1 CALLED
Dr.Hook calls ABOR1 ...
ABORT! 1 Dr.Hook calls ABOR1 ...
MPL_ABORT: CALLED FROM PROCESSOR 1 THRD 1
MPL_ABORT: THRD 1 Dr.Hook calls ABOR1 ...
MPL_ABORT: CALLED FROM PROCESSOR 2 THRD 1
MPL_ABORT: THRD 1 Dr.Hook calls ABOR1 ...
SDL_TRACEBACK: Calling LINUX_TRBK, THRD = 1
SDL_TRACEBACK: Calling LINUX_TRBK, THRD = 1
[LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
[LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:92 :
ARPCLIM-mvapich-gcc450-debug3 [0x1782106]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:172 :
ARPCLIM-mvapich-gcc450-debug3 [0x178244e]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sdl_module.F90:71 :
ARPCLIM-mvapich-gcc450-debug3 [0x17280b0]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_abort_mod.F90:36
: ARPCLIM-mvapich-gcc450-debug3 [0x1710927]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/abor1.F90:41
: ARPCLIM-mvapich-gcc450-debug3 [0x1718d72]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:1007
: ARPCLIM-mvapich-gcc450-debug3 [0x171dc90]
<Unknown>
: libpthread.so.0 [0x30e320de70]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x19e7bf2]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x19be54b]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_groups.F90:65
: ARPCLIM-mvapich-gcc450-debug3 [0x1753684]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sumpini.F90:274
: ARPCLIM-mvapich-gcc450-debug3 [0x52325f]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/su0yoma.F90:163
: ARPCLIM-mvapich-gcc450-debug3 [0x41419a]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/cnt0.F90:143
: ARPCLIM-mvapich-gcc450-debug3 [0x407fb8]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/master.F90:35
: ARPCLIM-mvapich-gcc450-debug3 [0x407d41]
<Unknown>
: libc.so.6(__libc_start_main+0xf4) [0x30e261d8b4]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x407c09]
[LinuxTraceBack] : End of backtrace(s)
SDL_TRACEBACK: Done LINUX_TRBK, THRD = 1
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:92 :
ARPCLIM-mvapich-gcc450-debug3 [0x1782106]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/linuxtrbk.c:172 :
ARPCLIM-mvapich-gcc450-debug3 [0x178244e]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sdl_module.F90:71 :
ARPCLIM-mvapich-gcc450-debug3 [0x17280b0]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_abort_mod.F90:36
: ARPCLIM-mvapich-gcc450-debug3 [0x1710927]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/abor1.F90:41
: ARPCLIM-mvapich-gcc450-debug3 [0x1718d72]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/drhook.c:1007
: ARPCLIM-mvapich-gcc450-debug3 [0x171dc90]
<Unknown>
: libpthread.so.0 [0x30e320de70]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x19e7bf2]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x19be54b]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/mpl_groups.F90:65
: ARPCLIM-mvapich-gcc450-debug3 [0x1753684]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/sumpini.F90:274
: ARPCLIM-mvapich-gcc450-debug3 [0x52325f]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/su0yoma.F90:163
: ARPCLIM-mvapich-gcc450-debug3 [0x41419a]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/cnt0.F90:143
: ARPCLIM-mvapich-gcc450-debug3 [0x407fb8]
/tmp/tmp.drebeix2/gmktmp.26225/Pcplpack/dirwork/master.F90:35
: ARPCLIM-mvapich-gcc450-debug3 [0x407d41]
<Unknown>
: libc.so.6(__libc_start_main+0xf4) [0x30e261d8b4]
<Unknown>
: ARPCLIM-mvapich-gcc450-debug3 [0x407c09]
[LinuxTraceBack] : End of backtrace(s)
SDL_TRACEBACK: Done LINUX_TRBK, THRD = 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
=====================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= EXIT CODE: 256
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
=====================================================================================
---------
Many thanks for your help,
MOuhamad Al sayed ali
--------
Jonathan Perkins <perkinjo at cse.ohio-state.edu> a écrit :
> Hello. It looks like there is a segmentation fault according to your
> message. Have you checked the backtrace to see where this is
> originating from? This information along with the version of mvapich
> or mvapich2 may help narrow down the cause of this issue.
>
> You may also want to verify that simple applications such as the OSU
> Micro Benchmarks are able to run on your system to help rule out
> installation issues.
>
> Please see http://mvapich.cse.ohio-state.edu/benchmarks/ for more
> information on downloading and using the benchmarks.
>
> On Wed, Apr 13, 2011 at 10:08 AM, Mouhamad Al-Sayed-Ali
> <Mouhamad.Al-Sayed-Ali at u-bourgogne.fr> wrote:
>> Dear all,
>>
>>
>> I have trying to run a binary ARPCLIM-mvapich-gcc450-debug3 (compiled by
>> MVAPICH) using
>>
>> mpirun -np 2 -machinefile file ARPCLIM-mvapich-gcc450-debug3
>>
>> But, I get the following error.
>>
>> --------
>> OASIS3 environment variable :
>> OASIS3DEBUGLEVEL environment variable :
>> LOASIS3,IOASIS3LVL,IOASIS3DEBUGLVL = F 0 0
>> MPL_INIT : LMPLUSERCOMM not used
>> Communicator : ********
>> OASIS3 environment variable :
>> OASIS3DEBUGLEVEL environment variable :
>> LOASIS3,IOASIS3LVL,IOASIS3DEBUGLVL = F 0 0
>> MPL_INIT : LMPLUSERCOMM not used
>> Communicator : ********
>> MPL_INIT : LMPLUSERCOMM not used
>> Communicator : ********
>> MPL_INIT : LMPLUSERCOMM not used
>> Communicator : ********
>> signal_drhook(SIGABRT=6): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGBUS=7): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGSEGV=11): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGILL=4): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGSTKFLT=16): New handler installed at 0x171d73f; old
>> preserved at 0x0
>> signal_drhook(SIGFPE=8): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGTRAP=5): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGINT=2): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGQUIT=3): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGTERM=15): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGXCPU=24): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> signal_drhook(SIGSYS=31): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> MPL_BUFFER_METHOD: 2 128000000
>> MPL_BUFFER_METHOD: 2 128000000
>> [myproc#2,tid#1,pid#-1,signal#11(SIGSEGV)]: Received signal :: 0MB (heap),
>> 0MB (rss), 0MB (stack), 0 (paging), nsigs 1, time 0.00
>> Activating SIGALRM=14 and calling alarm(10), time = 0.00
>> JSETSIG: sl->active = 0
>> signal_drhook(SIGALRM=14): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> tid#1 starting drhook traceback, time = 0.00
>> tid#1 starting sigdump traceback, time = 0.00
>> [gdb__sigdump] : Received signal#11(SIGSEGV), pid=-1
>> [LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
>> [myproc#1,tid#1,pid#-1,signal#11(SIGSEGV)]: Received signal :: 0MB (heap),
>> 0MB (rss), 0MB (stack), 0 (paging), nsigs 1, time 0.00
>> Activating SIGALRM=14 and calling alarm(10), time = 0.00
>> JSETSIG: sl->active = 0
>> signal_drhook(SIGALRM=14): New handler installed at 0x171d73f; old preserved
>> at 0x0
>> tid#1 starting drhook traceback, time = 0.00
>> tid#1 starting sigdump traceback, time = 0.00
>> [gdb__sigdump] : Received signal#11(SIGSEGV), pid=-1
>> [LinuxTraceBack]: Backtrace(s) for program 'ARPCLIM-mvapich-gcc450-debug3' :
>> [myproc#2,tid#1,pid#-1,signal#14(SIGALRM)]: Received signal :: 0MB (heap),
>> 0MB (rss), 0MB (stack), 0 (paging), nsigs 2, time 10.00
>>
>> =====================================================================================
>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> = EXIT CODE: 9
>> = CLEANING UP REMAINING PROCESSES
>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
>> =====================================================================================
>> APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9)
>> -----------------------------
>>
>>
>> Can anyone help me, please ?
>>
>>
>> Many thanks
>>
>> Mouhamad Al sayed Ali
>>
>>
>> post-doctoral in applied mathematics
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>
>
>
> --
> Jonathan Perkins
> http://www.cse.ohio-state.edu/~perkinjo
>
>
More information about the mvapich-discuss
mailing list