[mvapich-discuss] Segmentation fault in MPI_Reduce using MPI_IN_PLACE and root != 0

[Akshay Venkatesh]

Apologies. I missed the attachment.

On Tue, May 19, 2015 at 11:24 AM, Akshay Venkatesh <akshay.v.3.14 at gmail.com>
wrote:

> Hi Markus,
>
> Can you send the exact call that you're making at the root and at the
> non-root if it's not a hassle? The code snippet will rule out other things
> before we conclude anything.
>
> Thanks
>
>
> On Tue, May 19, 2015 at 10:55 AM, Markus Geimer <mg1102 at web.de> wrote:
>
>> Dear MVAPICH developers,
>>
>> While testing my MPI application with MVAPICH2 2.1, I ran
>> into a segmentation fault.  After an in-depth investigation
>> I could boil down the issue to the attached minimal example.
>>
>> Calling MPI_Reduce using MPI_IN_PLACE and root != 0 multiple
>> times causes the segfault.  The example works for me with
>> ITER <= 16, but fails with ITER >= 17.  If root == 0, it
>> works regardless of the number of iterations. Note that the
>> example also works fine with MPICH 3.1 and Open MPI 1.8.3.
>>
>> The test system is running Debian 7 with kernel 3.2.65; the
>> compiler used is GCC 4.8.3 (built from vanilla source).
>> Please find some more detailed information below.  If you
>> need more -- or have a patch for me to try out -- please let
>> me know.
>>
>> Thanks,
>> Markus
>>
>> ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< -----
>>
>> $ mpiexec -n 2 ./a.out
>> [host:mpi_rank_1][error_sighandler] Caught error: Segmentation fault
>> (signal 11)
>> [host:mpi_rank_1][print_backtrace]   0:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(print_backtrace+0x20) [0x7f553b6b404c]
>> [host:mpi_rank_1][print_backtrace]   1:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(error_sighandler+0x77) [0x7f553b6b419e]
>> [host:mpi_rank_1][print_backtrace]   2:
>> /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f553ae331e0]
>> [host:mpi_rank_1][print_backtrace]   3:
>> /lib/x86_64-linux-gnu/libc.so.6(+0x12abd0) [0x7f553af2bbd0]
>> [host:mpi_rank_1][print_backtrace]   4:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(+0x3f6857) [0x7f553b582857]
>> [host:mpi_rank_1][print_backtrace]   5:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Localcopy+0x357) [0x7f553b5833b4]
>> [host:mpi_rank_1][print_backtrace]   6:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_shmem_MV2+0x385)
>> [0x7f553b2a3d50]
>> [host:mpi_rank_1][print_backtrace]   7:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_two_level_helper_MV2+0x594)
>> [0x7f553b2a57ec]
>> [host:mpi_rank_1][print_backtrace]   8:
>>
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_index_tuned_intra_MV2+0x97b)
>> [0x7f553b2a6af3]
>> [host:mpi_rank_1][print_backtrace]   9:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_MV2+0x9f) [0x7f553b2a6ea0]
>> [host:mpi_rank_1][print_backtrace]  10:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_impl+0x8c) [0x7f553b21534b]
>> [host:mpi_rank_1][print_backtrace]  11:
>> /opt/mvapich2-2.1/lib/libmpi.so.12(PMPI_Reduce+0x1709) [0x7f553b216b9b]
>> [host:mpi_rank_1][print_backtrace]  12: ./a.out() [0x400838]
>> [host:mpi_rank_1][print_backtrace]  13:
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f553ae1fead]
>> [host:mpi_rank_1][print_backtrace]  14: ./a.out() [0x4006c9]
>>
>> $ mpiname -a
>> MVAPICH2 2.1 Fri Apr 03 20:00:00 EDT 2015 ch3:mrail
>>
>> Compilation
>> CC: gcc    -g -O0
>> CXX: g++   -g -O0
>> F77: gfortran -L/lib -L/lib   -g -O0
>> FC: gfortran   -g -O0
>>
>> Configuration
>> --prefix=/opt/mvapich2-2.1 --enable-shared --enable-g=dbg --disable-fast
>> --enable-debuginfo
>>
>> $ ibv_devinfo
>> hca_id: qib0
>>         transport:                      InfiniBand (0)
>>         fw_ver:                         0.0.0
>>         node_guid:                      0011:7500:00ff:d76b
>>         sys_image_guid:                 0011:7500:00ff:d76b
>>         vendor_id:                      0x1175
>>         vendor_part_id:                 16
>>         hw_ver:                         0x2
>>         board_id:                       InfiniPath_QLE7140
>>         phys_port_cnt:                  1
>>                 port:   1
>>                         state:                  PORT_ACTIVE (4)
>>                         max_mtu:                4096 (5)
>>                         active_mtu:             2048 (4)
>>                         sm_lid:                 1
>>                         port_lid:               1
>>                         port_lmc:               0x00
>>                         link_layer:             InfiniBand
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>
>
> --
> -Akshay
>



-- 
-Akshay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150519/574b39b3/attachment.html>