[mvapich-discuss] Segmentation fault in MPI_Reduce using MPI_IN_PLACE and root != 0
Akshay Venkatesh
akshay.v.3.14 at gmail.com
Tue May 19 11:24:39 EDT 2015
Hi Markus,
Can you send the exact call that you're making at the root and at the
non-root if it's not a hassle? The code snippet will rule out other things
before we conclude anything.
Thanks
On Tue, May 19, 2015 at 10:55 AM, Markus Geimer <mg1102 at web.de> wrote:
> Dear MVAPICH developers,
>
> While testing my MPI application with MVAPICH2 2.1, I ran
> into a segmentation fault. After an in-depth investigation
> I could boil down the issue to the attached minimal example.
>
> Calling MPI_Reduce using MPI_IN_PLACE and root != 0 multiple
> times causes the segfault. The example works for me with
> ITER <= 16, but fails with ITER >= 17. If root == 0, it
> works regardless of the number of iterations. Note that the
> example also works fine with MPICH 3.1 and Open MPI 1.8.3.
>
> The test system is running Debian 7 with kernel 3.2.65; the
> compiler used is GCC 4.8.3 (built from vanilla source).
> Please find some more detailed information below. If you
> need more -- or have a patch for me to try out -- please let
> me know.
>
> Thanks,
> Markus
>
> ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< -----
>
> $ mpiexec -n 2 ./a.out
> [host:mpi_rank_1][error_sighandler] Caught error: Segmentation fault
> (signal 11)
> [host:mpi_rank_1][print_backtrace] 0:
> /opt/mvapich2-2.1/lib/libmpi.so.12(print_backtrace+0x20) [0x7f553b6b404c]
> [host:mpi_rank_1][print_backtrace] 1:
> /opt/mvapich2-2.1/lib/libmpi.so.12(error_sighandler+0x77) [0x7f553b6b419e]
> [host:mpi_rank_1][print_backtrace] 2:
> /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f553ae331e0]
> [host:mpi_rank_1][print_backtrace] 3:
> /lib/x86_64-linux-gnu/libc.so.6(+0x12abd0) [0x7f553af2bbd0]
> [host:mpi_rank_1][print_backtrace] 4:
> /opt/mvapich2-2.1/lib/libmpi.so.12(+0x3f6857) [0x7f553b582857]
> [host:mpi_rank_1][print_backtrace] 5:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Localcopy+0x357) [0x7f553b5833b4]
> [host:mpi_rank_1][print_backtrace] 6:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_shmem_MV2+0x385)
> [0x7f553b2a3d50]
> [host:mpi_rank_1][print_backtrace] 7:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_two_level_helper_MV2+0x594)
> [0x7f553b2a57ec]
> [host:mpi_rank_1][print_backtrace] 8:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_index_tuned_intra_MV2+0x97b)
> [0x7f553b2a6af3]
> [host:mpi_rank_1][print_backtrace] 9:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_MV2+0x9f) [0x7f553b2a6ea0]
> [host:mpi_rank_1][print_backtrace] 10:
> /opt/mvapich2-2.1/lib/libmpi.so.12(MPIR_Reduce_impl+0x8c) [0x7f553b21534b]
> [host:mpi_rank_1][print_backtrace] 11:
> /opt/mvapich2-2.1/lib/libmpi.so.12(PMPI_Reduce+0x1709) [0x7f553b216b9b]
> [host:mpi_rank_1][print_backtrace] 12: ./a.out() [0x400838]
> [host:mpi_rank_1][print_backtrace] 13:
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f553ae1fead]
> [host:mpi_rank_1][print_backtrace] 14: ./a.out() [0x4006c9]
>
> $ mpiname -a
> MVAPICH2 2.1 Fri Apr 03 20:00:00 EDT 2015 ch3:mrail
>
> Compilation
> CC: gcc -g -O0
> CXX: g++ -g -O0
> F77: gfortran -L/lib -L/lib -g -O0
> FC: gfortran -g -O0
>
> Configuration
> --prefix=/opt/mvapich2-2.1 --enable-shared --enable-g=dbg --disable-fast
> --enable-debuginfo
>
> $ ibv_devinfo
> hca_id: qib0
> transport: InfiniBand (0)
> fw_ver: 0.0.0
> node_guid: 0011:7500:00ff:d76b
> sys_image_guid: 0011:7500:00ff:d76b
> vendor_id: 0x1175
> vendor_part_id: 16
> hw_ver: 0x2
> board_id: InfiniPath_QLE7140
> phys_port_cnt: 1
> port: 1
> state: PORT_ACTIVE (4)
> max_mtu: 4096 (5)
> active_mtu: 2048 (4)
> sm_lid: 1
> port_lid: 1
> port_lmc: 0x00
> link_layer: InfiniBand
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
--
-Akshay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150519/18c266dd/attachment-0001.html>
More information about the mvapich-discuss
mailing list