[mvapich-discuss] Seg-fault in MPI_Init with GCC's address sanitizer

[Deakin, Michael]

Hi,


I'm trying to debug a program (E3SM) using address sanitizer, and am getting seg-faults in mpi-init. It looks like it's trying to read from or write to a null pointer. in the log I've attached.

This issue seems to be dependent on other environment settings that I don't understand, as I'm able to run the same program with the address sanitizer and different settings without seg-faulting.

Finally, I'm reasonably certain this isn't an issue in E3SM as I'm able to run to completion with openmpi and the address sanitizer.

Would anyone here have ideas on why this is occurring? If you need more information to debug this and hopefully fix it in a future version, please let me know.


$ mpiname -a
MVAPICH2 2.2b Mon Nov 12 20:00:00 EST 2015 ch3:mrail

Compilation
CC: /blues/gpfs/home/software/spack/lib/spack/env/gcc/gcc    -DNDEBUG -DNVALGRIND -O2
CXX: /blues/gpfs/home/software/spack/lib/spack/env/gcc/g++   -DNDEBUG -DNVALGRIND -O2
F77: /blues/gpfs/home/software/spack/lib/spack/env/gcc/gfortran -L/lib -L/lib   -O2
FC: /blues/gpfs/home/software/spack/lib/spack/env/gcc/gfortran   -O2

Configuration
--prefix=/blues/gpfs/home/software/spack/opt/spack/linux-x86_64/gcc-5.3.0/mvapich2-2.2b-yzk3up5jlbhoyyfgu7tuezfbh2dhee6f --enable-shared --enable-romio --disable-silent-rules --enable-fortran=all --enable-fast=all --with-device=ch3:mrail --with-rdma=gen2


Launch command:

$ mpiexec  -l  -n 144  /lcrc/group/acme/mdeakin/acme_scratch/f_sanitize_test/bld/e3sm.exe  >> e3sm.log


Environment variables (should be the same in both runs):

MV2_ENABLE_AFFINITY=1
MV2_USE_SHARED_MEM=1
MV2_SMP_USE_CMA=1?

MV2_SHOW_CPU_BINDING=1


Thanks,

Michael Deakin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20171211/7b00d457/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acme.F1850C5AV1C-04P2.ne4_ne4.log.128.171211-120502.txt
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20171211/7b00d457/attachment.txt>