[mvapich-discuss] MVAPICH2 Invalid communicator errors

[Mark Potts]

Hi,
    Taking my first steps with MVAPICH2-0.9.8 from OFED 1.2.

    I am able to build and run apps using gcc to build MVAPICH2 and apps.

    I am also (apparently) successfully building MVAPICH2 and simple C
    code apps. with an icc Intel 9.1 compiler.  However, any
    built-with-Intel app that I attempt to run fails with an output
    similar to that shown at the bottom.  The same apps build and run
    quite nicely using gcc.  The failure occurs at the very beginning
    of the pgm, after:

       MPI_Init (&argc, &argv);

    and before the completion of the next statement:

       MPI_Comm_rank (MPI_COMM_WORLD, &rank);

    Since we need both gcc and Intel versions of MVAPICH2, I would
    appreciate any help understanding what is going wrong during
    startup of these apps.

    The output from mpich2version is:
Version:           MVAPICH2-0.9.8
Device:            osu_ch3:mrail
Configure Options: --prefix=/usr/mpi/mvapich2-0.9.8-12/intel 
--with-device=osu_ch3:mrail --with-rdma=gen2 --with-pm=mpd 
--enable-romio --enable-sharedlibs=gcc --without-mpe


   Typical failed mpiexec output:

  mpiexec -machinefile hostdews -n 3 ./mpisanity_mv2
Fatal error in MPI_Comm_rank: Invalid communicator, error stack:
MPI_Comm_rank(105): MPI_Comm_rank(comm=0x1, rank=0x7fff4fb01cd8) failed
MPI_Comm_rank(64).: Invalid communicatorrank 2 in job 1  dewberry4_32925 
   caused collective abort of all ranks
   exit status of rank 2: killed by signal 9
Fatal error in MPI_Comm_rank: Invalid communicator, error stack:
MPI_Comm_rank(105): MPI_Comm_rank(comm=0x1, rank=0x7fffddc500c8) failed
MPI_Comm_rank(64).: Invalid communicatorrank 1 in job 1  dewberry4_32925 
   caused collective abort of all ranks
   exit status of rank 1: killed by signal 9
Fatal error in MPI_Comm_rank: Invalid communicator, error stack:
MPI_Comm_rank(105): MPI_Comm_rank(comm=0x1, rank=0x7fff13581138) failed
MPI_Comm_rank(64).: Invalid communicatorrank 0 in job 1  dewberry4_32925 
   caused collective abort of all ranks
   exit status of rank 0: killed by signal 9

    regards,
-- 
***********************************
 >> Mark J. Potts, PhD
 >>
 >> HPC Applications Inc.
 >> phone: 410-992-8360 Bus
 >>        410-313-9318 Home
 >>        443-418-4375 Cell
 >> email: potts at hpcapplications.com
 >>        potts at excray.com
***********************************