[mvapich-discuss] default envs for mpiexec

[Christian Guggenberger]

Hi Wei,

> Are you willing to change the code and recompile? I can send you a patch
> to disable it in code.
> 

this would be an option as well. I was thinking of preparing
mvapich2-1.0.2p1 in the near future, so I'd appreciate your patch.

As the problem with SRQ is only visible on em64t with pci-ex adapters
(25208), but not on Opteron with pci-x (23108), I could keep it enabled
for the latter.

You might probably be interested, so I'll describe the SRQ problem we
are seeing on em64t with 25208 adapters (OFED-1.2.5.5, SLES9 SP4):

Even simplest MPI programs occasionnaly hang in MPI_FINALIZE (even for
intra-node only communciations). We have not been able to track this
down further, but disabling SRQ helps. 

->   pthread_cond_wait,  FP=7fbfffea20
     ibv_cmd_destroy_srq, FP=7fbfffea80
     mthca_destroy_srq,  FP=7fbfffea90
     ibv_destroy_srq,    FP=7fbfffeaa0
     MPIDI_CH3I_RMDA_finalize, FP=7fbfffeb10
     MPIDI_CH3_Finalize, FP=7fbfffeb30
     MPID_Finalize,      FP=7fbfffeb50
     PMPI_Finalize,      FP=7fbfffeb90
     pmpi_finalize_,	 FP=7fbfffeba0

cheers.
 - Christian