[mvapich-discuss] MVAPICH2 with OpenMP

[Ben Truscott]

Dear Martin

I'm working with a configuration very much like yours. I wonder, did you 
remember to set IPATH_NO_CPUAFFINITY=1? Without this, the PSM library makes 
its own affinity settings that pre-empt those made by OpenMP.

Regards

Ben Truscott
School of Chemistry
University of Bristol

> Hi all,
>
> I am having a strange problem with MVAPICH2 - the ones that I build don't
> seem to run multi-threaded programs with OpenMP. I suspect it's some kind
> of a configuration issue on my end as I tried to run MVAPICH2 from the
> stock OFED distribution and the multithreading there is fine.
>
> Also, I have been building OpenMPI with fairly standard options and the
> OpenMP there works fine.
>
> Does someone have similar experience, or, is there some trick with
> MVAPICH2 build or running to get multiple OpenMP threads going?
>
> I configure as:
> configure --enable-romio --with-file-system=nfs+ufs  --with-device=ch3:psm
> with Intel or GNU compilers.
>
> Then run as:
> mpiexec.hydra -genv OMP_NUM_THREADS 4 -genv MV2_ENABLE_AFFINITY 0 -n 16
> -machinefile nodefile ./program
>
> This is all on RHEL 5.5 with the Qlogic or Mellanox IB adapters.
>
> Thanks,
> MC
>
> --
> Martin Cuma
> Center for High Performance Computing
> University of Utah