[mvapich-discuss] MVAPICH2 with OpenMP

[Martin Cuma]

Hi Sayantan,

thanks for looking into this.

The problem is - I can't run in mixed MPI-OpenMP mode. I have programs 
that use MPI for coarse grained parallelization, and, to further 
parallelize  loops within each MPI process, I use OpenMP threads.

When I build these programs with MVAPICH2, I am unable to use the OpenMP 
threads - no matter how I set the number OpenMP threads to (via 
OMP_NUM_THREADS environment variable or via omp_set_num_threads function), 
it always just runs with 1 thread.

When I compile the programs with my own build of MPICH2, it works fine 
with multiple threads. I also noticed that OFED supplied MVAPICH2 works 
OK. So, as I said, I suspect it may have to do with either the environment 
we build MVAPICH2 in or the configuration flags we use.

I have always had this issue, but, just today I have it with the latest 
MVAPICH2 1.6rc3.

Thanks,
MC

On Tue, 1 Mar 2011, Sayantan Sur wrote:

> Hi Martin,
>
> We will look into the problem. Can you please mention what exactly the
> problem is? Also, if you could mention the MVAPICH2 version you are
> using, it will be good.
>
> Thanks.
>
> On Tue, Mar 1, 2011 at 5:03 PM, Martin Cuma <martin.cuma at utah.edu> wrote:
>> Hi all,
>>
>> I am having a strange problem with MVAPICH2 - the ones that I build don't
>> seem to run multi-threaded programs with OpenMP. I suspect it's some kind of
>> a configuration issue on my end as I tried to run MVAPICH2 from the stock
>> OFED distribution and the multithreading there is fine.
>>
>> Also, I have been building OpenMPI with fairly standard options and the
>> OpenMP there works fine.
>>
>> Does someone have similar experience, or, is there some trick with MVAPICH2
>> build or running to get multiple OpenMP threads going?
>>
>> I configure as:
>> configure --enable-romio --with-file-system=nfs+ufs  --with-device=ch3:psm
>> with Intel or GNU compilers.
>>
>> Then run as:
>> mpiexec.hydra -genv OMP_NUM_THREADS 4 -genv MV2_ENABLE_AFFINITY 0 -n 16
>> -machinefile nodefile ./program
>>
>> This is all on RHEL 5.5 with the Qlogic or Mellanox IB adapters.
>>
>> Thanks,
>> MC
>>
>> --
>> Martin Cuma
>> Center for High Performance Computing
>> University of Utah
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>
>
>
>

-- 
Martin Cuma
Center for High Performance Computing
University of Utah