[mvapich-discuss] Newbie question: Running hybrid applications

[Sayantan Sur]

Hi Matt,

Thanks for sharing the problem. We'd be happy to assist you in this
matter. I am attaching a simple OpenMP and MPI "Hello World" program
along with this email. I have verified that it works well with 1.5.1
(I didn't check with 1.5). I would urge you to upgrade to 1.5.1, since
there are some bugfixes. The easiest is to simply grab a nightly
tarball.

The program is launched on several nodes, and it reports back with how
many processes were launched and how many threads are present on the
node. I am using PGI-10.5 as my compiler. Did you by chance miss
passing the equivalent "-mp" flag to your OpenMP build?

# This is a run with -mp passed to the compiler; behavior as expected,
two ranks, each with two threads

surs at wci2:~/tmp 11:56 AM>
~/mvapich2/mvapich2-1.5-2010-11-04-pgi/bin/mpicc -mp omp_mpi_hello.c
surs at wci2:~/tmp 11:56 AM>
~/mvapich2/mvapich2-1.5-2010-11-04-pgi/bin/mpirun_rsh -np 2 wci1 wci2
OMP_NUM_THREADS=2 MV2_ENABLE_AFFINITY=0 ./a.out
This is rank 1 of 2 reporting for duty
This is rank 0 of 2 reporting for duty
Hello World from thread 0 on rank 0
Hello World from thread 1 on rank 0
There are 2 threads on rank 0
Hello World from thread 0 on rank 1
Hello World from thread 1 on rank 1
There are 2 threads on rank 1

# This is a run without -mp to the compiler; this behavior is as you
report, two ranks, but with only one thread. i.e. OMP_NUM_THREADS=2 is
ignored


surs at wci2:~/tmp 11:57 AM>
~/mvapich2/mvapich2-1.5-2010-11-04-pgi/bin/mpicc omp_mpi_hello.c
surs at wci2:~/tmp 12:00 PM>
~/mvapich2/mvapich2-1.5-2010-11-04-pgi/bin/mpirun_rsh -np 2 wci1 wci2
OMP_NUM_THREADS=2 MV2_ENABLE_AFFINITY=0 ./a.out
This is rank 1 of 2 reporting for duty
Hello World from thread 0 on rank 1
There are 1 threads on rank 1
This is rank 0 of 2 reporting for duty
Hello World from thread 0 on rank 0
There are 1 threads on rank 0



On Fri, Nov 5, 2010 at 10:54 AM, Matt Foster <Matthew.Foster at noaa.gov> wrote:
> Hello, I am new to this list.  I have searched the Internet and the MVAPICH2
> User's Guide for an answer to this question, but I'm still having trouble.
>  I have more experience running strictly distributed applications, but am
> somewhat new to hybrid apps.
>
> The application in question is the WRF weather forecast model.  I have
> compiled it in hybrid (dm+sm) mode using the Intel compilers.  I am able to
> run it successfully in hybrid mode when compiled against OpenMPI, but with
> it compiled against MVAPICH2 1.5 I have trouble.  Basically, the correct
> number of processes launch, but either no threads are being launched or all
> threads are running on the same core as their parent process.  Our nodes
> have 8 cores (2x quad-core), and I am trying to run 4 processes with 2
> threads each.
>
> I have attempted to follow the guidance in the user's guide, but I must be
> missing something.  This is how I am attempting to launch the program...
>
> mpirun_rsh -np 16 --hostfile hostfile MV2_ENABLE_AFFINITY=0
> OMP_NUM_THREADS=2 real.exe
>
> I have also tried setting MV2_ENABLE_AFFINITY and OMP_NUM_THREADS in the
> environment before launching the program, but it made no difference.
>
> What am I doing wrong?
>
> Matt
>
> --
> Do not go where the path may lead; go instead where there is no path and
> leave a trail.
> -- Ralph Waldo Emerson
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>



-- 
Sayantan Sur

Research Scientist
Department of Computer Science
http://www.cse.ohio-state.edu/~surs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omp_mpi_hello.c
Type: application/octet-stream
Size: 844 bytes
Desc: not available
Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20101105/b192dca0/omp_mpi_hello.obj