Why do I need MV2_CPU_MAPPING? (was :Re: [mvapich-discuss] MV2_CPU_MAPPING doesn't work on all the nodes)

[Devendar Bureddy]

Hi Craig

We also do not recommend to use MV2_CPU_MAPPING explicitly until
unless if user want some specific layout of cpu affinity across all
the nodes and do not want to modify hostfile.  MVAPICH2 also
automatically decides the affinity internally and users should not
worry much about it. There is also some high level abstract run-time
options (MV2_CPU_BINDING_POLICY,  MV2_CPU_BINDING_LEVEL)  to change
the default affinity settings if required. There is certainly no need
to explicitly specify affinity with MV2_CPU_MAPPING

-Devendar

On Mon, Jun 25, 2012 at 1:45 PM, Craig Tierney <craig.tierney at noaa.gov> wrote:
> Mvapich2 development team,
>
> This thread got me thinking again regarding one issue I have always had with
> process affinity and mvapich2.  Why do even need to use MV2_CPU_MAPPING?
>  For 99.9% of the cases that I could ever think of for MPI applications,
> there is one correct way to layout processes.  If I have the machine file, I
> would know what to do.  If I have OMP_NUM_THREADS, I would know what to do.
>  If for some reason I need a different mapping on different hosts, I would
> be able to deduce the correct layout from the machine file (which we do need
> to do and MV2_CPU_MAPPING cannot support this).
>
> The only rule I can think of is distribute process evenly across sockets,
> specify memory affinity should try and stay local.  The only other choice
> there is should you use a scatter or compact distribution.  That could be
> specified by a variable.
>
> Of course there are many things that could complicate my "simple" view of
> the world.  Besides launching multiple jobs on a node, what are they and how
> often do they really happen? For multiple jobs, you would probably through
> out affinity because there would be no consistent way by a user to ensure
> that two jobs are not stepping on each other (you would need a batch system
> to do this).
>
> Other MPI stacks do not require the use of something like MV2_CPU_MAPPING
> (Intel MPI and SGI MPI have methods built in).  Honestly most of my users
> wouldn't get the use of MV2_CPU_MAPPING correct.  This is something the
> system should be doing for them and not figure it out for themselves.
>
> Thanks,
> Craig
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss



-- 
Devendar