[mvapich-discuss] Cannot utilize CPU core

[Krishna Chaitanya]

Morris,
            If a parallel job  is run with VIADEV_USE_AFFINITY=1, we try to
map the processes to cores during MPI_Init and the processes stay bound to
those cores for the rest of the time. We allow users to experiment with
various CPU mapping patterns by using the run-time variable :
VIADEV_CPU_MAPPING. You can use these variables with the mpirun_rsh command.
  You can find more details about these variables here : (
http://nowlab.cse.ohio-state.edu/mvapich-website/support/mvapich_user_guide-1.2rc1.html#x1-1340009.6.5).

           However, if the job is run with VIADEV_USE_AFFINITY=0, we let the
kernel take care of binding processes to cores.  In this case, the kernel
"can" move the processes in any fashion during the application execution,
which might not be good for the performance of applications.
           In your case, since you are trying to run multiple jobs on the
same node at the same time, it would be better if you set
VIADEV_USE_AFFINITY to 0.  Or,  you can also use VIADEV_CPU_MAPPING to bind
the processes of different jobs such that they do not compete for the same
set of cores. (This is assuming that you do have idle cores when you are
submitting the second job).
           Please let us know if this helps. Also, as Dr. Panda had
indicated in the last mail, we would recommend you to use the latest version
of MVAPICH2, as we now offer better CPU mapping techniques.

Thanks,
Krishna


On Tue, Apr 6, 2010 at 9:12 PM, MORRIS LAW <morris at hkbu.edu.hk> wrote:

> The 'affinity' problem is very new to me.  May I know exactly how I can
> control running of the jobs with VIADEV_USE_AFFINITY=0?
>
> Should I place the line in /etc/profile or in the mpirun script?
>
> Best Regards,
>
> --
> Morris
>
>
> ----- Original Message -----
> From: Dhabaleswar Panda <panda at cse.ohio-state.edu>
> Date: Tuesday, March 30, 2010 8:29 pm
> Subject: Re: [mvapich-discuss] Cannot utilize CPU core
> To: MORRIS LAW <morris at hkbu.edu.hk>
> Cc: mvapich-discuss at cse.ohio-state.edu
>
>
> > You are seeing the effect of `affinity' of processes to cores here.
> > Try to
> >  run your jobs with VIADEV_USE_AFFINITY=0. More details on this are
> >  available from MVAPICH 1.1 user guide at the following location:
> >
> >
> http://mvapich.cse.ohio-state.edu/support/mvapich_user_guide-1.1.html#x1-1340009.6.5
> >
> >  Let us know if the problem goes away with this option.
> >
> >  Also note that MVAPICH2 has many more flexible ways to bind processes
> > to
> >  cores. You can use the latest version of MVAPICH2 (1.4.1) to take
> >  advantage of these features.
> >
> >  DK
> >
> >
> >
> >  On Tue, 30 Mar 2010, MORRIS LAW wrote:
> >
> >  > Dear all,
> >  >
> >  > I am new to the discussion group.
> >  >
> >  > Recently I found a problem of running mvapich 1.1 on Gen2-IB device
> > (Qlogic 9120).  When subsequent run of mvapich job was delivered to
> > the nodes, the later jobs will not run on free CPU cores but will
> > compete with the current running CPU cores.  Thus the whole node
> > cannot be utilized.  I don't know where the problem is.  Is it related
> > to the IB switch or some parameter when I built the mvapich 1.1?
> >  >
> >  > I built mvapich 1.1 using gcc 4.1 on CentOS 5.3.  I have also built
> > another version of mvapich 1.1 using icc and ifort on the same CentOS
> > 5.3.  Both run jobs having similar problem.
> >  >
> >  > Would someone give me some hints to tackle the problem?
> >  >
> >  > Thanks in advance.
> >  >
> >  > --
> >  > Morris Law
> >  > HK Baptist University
> >  >
> >  > _______________________________________________
> >  > mvapich-discuss mailing list
> >  > mvapich-discuss at cse.ohio-state.edu
> >  > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
> >  >
> >
> >
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20100406/15c476e7/attachment.html