[mvapich-discuss] How to verify that hwloc is enabled and used

[Craig Tierney]

Jonathan,

Thanks for the tips.  My original testing was set to with
POLICY=scatter and LEVEL=socket.  I believe this is correct for
standard OpenMP/MPI hybrids.  However, when that wasn't working I
tried lots of different things to get it to work.

Craig

On Thu, Sep 6, 2012 at 3:37 PM, Jonathan Perkins
<perkinjo at cse.ohio-state.edu> wrote:
> On Thu, Sep 06, 2012 at 02:52:26PM -0600, Craig Tierney wrote:
>> I am trying to run some openmp/mpi hybrid applications and the
>> performance is not good.  The first place to blame is affinity (since
>> the same code with Intel MPI and SGI MPI behave as I would want).  I
>> have run taskset -p $PID on some running processes and I see that the
>> mask is ffff, indicating that those processes have no affinity.  I
>> have tried lots of combinations of MV2_CPU_MAPPING,
>> MV2_CPU_BINDING_POLICY, MV2_CPU_BINDING_LEVEL and I get no indication
>> of any affinity.  I am looking back at my build, I see the message "no
>> suitable hwloc found, using internal version".  But is there a way to
>> verify that that is even working?
>
> Hi Craig,
> I suggest setting MV2_SHOW_CPU_BINDING=1 so that the processes will
> report their binding.
>
> Since you're running a hybrid application it may take a little work to
> determine the best MAPPING to use.  I think you would like to set
> MV2_CPU_BINDING_LEVEL to either socket or numanode and maybe the
> MV2_CPU_BINDING_POLICY=scatter.  Depending on the number of MPI
> processes and OpenMP processes you're using will determine the exact
> mapping you'll want.
>
> Please take a look at the following url for more information.
> http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.8.html#x1-530006.3.1
>
> These environment variables are available with MVAPICH2-1.8.
>
> --
> Jonathan Perkins
> http://www.cse.ohio-state.edu/~perkinjo