[mvapich-discuss] Bunch and Scatter CPU Binding policy

[Jonathan Perkins]

On Mon, Mar 19, 2012 at 12:21:08PM -0500, vaibhav dutt wrote:
> Hi,
> 
> The  MVAPICH2 user guide (
> http://mvapich.cse.ohio-state.edu/support/mvapich2-1.7_user_guide.pdf)
> provides
> the examples with figures for bunch and scatter CPU binding policies in
> case block rank placement  is used. I was wondering
> what would be the case for bunch and scatter policies, when cyclic rank
> placement is used. Can anybody help me with this.
> 
> Block rank placement ( For a 4 node cluster, each node having 8 cores)
> 
> Node 0 - Rank 0 to 7
> Node 1-  Rank 8 to 15
> Node 3-  Rank 16 to 23
> Node 4- Rank 24 to 31
> 
> 
> Cyclic Rank Placement
> 
> Node 0 - Ranks 0,4,8,12,16,20,24,28
> Node 1- Ranks 1,5,9,13,17,21,25,29
> Node 2- Ranks 2,6,10,14,18,22,26,30
> Node 3- Ranks 3,7,11,15,19,23,27,31

The mapping is the same if you consider local ranks.  It's just that the
local rank to global rank mapping will be different based on the host
file contents.

In the block mapping case:
node0: rank 0 to 7 -> i (where i = 0 to 7)
node1: rank 0 to 7 -> i + 7
node2: rank 0 to 7 -> i + 14
node3: rank 0 to 7 -> i + 21

cyclic mapping case:
node0: rank 0 to 7 -> 4i (where i = 0 to 7)
node1: rank 0 to 7 -> 4i + 1 
node2: rank 0 to 7 -> 4i + 2
node3: rank 0 to 7 -> 4i + 3

You can use those mapping formulas to determine what global rank
corresponds to the local rank and since you already know where the local
ranks are bound this should be all you need.

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo