[mvapich-discuss] MPI_Comm_spawn always uses just 1 host

[Jonathan Perkins]

On 02/10/2012 10:01 AM, Rutger Hofman wrote:
> Good afternoon list,
>
> I want to control the hosts on which processes are spawned by 
> MPI_Comm_spawn.
>
> But in our cluster (DAS4 http://www.das4.cs.vu.nl RHEL6 w/ QDR 
> Mellanox Infiniband), a call to MPI_Comm_spawn() does spawn processes, 
> but they always all run on one single host (something like rank 
> (n+1)%size when I do -np n).
>
> Behaviour is the same whatever MPI_Info properties I specify to the 
> MPI_Comm_spawn() call:
>   "host" "node048"
>   "hostfile" "hostfile"
>   "hosts" "node048 node049 node050"
> etc etc. None of them generates an error; does that mean they are 
> actually valid?
>
> My mpirun_rsh incantation is something like
> $ mpirun_rsh -ssh -np 3 -hostfile hostfile \
>       MV2_SUPPORT_DPM=1 ./spawn ./slave
> and, as stated, slave instances do run, only I have no control where.
>
> I tried the mvapich2-1.6 packaged with our cluster, and a freshly 
> downloaded mvapich2-1.8a2. The behaviour is the same.
>
> In the source code I see mention of -spawnfile. This option is 
> rejected by mpirun_rsh.
>
> So, my question: how can I specify the set of hosts that processes 
> should be spawned on?
>

Hello, in order to control which processes are spawned on which hosts 
you can modify your hostfile.

In your mpirun_rsh invocation I see that you start out with 3 
processes.  If your hostfile contains

nodeA
nodeB
nodeC
nodeD
nodeE

then the first spawned process will be on nodeD and the next spawned 
process will be on nodeE.  Let me know if this helps.

--
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo