[mvapich-discuss] Specifying multiple HCAs in hostfile

[Jonathan Perkins]

Okay.  We'll investigate the option of updating the hostfile format to
support multiple HCAs.  Thanks for bringing this up.

On Tue, Mar 10, 2015 at 8:43 AM Dalai, Santak <Santak.Dalai at kla-tencor.com>
wrote:

>  Yes, that’s what I have been doing till now. In my case, I am using many
> processes and hence specifying the list of HCAs in the command-line crowds
> it. I was thinking if I can do that using a hostfile and keep the
> command-line clean and readable.
>
>
>
> Thanks,
>
> Santak Dalai
>
>
>
> *From:* Jonathan Perkins [mailto:perkinjo at cse.ohio-state.edu]
> *Sent:* Tuesday, March 10, 2015 6:02 PM
> *To:* Dalai, Santak; mvapich-discuss at cse.ohio-state.edu
> *Subject:* Re: [mvapich-discuss] Specifying multiple HCAs in hostfile
>
>
>
> I'm sorry, but at this time the hostfile format doesn't allow for multiple
> HCAs to be specified.  Perhaps you can try using MPMD if you need to have
> each process to have distinct HCAs specified by having the executable be
> something like "env MV2_IBA_HCA=hcalist ./mpi-app".
>
>
>
> Please look for Job Launch Using MPMD section available at
> http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.1rc1-userguide.html#x1-240005.2.1
> .
>
>
>
> I hope this helps.  Let us know if you have any further questions.
>
>
>
> On Tue, Mar 10, 2015 at 8:14 AM Dalai, Santak <Santak.Dalai at kla-tencor.com>
> wrote:
>
>  Hello all,
>
>
>
> I saw the format of hostfile in the userguide and it says a way to specify
> the host, the ranks and the HCA.
>
>
>
> Hostfile Format
>
> The mpirun_rsh hostfile format allows for users to specify hostnames, one
> per line, optionally with a multiplier, and HCA specification.
>
> The multiplier allows you to save typing by allowing you to specify
> blocked distribution of MPI ranks using one line per hostname. The HCA
> specification allows you to force an MPI rank to use a particular HCA.
>
> The optional components are delimited by a ‘:’. Comments and empty lines
> are also allowed. Comments start with ‘#’ and continue to the next newline.
>
>
>
> Sample hostfile
>
>
>
> $ cat hosts
>
> # sample hostfile for mpirun_rsh
>
> host1           # rank 0 will be placed on host1
>
> host2:2         # rank 1 and 2 will be placed on host 2
>
> host3:hca1      # rank 3 will be on host3 and will used hca1
>
> host4:4:hca2    # ranks 4 through 7 will be on host4 and use hca2
>
>
>
> # if the number of processes specified for this job is greater than 8
>
> # then the additional ranks will be assigned to the hosts in a cyclic
>
> # fashion.  For example, rank 8 will be on host1 and ranks 9 and 10 will
>
> # be on host2.
>
>
>
>
>
> Is there a way to specify multiple HCAs for a rank? Let say for the above
> example I want to run rank3 on host3 with2 hcas-hca1 and hca2. How to
> achieve it?
>
>
>
> Thanks,
>
> Santak Dalai
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150310/a076bc78/attachment.html>