[mvapich-discuss] osu_mbw_mr does not recognize option -d

[Subramoni, Hari]

Hi, Marius.

I apologize for the error. We will correct that ASAP. I am sure that osu_mbw_mr does not support device to device transfers. Could you please try osu_bw, osu_bibw or osu_latency instead?

Could you please let me know what sort of errors you're facing with running your application with our stacks? Were you using MVAPICH2 or MVAPICH2-GDR? Did you use the LD_PRELOAD option?

http://mvapich.cse.ohio-state.edu/userguide/gdr/2.3a/#_example_use_of_ld_preload

Best Regards,
Hari.

-----Original Message-----
From: mvapich-discuss-bounces at cse.ohio-state.edu On Behalf Of Marius Brehler
Sent: Wednesday, June 27, 2018 1:15 PM
To: Subramoni, Hari <subramoni.1 at osu.edu>
Cc: mvapich-discuss at cse.ohio-state.edu <mvapich-discuss at mailman.cse.ohio-state.edu>
Subject: Re: [mvapich-discuss] osu_mbw_mr does not recognize option -d

Hello Harri,

actually I have chosen this benchmark randomly to verify our system setup, since our application is miserably failing whereas openmpi works.
Both processes were executed on the same node, just picked osu_mbw_mr since the example is given in the docs:

http://mvapich.cse.ohio-state.edu/userguide/gdr/2.3a/#_examples_using_osu_micro_benchmarks_with_multi_rail_support

In this example the '-d cuda' option is passed. osu_mbw_mr reports
'pairs: 1' and both GPUs have some load, showing that a process is running on each of them with nvidia-smi.

Regards

Marius

On 06/27/2018 12:51 PM, Subramoni, Hari wrote:
> Hello, Marius.
>
> The multi-pair bandwidth/message rate benchmark does not support 
> transfer from/to device buffers. This is a known limitation. We have 
> plans to add this support in the future.
>
> Could you please take a look at the numbers reported with 
> '--accelerator cuda' and compare it with numbers seen for host to 
> host? It's likely that it is just running host to host and not 
> throwing the error message correctly.
>
> Regards,
> Hari.
>
> Sent from my phone
>
>
> On Jun 27, 2018 12:34 PM, Marius Brehler 
> <marius.brehler at tu-dortmund.de>
> wrote:
>
>     Hi,
>
>     testing mvapich2-gdr on our machine, I noticed that passing '-d cuda' to
>     the osu_mbw_mr benchmark (shipped with the RPM) fails:
>
>
>     [brehler at fermi tests]$ mpirun -np 2
>     /opt/mvapich2/gdr/2.3a/libexec/osu-micro-benchmarks/get_local_rank
>     /opt/mvapich2/gdr/2.3a/libexec/osu-micro-benchmarks/mpi/pt2pt/osu_mbw_mr
>     -d cuda
>     /opt/mvapich2/gdr/2.3a/libexec/osu-micro-benchmarks/mpi/pt2pt/osu_mbw_mr:
>     invalid option -- 'd'
>     Invalid Option [-d]
>
>     /opt/mvapich2/gdr/2.3a/libexec/osu-micro-benchmarks/mpi/pt2pt/osu_mbw_mr:
>     invalid option -- 'd'
>     Usage: (null) [options]
>
>
>     Passing the option '--accelerator cuda' works as expected.
>     Regards
>
>     Marius
>     Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich.
>     Sie ist ausschließlich für den Adressaten bestimmt. Sollten Sie
>     nicht der für diese E-Mail bestimmte Adressat sein, unterrichten Sie
>     bitte den Absender und vernichten Sie diese Mail. Vielen Dank.
>     Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen
>     ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher
>     Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung
>     eines solchen Schriftstücks per Telefax erfolgen.
>
>     Important note: The information included in this e-mail is
>     confidential. It is solely intended for the recipient. If you are
>     not the intended recipient of this e-mail please contact the sender
>     and delete this message. Thank you. Without prejudice of e-mail
>     correspondence, our statements are only legally binding when they
>     are made in the conventional written form (with personal signature)
>     or when such documents are sent by fax.
>
>     _______________________________________________
>     mvapich-discuss mailing list
>     mvapich-discuss at cse.ohio-state.edu
>     http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>

--
M.Sc. Marius Brehler
Wissenschaftlicher Mitarbeiter

Technische Universität Dortmund
Lehrstuhl für Hochfrequenztechnik
Friedrich-Wöhler-Weg 4
D-44227 Dortmund

Tel.: +49 231-755 6674
Fax:  +49 231-755 4631
E-Mail: marius.brehler at tu-dortmund.de
Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der für diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Vielen Dank.
Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung eines solchen Schriftstücks per Telefax erfolgen.

Important note: The information included in this e-mail is confidential. It is solely intended for the recipient. If you are not the intended recipient of this e-mail please contact the sender and delete this message. Thank you. Without prejudice of e-mail correspondence, our statements are only legally binding when they are made in the conventional written form (with personal signature) or when such documents are sent by fax.

_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 6126 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20180627/e45c1053/attachment-0001.bin>