<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Latha;
panose-1:2 0 4 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi, Carl.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for pointing this out. We appreciate it. We have updated the readme file with an acknowledgement to you.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best,<o:p></o:p></p>
<p class="MsoNormal">Hari.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Mvapich-discuss <mvapich-discuss-bounces+subramon=cse.ohio-state.edu@lists.osu.edu>
<b>On Behalf Of </b>Carl Ponder via Mvapich-discuss<br>
<b>Sent:</b> Sunday, October 31, 2021 9:10 AM<br>
<b>To:</b> mvapich-discuss@lists.osu.edu<br>
<b>Subject:</b> [Mvapich-discuss] osu_latency_mp & osu_latency_mt not working with "-d cuda" in 5.8<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The README from the 5.8 tarball says this<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-family:"Courier New"">ROCm, CUDA and OpenACC Extensions to OMB</span><br>
<span style="font-family:"Courier New"">----------------------------------------</span><br>
<span style="font-family:"Courier New"">CUDA Extensions to OMB can be enable by configuring the benchmark suite with</span><br>
<span style="font-family:"Courier New"">--enable-cuda option as shown below. </span>
<br>
<br>
<span style="font-family:"Courier New""> ./configure CC=/path/to/mpicc </span>
<br>
<span style="font-family:"Courier New""> CXX=/path/to/mpicxx</span><br>
<span style="font-family:"Courier New""> --enable-cuda </span><br>
<span style="font-family:"Courier New""> --with-cuda-include=/path/to/cuda/include</span><br>
<span style="font-family:"Courier New""> --with-cuda-libpath=/path/to/cuda/lib</span><br>
<span style="font-family:"Courier New""> make</span><br>
<span style="font-family:"Courier New""> make install</span><br>
<br>
<span style="font-family:"Courier New""> .......</span><br>
<br>
<span style="font-family:"Courier New"">Similarly, OpenACC Extensions can be enabled by specifying the --enable-openacc</span><br>
<span style="font-family:"Courier New"">option. The MPI library used should be able to support MPI communication from</span><br>
<span style="font-family:"Courier New"">buffers in GPU Device memory.</span><br>
<br>
<span style="font-family:"Courier New"">The following benchmarks have been extended to evaluate performance of</span><br>
<span style="font-family:"Courier New"">MPI communication using buffers on AMD and NVIDIA GPU devices.</span><br>
<br>
<span style="font-family:"Courier New""> osu_bibw - Bidirectional Bandwidth Test</span><br>
<span style="font-family:"Courier New""> osu_bw - Bandwidth Test</span><br>
<span style="font-family:"Courier New""> osu_latency - Latency Test</span><br>
<span style="font-family:"Courier New""> osu_mbw_mr - Multiple Bandwidth / Message Rate Test</span><br>
<span style="font-family:"Courier New""> osu_multi_lat - Multi-pair Latency Test</span><br>
<b><span style="font-family:"Courier New""> osu_latency_mt - Multi-threaded Latency Test</span></b><b><br>
</b><b><span style="font-family:"Courier New""> osu_latency_mp - Multi-process Latency Test</span></b><br>
<br>
<span style="font-family:"Courier New"">......</span><br>
<br>
<b><span style="font-family:"Courier New"">If both CUDA and OpenACC support is enabled you can switch between the modes</span></b><b><br>
</b><b><span style="font-family:"Courier New"">using the -d [cuda|openacc] option to the benchmarks.</span></b><span style="font-family:"Courier New""> If ROCm support is</span><br>
<span style="font-family:"Courier New"">enabled, you need to use -d rocm option to make the benchmarks use this feature.</span><br>
<span style="font-family:"Courier New"">Whether a process allocates its communication buffers on the GPU device or on</span><br>
<span style="font-family:"Courier New"">the host can be controlled at run-time. Use the -h option for more help.</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal">The <span style="font-family:"Courier New"">-d cuda</span> and
<span style="font-family:"Courier New"">-d openacc</span> options don't work with the
<span style="font-family:"Courier New"">osu_latency_mp</span> & <span style="font-family:"Courier New"">
osu_latency_mt</span> benchmarks.<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-family:"Courier New"">Invalid option [-d]</span><br>
<span style="font-family:"Courier New"">Usage: osu_latency_mp [options]<br>
.....</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal">Also, the online documentation<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><a href="https://mvapich.cse.ohio-state.edu/benchmarks/">https://mvapich.cse.ohio-state.edu/benchmarks/</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt">includes the <span style="font-family:"Courier New"">
osu_latency_mt</span> in the list but not the <span style="font-family:"Courier New"">
osu_latency_mp</span>. It just looks like the docs need to be updated.<o:p></o:p></p>
</div>
</body>
</html>