<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1256">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Latha;
panose-1:2 0 4 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hello, Dr. Levi.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">We only have basic support for GPU-enabled clusters in MVAPICH2 2.3.7.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">For best performance, functionality, and the latest features on GPU-enabled clusters, we strongly recommend using MVAPICH2-GDR. It is available as an RPM package from our download page.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">If you do not find the exact version youre looking for, kindly fill out this form and we can build it for you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><a href="http://mvapich.cse.ohio-state.edu/GDRform/">http://mvapich.cse.ohio-state.edu/GDRform/</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Hari.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt">From:</span></b><span style="font-size:11.0pt"> Mvapich-discuss <mvapich-discuss-bounces@lists.osu.edu>
<b>On Behalf Of </b>Levi, Mariana via Mvapich-discuss<br>
<b>Sent:</b> Tuesday, August 16, 2022 2:48 PM<br>
<b>To:</b> mvapich-discuss@lists.osu.edu<br>
<b>Subject:</b> [Mvapich-discuss] Error compiling with CUDA 11.4 and GNU 11.1.0<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Hi MVAPICH2 team, Im trying to install a MVAPICH 2.3.7 with GNU 11.1.0 and CUDA 11.4 on an HPC cluster (Centos 7). Im using the following command: FFLAGS="-w
-fallow-argument-mismatch -O2" ../configure --prefix=/shared/centos7/mvapich2/2.3.7-gcc11.1-cuda11.4
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;border-radius:4px">
<tbody>
<tr>
<td style="padding:12.0pt 0in 12.0pt 0in">
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#CFD3D7;border:none;border-top:solid #8C8E91 3.0pt">
<tbody>
<tr>
<td valign="top" style="border:none;padding:0in 7.5pt 3.75pt 4.5pt">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="right">
<tbody>
<tr>
<td style="padding:3.0pt 0in 3.0pt 0in">
<p class="MsoNormal"> <a href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vwQdMiaND6YApRdxfo6ldr_NiARY1aK_J5CDbp34OLr9daJwvRXAc56tFlO87kf5FpxQOzYhfnwwN7JfmI6qf_b0CyCEaSE20JNWWzPpI-87bF8AA25c8F-u4i5LK4uWxGDOnOMUTXCRBre1mlESBA$" target="_blank"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;border:solid #666666 1.0pt;padding:6.0pt;font-weight:normal;text-decoration:none"> Report Suspicious </span></strong></a>
<span style="font-size:11.0pt"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hi MVAPICH2 team,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Im trying to install a MVAPICH 2.3.7 with GNU 11.1.0 and CUDA 11.4 on an HPC cluster (Centos 7). Im using the following command:<br>
<br>
</span><span style="font-size:10.0pt;font-family:"Courier New"">FFLAGS="-w -fallow-argument-mismatch -O2" ../configure --prefix=/shared/centos7/mvapich2/2.3.7-gcc11.1-cuda11.4 --with-device=ch3:mrail --with-rdma=gen2 --enable-threads=multiple --enable-fortran=all
--enable-fast --with-pmi=pmi2 --with-pm=slurm --enable-slurm=yes --with-libcuda=/shared/centos7/cuda/11.4/targets/x86_64-linux/lib/stubs --with-libcudart=/shared/centos7/cuda/11.4/targets/x86_64-linux/lib --with-cuda=/shared/centos7/cuda/11.4 --with-cuda-include=/shared/centos7/cuda/11.4/include
--with-cuda-libpath=/shared/centos7/cuda/11.4/lib64<br>
<br>
make -j</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The architecture Im building on is Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz (Skylake_avx512 microarchitecture) with V100 NVIDIA GPUs, and InfiniBand network (Mellanox OFED version 5.3) support.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Could you please assist with the following errors Im getting:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:</span></b><span style="font-size:10.0pt;font-family:"Courier New""> In function hwloc_opencl_query_device<b>s</b>:<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:108:5:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>unknown type name <b>cl_device_topology_amd</b><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 108 |
<b>cl_device_topology_amd</b> amdtopo;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^~~~~~~~~~~~~~~~~~~~~~</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:171:9:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b><b>CL_DEVICE_TOPOLOGY_TYPE_PCIE_AMD</b> undeclared (first use in this function); did you mean <b>CL_DEVICE_TOPOLOGY_AMD</b>?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 171 | if (<b>CL_DEVICE_TOPOLOGY_TYPE_PCIE_AMD</b> != amdtopo.raw.type) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</b><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> | CL_DEVICE_TOPOLOGY_AMD<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:171:9:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>note: </b>each undeclared identifier is reported only once for each function it appears in<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:171:52:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>raw</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 171 | if (CL_DEVICE_TOPOLOGY_TYPE_PCIE_AMD != amdtopo<b>.</b>raw.type) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:172:53:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>raw</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 172 | hwloc_debug("not a PCIe device: %u\n", amdtopo<b>.</b>raw.type);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:178:40:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 178 | info->specific.amd.pcibus = amdtopo<b>.</b>pcie.bus;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:179:40:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 179 | info->specific.amd.pcidev = amdtopo<b>.</b>pcie.device;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:180:41:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 180 | info->specific.amd.pcifunc = amdtopo<b>.</b>pcie.function;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:183:35:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 183 | (unsigned) amdtopo<b>.</b>pcie.bus, (unsigned) amdtopo.pcie.device, (unsigned) amdtopo.pcie.function);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:183:64:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 183 | (unsigned) amdtopo.pcie.bus, (unsigned) amdtopo<b>.</b>pcie.device, (unsigned) amdtopo.pcie.function);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Courier New"">../../../../contrib/hwloc_v1/src/topology-opencl.c:183:96:</span></b><span style="font-size:10.0pt;font-family:"Courier New"">
<b>error: </b>request for member <b>pcie</b> in something not a structure or union<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> 183 | (unsigned) amdtopo.pcie.bus, (unsigned) amdtopo.pcie.device, (unsigned) amdtopo<b>.</b>pcie.function);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> |
<b>^</b><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[3]: *** [topology-opencl.lo] Error 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[3]: *** Waiting for unfinished jobs....<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[3]: Leaving directory `/shared/centos7/mvapich2/src/mvapich2-2.3.7/build-gcc11.1-cuda11.4/contrib/hwloc_v1/src'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[2]: *** [all-recursive] Error 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[2]: Leaving directory `/shared/centos7/mvapich2/src/mvapich2-2.3.7/build-gcc11.1-cuda11.4/contrib/hwloc_v1'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[1]: *** [all-recursive] Error 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make[1]: Leaving directory `/shared/centos7/mvapich2/src/mvapich2-2.3.7/build-gcc11.1-cuda11.4'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">make: *** [all] Error 2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ive also attached the config.log file for reference.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Thanks in advance for your assistance.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Mariana Levi, Ph.D.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Computational Scientist<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Research Computing, Information Technology Services</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Northeastern University</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">617-470-4022<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>