<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Hi ZQ, <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
This looks to be a newer version of pmi2.h. Older version's of slurm's PMI2 had that uncommented which allowed us to use
<code>PMI_keyval_t</code> in both pmi1 and pmi2 builds. We have seen a similar issues with Cray pmi support recently. l will take a look to see how this can best be resolved to work with both versions. Can you tell me what version of Slurm you are using?<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Nat<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> You, Zhi-Qiang <zyou@osc.edu><br>
<b>Sent:</b> Wednesday, May 10, 2023 11:19<br>
<b>To:</b> Shineman, Nat <shineman.5@osu.edu>; Announcement about MVAPICH2 (MPI over InfiniBand, RoCE, Omni-Path, iWARP and EFA) Libraries developed at NBCL/OSU <mvapich-discuss@lists.osu.edu><br>
<b>Subject:</b> Re: [Mvapich-discuss] mvapich2 3.0a build failure finding pmi2.h</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:PMingLiU}
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif}
code
{font-family:"Courier New"}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt">Hello,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">I was able to fix the "pmi2.h" error by adding `--with-slurm-include=/usr/include/slurm` to the configure command. However, I encountered another error:<br>
<br>
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">>> 9796 src/util/mpir_pmi.c(781): error: identifier "PMI_keyval_t" is undefined</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> 9797 static int mpi_to_pmi_keyvals(MPIR_Info * info_ptr, PMI_keyval_t ** kv_ptr, int *nkeys_ptr);</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> 9798 ^</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> 9799 </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> >> 9800 src/util/mpir_pmi.c(782): error: identifier "PMI_keyval_t" is undefined</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> 9801 static void free_pmi_keyvals(PMI_keyval_t ** kv, int size, int *counts);</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> 9802 ^</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Upon checking the pmi2.h file, I found the following:<br>
<br>
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">/* This is here to allow spawn multiple functions to compile. This</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> needs to be removed once those functions are fixed for pmi2 */</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">/*</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">typedef struct PMI_keyval_t</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">{</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> char * key;</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> char * val;</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">} PMI_keyval_t;</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">*/</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">The “PMI_keyval_t" struct is commented out. We are using the same SLURM version for other mvapich2 installations. Does this mean we need to upgrade SLURM or PMI2?<br>
<br>
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Thank you,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">-ZQ</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt; color:black">From:
</span></b><span style="font-size:12.0pt; color:black">Mvapich-discuss <mvapich-discuss-bounces@lists.osu.edu> on behalf of Shineman, Nat via Mvapich-discuss <mvapich-discuss@lists.osu.edu><br>
<b>Date: </b>Wednesday, May 10, 2023 at 9:20 AM<br>
<b>To: </b>christof.koehler@bccms.uni-bremen.de <christof.koehler@bccms.uni-bremen.de>, c.koehler@uni-bremen.de <c.koehler@uni-bremen.de>, Announcement about MVAPICH2 (MPI over InfiniBand, RoCE, Omni-Path, iWARP and EFA) Libraries developed at NBCL/OSU <mvapich-discuss@lists.osu.edu><br>
<b>Subject: </b>Re: [Mvapich-discuss] mvapich2 3.0a build failure finding pmi2.h</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Hi Cristof, </span>
</p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Thanks for the bug report. Looks like our configure script is checking for the file
</span><code><span style="color:black">slurm/pmi2.h</span></code><span style="font-size:12.0pt; color:black"> while the actual file includes
</span><code><span style="color:black">pmi.h</span></code><span style="font-size:12.0pt; color:black"> without the slurm prefix. This include was changed in 3.0 to support non-slurm installations of pmi2. The easiest way to solve this is to set the environment
variable </span><code><span style="color:black">CPATH=/usr/include/slurm:$CPATH</span></code><span style="font-size:12.0pt; color:black"> (or using whatever path your slurm include directory resides on). Alternatively, you can add
</span><code><span style="color:black">MPICHLIB_CFLAGS=-I/usr/include/slurm</span></code><span style="font-size:12.0pt; color:black"> as an argument to configure.</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Either of the above options should get you past the build error for now. We will also look into having our configure script automatically detect slurm based pmi installations and adapt the paths
appropriately. As a final note, we do recomend using pmi1 with slurm when possible (as do the upstream developers of MPICH) as it is a more stable and standardized provider than pmi2, which was never completed. Slurm's
</span><code><span style="color:black">--with-mpi=pmi2</span></code><span style="font-size:12.0pt; color:black"> actually supports both the pmi1 and and pmi2 interface.
</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Please let me know if you have any further issues building MVAPICH 3.0a.</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Thanks,</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black">Nat</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt">
<hr size="0" width="100%" align="center">
</span></div>
<div id="x_divRplyFwdMsg">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt; color:black">From:</span></b><span style="font-size:11.0pt; color:black"> Mvapich-discuss <mvapich-discuss-bounces+shineman.5=osu.edu@lists.osu.edu> on behalf of christof.koehler--- via Mvapich-discuss
<mvapich-discuss@lists.osu.edu><br>
<b>Sent:</b> Wednesday, May 10, 2023 04:06<br>
<b>To:</b> mvapich-discuss@lists.osu.edu <mvapich-discuss@lists.osu.edu><br>
<b>Subject:</b> [Mvapich-discuss] mvapich2 3.0a build failure finding pmi2.h</span><span style="font-size:11.0pt">
</span></p>
<div>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
</div>
</div>
<div>
<div>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Hello everybody,<br>
<br>
we have an Cornelis Omni-Path system and therefore would like to start<br>
testing mvapich2 3.0a. However, I am currently unable to build it.<br>
I would like to add that building mvpaich2 2.3.7-1 with slurm/pmi2 on<br>
the same system works fine.<br>
<br>
For mvapich2 3.0a the configure step using<br>
FFLAGS=-fallow-argument-mismatch ./configure --with-pm=slurm --with-pmi=pmi2<br>
succeeds.<br>
<br>
But in the make step the build stops with<br>
<br>
make[2]: Entering directory '/backup1/build_temp/mvapich2-3.0a'<br>
CC src/mpi/attr/lib_libmpi_la-attr_delete.lo<br>
In file included from ./src/include/mpiimpl.h:130,<br>
from src/mpi/attr/attr_delete.c:6:<br>
./src/include/mpir_pmi.h:18:10: fatal error: pmi2.h: No such file or<br>
directory<br>
18 | #include <pmi2.h><br>
| ^~~~~~~~<br>
compilation terminated.<br>
<br>
In fact we have /usr/include/slurm/pmi2.h and this is apparently picked<br>
up correctly by configure:<br>
<br>
> grep pmi2.h configure.out <br>
checking slurm/pmi2.h usability... yes<br>
checking slurm/pmi2.h presence... yes<br>
checking for slurm/pmi2.h... yes<br>
<br>
So this is a bit confusing. Also, because according to the mailing list<br>
some people already built it and did not report such an error.<br>
<br>
I attach the gzipped config.log, the configure stdout (configure.out) and <br>
the stdout of the make step (make.out) to this email.<br>
<br>
Best Regards<br>
<br>
Christof<br>
<br>
-- <br>
Dr. rer. nat. Christof Köhler email: c.koehler@uni-bremen.de<br>
Universitaet Bremen/FB1/BCCMS phone: +49-(0)421-218-62334<br>
Am Fallturm 1/ TAB/ Raum 3.06 fax: +49-(0)421-218-62770<br>
28359 Bremen </span></p>
</div>
</div>
</div>
</div>
</body>
</html>