<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Hi Ron, <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
After looking at your hydra config logs it looks like what's happening is that your configuration is not correctly detecting the location of
<code>libpbs.so</code>. This means there is probably an issue with your installation or your
<code>--with-pbs</code> flag. If you have a nonstandard installation, please try to get
<code>libpbs.so</code> into your <code>LD_LIBRARY_PATH</code> and <code>LIBRARY_PATH</code> environment variables. You can check this by doing
<code>gcc -lpbs</code> and verifying that the error you get is related to an undefined reference to main, not
<code>/usr/bin/ld: cannot find -lpbs</code>. <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Without knowing your installation details, I cannot provide more detailed instructions.
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Here are the relevant lines in the Hydra config.log: <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
<pre class="ContentPasted0 elementToProof">configure:15781: checking tm.h usability
configure:15781: gcc -c -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -DNDEBUG -DNVALGRIND -O2 -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include conftest.c >&5
configure:15781: $? = 0
configure:15781: result: yes
configure:15781: checking tm.h presence
configure:15781: gcc -E -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include conftest.c
configure:15781: $? = 0
configure:15781: result: yes
configure:15781: checking for tm.h
configure:15781: result: yes
configure:15789: checking for tm_init in -ltorque
configure:15814: gcc -o conftest -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -DNDEBUG -DNVALGRIND -O2 -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include -L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib conftest.c -ltorque >&5
configure:15814: $? = 0
configure:15823: result: yes
WRAPPER_LIBS(=' ') does not contain '-ltorque', appending
CPPFLAGS(='-I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include') does not contain '-I/opt/torque/6.1.3/include', appending
LDFLAGS(='-L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib') does not contain '-L/opt/torque/6.1.3/lib', appending
configure:15982: checking for tm.h
configure:15982: result: yes
configure:15990: checking for tm_init in -lpbs
configure:16015: gcc -o conftest -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -DNDEBUG -DNVALGRIND -O2 -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include -I/opt/torque/6.1.3/include -L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib -L/opt/torque/6.1.3/lib conftest.c -lpbs >&5
/bin/ld: cannot find -lpbs
collect2: error: ld returned 1 exit status
configure:16015: $? = 1<br><br></pre>
<pre class="ContentPasted0 elementToProof"><span style="font-family: Calibri, Helvetica, sans-serif;">Please let me know if you have any further questions. <br><br>Thanks, <br>Nat<br></span></pre>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Mvapich-discuss <mvapich-discuss-bounces@lists.osu.edu> on behalf of Subramoni, Hari via Mvapich-discuss <mvapich-discuss@lists.osu.edu><br>
<b>Sent:</b> Monday, May 8, 2023 13:13<br>
<b>To:</b> Rahaman, Ronald O <rrahaman6@gatech.edu>; Announcement about MVAPICH2 (MPI over InfiniBand, RoCE, Omni-Path, iWARP and EFA) Libraries developed at NBCL/OSU <mvapich-discuss@lists.osu.edu><br>
<b>Subject:</b> Re: [Mvapich-discuss] Compiler error on Torque cluster</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:Latha}
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
span.x_EmailStyle20
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt">Hi, Ron.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Sorry for the delay. We’re taking a look at it and will get back soon.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Best,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Hari.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b><span style="font-size:11.0pt">From:</span></b><span style="font-size:11.0pt"> Mvapich-discuss <mvapich-discuss-bounces+subramon=cse.ohio-state.edu@lists.osu.edu>
<b>On Behalf Of </b>Rahaman, Ronald O via Mvapich-discuss<br>
<b>Sent:</b> Thursday, May 4, 2023 3:48 PM<br>
<b>To:</b> mvapich-discuss@lists.osu.edu<br>
<b>Subject:</b> [Mvapich-discuss] Compiler error on Torque cluster</span></p>
</div>
<p class="x_MsoNormal"> </p>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:1.0pt; color:white">’m building MVAPICH 2. 3. 7 for a Torque cluster with RHEL 7. 9 and GCC 10. I’m getting a compiler error about 'HYDT_bsci_launcher_pbs_init' being undeclared. Here is my config. log
and src/pm/hydra/config. log: https: //www. dropbox. com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0
</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:1.0pt; color:white">ZjQcmQRYFpfptBannerStart</span></p>
</div>
<table class="x_MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%; border-radius:4px">
<tbody>
<tr>
<td style="padding:12.0pt 0in 12.0pt 0in">
<table class="x_MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%; background:#CFD3D7; border:none; border-top:solid #8C8E91 3.0pt">
<tbody>
<tr>
<td valign="top" style="border:none; padding:0in 7.5pt 3.75pt 4.5pt">
<table class="x_MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="x_MsoNormal"><b><span style="font-size:10.5pt; font-family:"Arial",sans-serif; color:black">This Message Is From an External Sender
</span></b></p>
</td>
</tr>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="x_MsoNormal"><span style="font-size:9.0pt; font-family:"Arial",sans-serif; color:black">This message came from outside your organization.
</span></p>
</td>
</tr>
</tbody>
</table>
<table class="x_MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="right">
<tbody>
<tr>
<td style="padding:3.0pt 0in 3.0pt 0in">
<p class="x_MsoNormal"> <a href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vwQd0oZND6YAhRdwHo7Fd0rJ4aAOeq2b5G9bRFjIg3wrDHdqgr6-kEeKQABbt1IaZGX4GQmxf5ojfeeEaTpLezdajC4qkmq7M_I6coxrq0m5dScFE2DrnUktzh7ezGS_H8w7140GIslYk6JWHmSG_g$" target="_blank"><strong><span style="font-size:10.5pt; font-family:"Arial",sans-serif; color:black; border:solid #666666 1.0pt; padding:6.0pt; font-weight:normal; text-decoration:none"> Report Suspicious </span></strong></a>
<span style="font-size:11.0pt"></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:1.0pt; color:white">ZjQcmQRYFpfptBannerEnd</span></p>
</div>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121">’m building MVAPICH 2.3.7 for a Torque cluster with RHEL 7.9 and GCC 10. I’m getting a compiler error about 'HYDT_bsci_launcher_pbs_init' being undeclared. Here is my config.log and src/pm/hydra/config.log:</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121"><a href="https://urldefense.com/v3/__https:/www.dropbox.com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0__;!!KGKeukY!2wdfpNssoTC7KRMsB2LkULjCWHcN3WDucL1rgKAU7b0Eo22L0HWpr-wpKlQAFxo7JjX20Xo6b3ntpcNL0QXa2o5B7klDHqRt$">https://www.dropbox.com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0</a></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121">And the compiler error is below. </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121">Thanks!</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121">Ron</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:#212121"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[3]: Entering directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121"> CC tools/bootstrap/src/bsci_init.lo</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">tools/bootstrap/src/bsci_init.c:17:266: error: 'HYDT_bsci_launcher_pbs_init' undeclared here (not in a function); did you mean 'HYDT_bsci_launcher_sge_init'?</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121"> 17 | t_array[])(void) = { HYDT_bsci_launcher_ssh_init, HYDT_bsci_launcher_rsh_init, HYDT_bsci_launcher_fork_init, HYDT_bsci_launcher_slurm_init, HYDT_bsci_launcher_ll_init,
HYDT_bsci_launcher_lsf_init, HYDT_bsci_launcher_sge_init, HYDT_bsci_launcher_pbs_init, HYDT_bsci_launcher_manual_init, HYDT_bsci_launcher_persist_init, NULL };</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121"> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121"> | HYDT_bsci_launcher_sge_init</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[3]: *** [tools/bootstrap/src/bsci_init.lo] Error 1</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[3]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[2]: *** [all-recursive] Error 1</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[2]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[1]: *** [all-recursive] Error 1</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make[1]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1'</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Courier New"; color:#212121">make: *** [all] Error 2</span><span style="font-size:11.0pt; color:#212121"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<div>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">--------</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">Ron Rahaman</span><span style="font-size:11.0pt"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Research Scientist II, Research Software Engineer</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Partnership for an Advanced Computing Environment (PACE)</span></p>
</div>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Georgia Institute of Technology
</span><span style="font-size:11.0pt"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
</div>
</div>
</body>
</html>