<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Latha;
panose-1:2 0 4 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:方正正粗黑简体;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"\@方正正粗黑简体";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hello.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Could you please let us know what MPI library you are using here?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thx,<br>
Hari.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Mvapich-discuss <mvapich-discuss-bounces@lists.osu.edu>
<b>On Behalf Of </b>s2633806413@126.com via Mvapich-discuss<br>
<b>Sent:</b> Friday, September 23, 2022 3:23 AM<br>
<b>To:</b> mvapich-discuss@lists.osu.edu<br>
<b>Subject:</b> [Mvapich-discuss] ERROR:running osu_benchmark on multiple nodes<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Hello, when I running osu_benchmark on eight nodes. The command used us: ./mpirun -np 8 -ppn 1 -host 10. 3. 1. 1,10. 3. 1. 2,10. 3. 1. 3,10. 3. 1. 4,10. 3. 1. 5,10. 3. 1. 6,10. 3. 1. 7,10. 3. 1. 9
./osu_reduce Get wrong result: Caught error: Segmentation fault(signal <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;border-radius:4px">
<tbody>
<tr>
<td style="padding:12.0pt 0in 12.0pt 0in">
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#CFD3D7;border:none;border-top:solid #8C8E91 3.0pt">
<tbody>
<tr>
<td valign="top" style="border:none;padding:0in 7.5pt 3.75pt 4.5pt">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="right">
<tbody>
<tr>
<td style="padding:3.0pt 0in 3.0pt 0in">
<p class="MsoNormal"> <a href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!se0ZFkwrqC6NivkQP-DE99PagLIZu8Kks6rkWo20OBm_qXht5Bf2U8yS0tTovqjUKM50UsSbnHekuyq__Rg77xPGZYGcn60$" target="_blank"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;border:solid #666666 1.0pt;padding:6.0pt;font-weight:normal;text-decoration:none"> Report Suspicious </span></strong></a>
<o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Hello, when I running osu_benchmark on eight nodes.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The command used us:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">./mpirun -np 8 -ppn 1 -host 10.3.1.1,10.3.1.2,10.3.1.3,10.3.1.4,10.3.1.5,10.3.1.6,10.3.1.7,10.3.1.9 ./osu_reduce <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Get wrong result:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Caught error:Segmentation fault(signal 11)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">===================================================================================<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>= PID 725097 RUNNING AT yuanhe<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>= EXIT CODE: 139<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>= CLEANING UP REMAINING PROCESSES<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>===================================================================================<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-01] HYD_pmcd_pmip_control_cmd_cb (pm/pmiserv/pmip_cb.c:911): assert (!closed) failed<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-01] HYDT_dmxu_poll_wait_for_event (tools/demux/demux_poll.c:76): callback returned error status<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-01] main (pm/pmiserv/pmip.c:202): demux engine error waiting for event<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-03] HYD_pmcd_pmip_control_cmd_cb (pm/pmiserv/pmip_cb.c:911): assert (!closed) failed<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-03] HYDT_dmxu_poll_wait_for_event (tools/demux/demux_poll.c:76): callback returned error status<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-03] main (pm/pmiserv/pmip.c:202): demux engine error waiting for event<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-05] HYD_pmcd_pmip_control_cmd_cb (pm/pmiserv/pmip_cb.c:911): assert (!closed) failed<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-05] HYDT_dmxu_poll_wait_for_event (tools/demux/demux_poll.c:76): callback returned error status<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[proxy:0:0@swat3-05] main (pm/pmiserv/pmip.c:202): demux engine error waiting for event<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">[mpiexec@swat3-01] HYDT_bscu_wait_for_completion (tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>[mpiexec@swat3-01] HYDT_bsci_wait_for_completion (tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br>
</span>[mpiexec@swat3-01] HYD_pmci_wait_for_completion (pm/pmiserv/pmiserv_pmci.c:218): launcher returned error waiting for completion<span style="font-size:12.0pt;font-family:"Arial",sans-serif"><br style="box-sizing: border-box">
<br>
</span><o:p></o:p></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">[mpiexec@swat3-01] main (ui/mpich/mpiexec.c:344): process manager error waiting for completion<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">This above is the error log.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">I also test :<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">1.</span><span style="font-size:12.0pt;color:#222222"> </span><span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222"><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">./mpirun -np 7 -ppn 1 -host 10.3.1.1,10.3.1.2,10.3.1.3,10.3.1.4,10.3.1.5,10.3.1.6,10.3.1.7 ./osu_reduce</span><span style="font-size:12.0pt;color:#222222"> </span><span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222"><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">---->have a current result<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">2.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">./mpirun -np 7 -ppn 1 -host 10.3.1.1,10.3.1.2,10.3.1.3,10.3.1.4,10.3.1.5,10.3.1.6,10.3.1.9 ./osu_reduce</span><span style="font-size:12.0pt;color:#222222"> </span><span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222"><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">---->have a current result<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">3.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">./mpirun -np 8 -ppn 1 -host 10.3.1.1,10.3.1.2,10.3.1.3,10.3.1.4,10.3.1.5,10.3.1.6,10.3.1.7,10.3.1.9 ./osu_reduce</span><span style="font-size:12.0pt;color:#222222"> </span><span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222"><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">---->have a wrong result<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">Thanks to anyone who may shed light into this.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:0in;line-height:20.25pt;box-sizing: border-box;overflow:hidden">
<span style="font-size:12.0pt;font-family:"方正正粗黑简体",serif;color:#222222">-sirui<o:p></o:p></span></p>
</div>
</div>
</body>
</html>