<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle24
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi, Kent.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Glad to know that you’re able to work around this. One way to avoid the configure error would be to install the OFED libraries. We will see how we can get around the issues in the meantime.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thx,<o:p></o:p></p>
<p class="MsoNormal">Hari.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Kent Cheung <Kent.Cheung@arm.com> <br>
<b>Sent:</b> Wednesday, April 28, 2021 11:55 AM<br>
<b>To:</b> Subramoni, Hari <subramoni.1@osu.edu><br>
<b>Cc:</b> mvapich-discuss@lists.osu.edu<br>
<b>Subject:</b> Re: MPI_Ssend causes segfault<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Hi Hari,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Without the device flag, the configure script fails with:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">configure: error: 'infiniband/verbs.h not found. Did you specify --with-ib-include=?'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">However, after configuring and building with<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">--with-device=ch3:nemesis<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">the crash does not reoccur. Is it possible to fix this issue for the other device setting too? Thanks.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Kent<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Subramoni, Hari <<a href="mailto:subramoni.1@osu.edu">subramoni.1@osu.edu</a>><br>
<b>Sent:</b> 28 April 2021 15:48<br>
<b>To:</b> Kent Cheung <<a href="mailto:Kent.Cheung@arm.com">Kent.Cheung@arm.com</a>><br>
<b>Cc:</b> <a href="mailto:mvapich-discuss@lists.osu.edu">mvapich-discuss@lists.osu.edu</a> <<a href="mailto:mvapich-discuss@lists.osu.edu">mvapich-discuss@lists.osu.edu</a>>; Subramoni, Hari <<a href="mailto:subramoni.1@osu.edu">subramoni.1@osu.edu</a>><br>
<b>Subject:</b> RE: MPI_Ssend causes segfault</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal">Hi, Kent.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Can you please try to remove <span style="font-size:12.0pt;color:black">
--with-device=ch3:sock from the configure line, reconfigure the MPI library and try again?</span><o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Thx,<o:p></o:p></p>
<p class="xmsonormal">Hari.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal"><b>From:</b> Mvapich-discuss <<a href="mailto:mvapich-discuss-bounces+subramoni.1=osu.edu@lists.osu.edu">mvapich-discuss-bounces+subramoni.1=osu.edu@lists.osu.edu</a>>
<b>On Behalf Of </b>Kent Cheung via Mvapich-discuss<br>
<b>Sent:</b> Friday, April 23, 2021 11:47 AM<br>
<b>To:</b> <a href="mailto:mvapich-discuss@lists.osu.edu">mvapich-discuss@lists.osu.edu</a><br>
<b>Subject:</b> [Mvapich-discuss] MPI_Ssend causes segfault<o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Hello,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">I'm currently running into an issue with this example on a Rhel 8.3 x86_64 single node machine:</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#include "mpi.h"
</span><o:p></o:p></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#include <stdio.h></span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">int main()</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">{</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> int rank;</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_Init(NULL, NULL);</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_Comm_rank(MPI_COMM_WORLD, &rank);</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> int number;</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> if (rank == 0) {</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> number = 42;</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_Ssend(&number, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> } else if (rank == 1) {</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_Recv(&number, 1, MPI_INT, 0, 0, MPI_COMM_WORLD,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_STATUS_IGNORE);</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> printf("Process 1 received number %d from process 0\n",</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> number);</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> }</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> MPI_Finalize();</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> return 0;</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">}</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Here is the backtrace:</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#0 0x00007f96df5a6e28 in __memmove_avx_unaligned_erms () from /lib64/libc.so.6
</span><o:p></o:p></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#1 0x00007f96df93c8cc in MPIDI_CH3U_Receive_data_found () from /lib/libmpi.so.12</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#2 0x00007f96df95ef02 in MPIDI_CH3_PktHandler_EagerSyncSend () from /lib/libmpi.so.12</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#3 0x00007f96df96eb71 in MPIDI_CH3I_Progress_handle_sock_event () from /lib/libmpi.so.12</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#4 0x00007f96df96ef1e in MPIDI_CH3I_Progress () from /lib/libmpi.so.12</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#5 0x00007f96df8e5713 in PMPI_Recv () from /lib/libmpi.so.12</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">#6 0x00000000004007d6 in main () at main.c:17</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">If I change MPI_Ssend to MPI_Send, the program runs to completion. If I build and test with OpenMPI 4.1.0, the program runs to completion with either call. For reference, I am using GCC 10.2.0
and the configuration options for MVAPICH2 were</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">--enable-debug --enable-shared --enable-debuginfo --enable-sharedlib=gcc --with-device=ch3:sock</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Is there are configuration flag I require to resolve this or an issue with the example?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Thanks,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Kent</span><o:p></o:p></p>
</div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<p class="xmsonormal">IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person,
use it for any purpose, or store or copy the information in any medium. Thank you.
<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person,
use it for any purpose, or store or copy the information in any medium. Thank you.
<o:p></o:p></p>
</div>
</body>
</html>