<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Thanks for contacting us. We will take a look at the issue and get back to you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">-Akshay Paniraja Guptha<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Mvapich-discuss <mvapich-discuss-bounces@lists.osu.edu>
<b>On Behalf Of </b>???(??) via Mvapich-discuss<br>
<b>Sent:</b> Wednesday, May 22, 2024 10:55 PM<br>
<b>To:</b> mvapich-discuss <mvapich-discuss@lists.osu.edu><br>
<b>Subject:</b> [Mvapich-discuss] Got "Initial job has not accepted any resources" when run MPI4Spark<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Hello, I am very interested in the MPI4Spark framework and try to run it in a cluster with 4 nodes equipped with RoCE network. I ran mvapich2 and mvapich2-j examples
successfully. Unfortunately</span><span style="font-size:1.0pt;font-family:"MS Gothic";color:white">，</span><span style="font-size:1.0pt;color:white">I got the “Initial job has not accepted any resources;
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;border-radius:4px">
<tbody>
<tr>
<td style="padding:12.0pt 0in 12.0pt 0in">
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#CFD3D7;border:none;border-top:solid #8C8E91 3.0pt">
<tbody>
<tr>
<td valign="top" style="border:none;padding:0in 7.5pt 3.75pt 4.5pt">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="right">
<tbody>
<tr>
<td style="padding:3.0pt 0in 3.0pt 0in">
<p class="MsoNormal"> <a href="https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vUQTVyxhgsYqSlqbVSSu1IJRp4bxOiNDMCRJjN24QddHGhK8J-CpX_7SWvT_VGFjfEGiH6tyZ4jGywwiBXidU4KQbkJOIdsWawviVTOrnenjg66cutL4-AKILdzeMyfKY18$" target="_blank"><strong><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black;border:solid #666666 1.0pt;padding:6.0pt;font-weight:normal;text-decoration:none"> Report Suspicious </span></strong></a>
<o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Hello, I am very interested in the MPI4Spark framework and try to run it in a cluster with 4 nodes equipped with RoCE network. I ran mvapich2 and mvapich2-j examples
successfully. Unfortunately</span><span style="font-size:10.5pt;font-family:"MS Gothic";color:black">，</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">I got the “Initial job has not accepted any resources; check your cluster
UI to ensure that workers are registered and have sufficient resources” warnings when I run MPI4Spark in both standalone and yarn mode. I read the worker.log and found that both workers got the job request but they seemed not to launch the executor. Is this
problem caused by some misconfigurations? The detailed configuraions and logs are listed below. I will appreciate if someone could help me to figure it out. :)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Hostfile<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">mpi4spark000</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">mpi4spark001<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">mpi4spark002<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">mpi4spark003</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">spark-env.sh<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">export SPARK_HOME=/root/mpi4spark/mpi4spark-0.2-x86-bin</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export SPARK_NO_DAEMONIZE=1<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export SPARK_USE_MPI=1<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$MV2J_HOME<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MV2J_HOME/lib<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export SPARK_LIBRARY_PATH=$MV2J_HOME/lib<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">export JAVA_BINARY=$JAVA_HOME/bin<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">export WORK_DIR=$SPARK_HOME/exec-wdir</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">spark-defaults.conf</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">spark.executor.extraJavaOptions -Djava.library.path=$MV2J_HOME/lib</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">app.sh</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">./bin/spark-submit --master spark://$1:7077 --class org.apache.spark.examples.SparkPi examples/jars/spark-examples_2.12-3.3.0-SNAPSHOT.jar</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Master.log<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">starting org.apache.spark.deploy.master.Master, logging to /root/mpi4spark/mpi4spark-0.2-x86-bin/logs/spark-root-org.apache.spark.deploy.master.Master-1-mpi4spark002.out</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Spark Command: /root/mpi4spark/jdk1.8.0_321//bin/java -cp /root/mpi4spark/mpi4spark-0.2-x86-bin//conf/:/root/mpi4spark/mpi4spark-0.2-x86-bin/jars/*:/root/mpi4spark/hadoop-3.3.4//etc/hadoop/
-Xmx1g org.apache.spark.deploy.master.Master --host mpi4spark002 --port 7077 --webui-port 8080<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">========================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,724 INFO master.Master: Started daemon with process name: 2621571@mpi4spark002<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,728 INFO util.SignalUtils: Registering signal handler for TERM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,728 INFO util.SignalUtils: Registering signal handler for HUP<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,728 INFO util.SignalUtils: Registering signal handler for INT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,932 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,968 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,968 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,968 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,969 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:50,969 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with
view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,395 INFO nio.NioEventLoop: Starting MPI4Spark.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,495 INFO util.Utils: Successfully started service 'sparkMaster' on port 7077.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,511 INFO master.Master: Starting Spark master at spark://mpi4spark002:7077<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,514 INFO master.Master: Running Spark version 3.3.0-SNAPSHOT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,534 INFO util.log: Logging initialized @3209ms to org.sparkproject.jetty.util.log.Slf4jLog<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,567 INFO server.Server: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_321-b07<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,579 INFO server.Server: Started @3255ms<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,605 INFO server.AbstractConnector: Started
<a href="mailto:ServerConnector@2efb59fc%7bHTTP/1.1">ServerConnector@2efb59fc{HTTP/1.1</a>, (http/1.1)}{0.0.0.0:8080}<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,605 INFO util.Utils: Successfully started service 'MasterUI' on port 8080.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,606 INFO ui.MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at
<a href="https://urldefense.com/v3/__http:/mpi4spark002:-1__;!!KGKeukY!3NXRmAG6eoLkZWP5gVYrEkoDx_fki5KEAjcJEOyLIUlOcmopE4jesI-LWd7MS8NM69GSnuqN3rBbunjqS0bJMG778G3nTxJ4si3J$">
http://mpi4spark002:-1</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,620 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1332e5ce%7b/app,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1332e5ce{/app,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,621 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1f910681%7b/app/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1f910681{/app/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,622 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@2a186645%7b/,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@2a186645{/,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,623 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@772642ad%7b/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@772642ad{/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,628 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@751cafb2%7b/static,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@751cafb2{/static,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,628 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1b2adb3b%7b/app/kill,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1b2adb3b{/app/kill,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,629 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1d5107f3%7b/driver/kill,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1d5107f3{/driver/kill,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,630 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@7e39159e%7b/workers/kill,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@7e39159e{/workers/kill,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,715 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@64ff458a%7b/metrics/master/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@64ff458a{/metrics/master/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,716 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@5b0b91df%7b/metrics/applications/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@5b0b91df{/metrics/applications/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,725 INFO master.Master: I have been elected leader! New state: ALIVE<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,826 INFO master.Master: Registering worker 200.2.149.6:46337 with 64 cores, 502.3 GiB RAM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,831 INFO master.Master: Registering worker 200.2.149.2:40247 with 64 cores, 502.3 GiB RAM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,906 INFO master.Master: Registering app Spark Pi<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,908 INFO master.Master: Registered app Spark Pi with ID app-20240522163753-0000<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,923 INFO master.Master: Launching executor app-20240522163753-0000/0 on worker worker-20240522163753-200.2.149.2-40247<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:53,927 INFO master.Master: Launching executor app-20240522163753-0000/1 on worker worker-20240522163753-200.2.149.6-46337</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Worker0.log<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">starting org.apache.spark.deploy.worker.Worker, logging to /root/mpi4spark/mpi4spark-0.2-x86-bin/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-mpi4spark000.out</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Spark Command: /root/mpi4spark/jdk1.8.0_321//bin/java -cp /root/mpi4spark/mpi4spark-0.2-x86-bin//conf/:/root/mpi4spark/mpi4spark-0.2-x86-bin/jars/*:/root/mpi4spark/hadoop-3.3.4//etc/hadoop/
-Xmx1g org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://mpi4spark002:7077<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">========================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,718 INFO worker.Worker: Started daemon with process name: 2807211@mpi4spark000<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,721 INFO util.SignalUtils: Registering signal handler for TERM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,722 INFO util.SignalUtils: Registering signal handler for HUP<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,722 INFO util.SignalUtils: Registering signal handler for INT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,931 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,966 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,966 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,966 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,967 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,967 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with
view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">[mpi4spark000:mpi_rank_0][MPID_Init] [Performance Suggestion]: Application has requested for multi-thread capability. If allocating memory from different pthreads/OpenMP
threads, please consider setting MV2_USE_ALIGNED_ALLOC=1 for improved performance.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Use MV2_USE_THREAD_WARNING=0 to suppress this error message.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,396 INFO nio.NioEventLoop: Starting MPI4Spark.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,496 INFO util.Utils: Successfully started service 'sparkWorker' on port 40247.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,497 INFO worker.Worker: Worker decommissioning not enabled.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,601 INFO worker.Worker: Starting Spark worker 200.2.149.2:40247 with 64 cores, 502.3 GiB RAM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,604 INFO worker.Worker: Running Spark version 3.3.0-SNAPSHOT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,605 INFO worker.Worker: Spark home: /root/mpi4spark/mpi4spark-0.2-x86-bin<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,616 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,616 INFO resource.ResourceUtils: No custom resources configured for spark.worker.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,616 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,635 INFO util.log: Logging initialized @2311ms to org.sparkproject.jetty.util.log.Slf4jLog<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,667 INFO server.Server: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_321-b07<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,678 INFO server.Server: Started @2355ms<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,708 INFO server.AbstractConnector: Started
<a href="mailto:ServerConnector@41cf10a2%7bHTTP/1.1">ServerConnector@41cf10a2{HTTP/1.1</a>, (http/1.1)}{0.0.0.0:8081}<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,708 INFO util.Utils: Successfully started service 'WorkerUI' on port 8081.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,710 INFO ui.WorkerWebUI: Bound WorkerWebUI to 0.0.0.0, and started at
<a href="https://urldefense.com/v3/__http:/mpi4spark000:-1__;!!KGKeukY!3NXRmAG6eoLkZWP5gVYrEkoDx_fki5KEAjcJEOyLIUlOcmopE4jesI-LWd7MS8NM69GSnuqN3rBbunjqS0bJMG778G3nT819oP5I$">
http://mpi4spark000:-1</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,722 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@8e107a9%7b/logPage,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@8e107a9{/logPage,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,724 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@4375cbe0%7b/logPage/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@4375cbe0{/logPage/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,724 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@5e18e1c2%7b/,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@5e18e1c2{/,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,726 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@6ef7379%7b/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@6ef7379{/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,731 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@548ada47%7b/static,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@548ada47{/static,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,731 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@4a02c874%7b/log,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@4a02c874{/log,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,732 INFO worker.Worker: Connecting to master mpi4spark002:7077...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,742 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@73e20207%7b/metrics/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@73e20207{/metrics/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,775 INFO client.TransportClientFactory: Successfully created connection to mpi4spark002/200.2.151.2:7077 after 31 ms (0 ms spent in bootstraps)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,837 INFO worker.Worker: Successfully registered with master spark://mpi4spark002:7077<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,944 INFO worker.Worker: Asked to launch executor app-20240522163753-0000/0 for Spark Pi<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,959 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,959 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,959 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,960 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:53,960 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with
view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Worker1.log<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">starting org.apache.spark.deploy.worker.Worker, logging to /root/mpi4spark/mpi4spark-0.2-x86-bin/logs/spark-root-org.apache.spark.deploy.worker.Worker-1-mpi4spark001.out</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">Spark Command: /root/mpi4spark/jdk1.8.0_321//bin/java -cp /root/mpi4spark/mpi4spark-0.2-x86-bin//conf/:/root/mpi4spark/mpi4spark-0.2-x86-bin/jars/*:/root/mpi4spark/hadoop-3.3.4//etc/hadoop/
-Xmx1g org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://mpi4spark002:7077<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">========================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,720 INFO worker.Worker: Started daemon with process name: 2883486@mpi4spark001<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,723 INFO util.SignalUtils: Registering signal handler for TERM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,724 INFO util.SignalUtils: Registering signal handler for HUP<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,724 INFO util.SignalUtils: Registering signal handler for INT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,936 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,971 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,971 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,972 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,972 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:51,972 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with
view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,397 INFO nio.NioEventLoop: Starting MPI4Spark.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,506 INFO util.Utils: Successfully started service 'sparkWorker' on port 46337.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,507 INFO worker.Worker: Worker decommissioning not enabled.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,616 INFO worker.Worker: Starting Spark worker 200.2.149.6:46337 with 64 cores, 502.3 GiB RAM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,619 INFO worker.Worker: Running Spark version 3.3.0-SNAPSHOT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,620 INFO worker.Worker: Spark home: /root/mpi4spark/mpi4spark-0.2-x86-bin<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,632 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,632 INFO resource.ResourceUtils: No custom resources configured for spark.worker.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,632 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,651 INFO util.log: Logging initialized @2328ms to org.sparkproject.jetty.util.log.Slf4jLog<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,685 INFO server.Server: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_321-b07<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,696 INFO server.Server: Started @2373ms<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,723 INFO server.AbstractConnector: Started
<a href="mailto:ServerConnector@679db676%7bHTTP/1.1">ServerConnector@679db676{HTTP/1.1</a>, (http/1.1)}{0.0.0.0:8081}<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,723 INFO util.Utils: Successfully started service 'WorkerUI' on port 8081.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,724 INFO ui.WorkerWebUI: Bound WorkerWebUI to 0.0.0.0, and started at
<a href="https://urldefense.com/v3/__http:/mpi4spark001:-1__;!!KGKeukY!3NXRmAG6eoLkZWP5gVYrEkoDx_fki5KEAjcJEOyLIUlOcmopE4jesI-LWd7MS8NM69GSnuqN3rBbunjqS0bJMG778G3nT8EPCLCb$">
http://mpi4spark001:-1</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,737 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@648bb0dd%7b/logPage,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@648bb0dd{/logPage,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,738 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@15e3912%7b/logPage/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@15e3912{/logPage/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,739 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@5f9f4fef%7b/,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@5f9f4fef{/,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,740 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@50387b20%7b/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@50387b20{/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,745 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1117f451%7b/static,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1117f451{/static,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,746 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@35f11bf7%7b/log,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@35f11bf7{/log,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,747 INFO worker.Worker: Connecting to master mpi4spark002:7077...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,757 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@113317dd%7b/metrics/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@113317dd{/metrics/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,785 INFO client.TransportClientFactory: Successfully created connection to mpi4spark002/200.2.151.2:7077 after 25 ms (0 ms spent in bootstraps)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,835 INFO worker.Worker: Successfully registered with master spark://mpi4spark002:7077<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,946 INFO worker.Worker: Asked to launch executor app-20240522163753-0000/1 for Spark Pi<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,961 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,961 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,961 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,961 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:53,961 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with
view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">app.log</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:53,006 INFO spark.SparkContext: Running Spark version 3.3.0-SNAPSHOT</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,043 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,116 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,116 INFO resource.ResourceUtils: No custom resources configured for spark.driver.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,117 INFO resource.ResourceUtils: ==============================================================<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,117 INFO spark.SparkContext: Submitted application: Spark Pi<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,133 INFO resource.ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor:
, memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,141 INFO resource.ResourceProfile: Limiting resource is cpu<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,141 INFO resource.ResourceProfileManager: Added ResourceProfile id: 0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,173 INFO spark.SecurityManager: Changing view acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,174 INFO spark.SecurityManager: Changing modify acls to: root<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,174 INFO spark.SecurityManager: Changing view acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,174 INFO spark.SecurityManager: Changing modify acls groups to:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,174 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with
view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,395 INFO nio.NioEventLoop: Starting MPI4Spark.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,462 INFO util.Utils: Successfully started service 'sparkDriver' on port 45597.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,478 INFO spark.SparkEnv: Registering MapOutputTracker<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,498 INFO spark.SparkEnv: Registering BlockManagerMaster<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,509 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,510 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,512 INFO spark.SparkEnv: Registering BlockManagerMasterHeartbeat<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,524 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-4d5b7f26-5a1f-4c8c-9b8c-f60132a6f261<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,538 INFO memory.MemoryStore: MemoryStore started with capacity 408.9 MiB<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,548 INFO spark.SparkEnv: Registering OutputCommitCoordinator<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,571 INFO util.log: Logging initialized @1227ms to org.sparkproject.jetty.util.log.Slf4jLog<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,641 INFO server.Server: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_321-b07<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,654 INFO server.Server: Started @1311ms<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,680 INFO server.AbstractConnector: Started
<a href="mailto:ServerConnector@5225305d%7bHTTP/1.1">ServerConnector@5225305d{HTTP/1.1</a>, (http/1.1)}{0.0.0.0:4040}<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,680 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,681 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at
<a href="https://urldefense.com/v3/__http:/mpi4spark003:-1__;!!KGKeukY!3NXRmAG6eoLkZWP5gVYrEkoDx_fki5KEAjcJEOyLIUlOcmopE4jesI-LWd7MS8NM69GSnuqN3rBbunjqS0bJMG778G3nT85cx_0v$">
http://mpi4spark003:-1</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,694 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@3fcdcf%7b/,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@3fcdcf{/,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,703 INFO spark.SparkContext: Added JAR file:/root/mpi4spark/mpi4spark-0.2-x86-bin/examples/jars/spark-examples_2.12-3.3.0-SNAPSHOT.jar at spark://mpi4spark003:45597/jars/spark-examples_2.12-3.3.0-SNAPSHOT.jar
with timestamp 1716367073001<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,818 INFO client.StandaloneAppClient$ClientEndpoint: Connecting to master spark://mpi4spark002:7077...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,853 INFO client.TransportClientFactory: Successfully created connection to mpi4spark002/200.2.151.2:7077 after 24 ms (0 ms spent in bootstraps)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,912 INFO cluster.StandaloneSchedulerBackend: Connected to Spark cluster with app ID app-20240522163753-0000<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,915 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 36911.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,916 INFO netty.NettyBlockTransferService: Server created on mpi4spark003:36911<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,916 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,920 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, mpi4spark003, 36911, None)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,922 INFO storage.BlockManagerMasterEndpoint: Registering block manager mpi4spark003:36911 with 408.9 MiB RAM, BlockManagerId(driver, mpi4spark003,
36911, None)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,923 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, mpi4spark003, 36911, None)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,924 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, mpi4spark003, 36911, None)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,929 INFO client.StandaloneAppClient$ClientEndpoint: Executor added: app-20240522163753-0000/0 on worker-20240522163753-200.2.149.2-40247 (200.2.149.2:40247)
with 64 core(s)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,930 INFO cluster.StandaloneSchedulerBackend: Granted executor ID app-20240522163753-0000/0 on hostPort 200.2.149.2:40247 with 64 core(s), 1024.0
MiB RAM<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:53,930 INFO client.StandaloneAppClient$ClientEndpoint: Executor added: app-20240522163753-0000/1 on worker-20240522163753-200.2.149.6-46337 (200.2.149.6:46337)
with 64 core(s)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,020 INFO handler.ContextHandler: Stopped
<a href="mailto:o.s.j.s.ServletContextHandler@3fcdcf%7b/,null,STOPPED,@Spark%7d">
o.s.j.s.ServletContextHandler@3fcdcf{/,null,STOPPED,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,021 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@28d79cba%7b/jobs,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@28d79cba{/jobs,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,021 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@29f0c4f2%7b/jobs/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@29f0c4f2{/jobs/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,022 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@13047d7d%7b/jobs/job,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@13047d7d{/jobs/job,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,022 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@65bb9029%7b/jobs/job/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@65bb9029{/jobs/job/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,022 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@2b214b94%7b/stages,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@2b214b94{/stages,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,023 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@49601f82%7b/stages/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@49601f82{/stages/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,023 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@24fabd0f%7b/stages/stage,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@24fabd0f{/stages/stage,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:54,024 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@61f3fbb8%7b/stages/stage/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@61f3fbb8{/stages/stage/json,null,AVAILABLE,@Spark}</a></span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:37:54,024 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@432034a%7b/stages/pool,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@432034a{/stages/pool,null,AVAILABLE,@Spark}</a></span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,025 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@60e5272%7b/stages/pool/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@60e5272{/stages/pool/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,025 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@69c93ca4%7b/storage,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@69c93ca4{/storage,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,025 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@173373b4%7b/storage/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@173373b4{/storage/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,026 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@60dd3c23%7b/storage/rdd,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@60dd3c23{/storage/rdd,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,026 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@5e9456ae%7b/storage/rdd/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@5e9456ae{/storage/rdd/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,026 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@1f1cae23%7b/environment,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@1f1cae23{/environment,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,027 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@985696%7b/environment/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@985696{/environment/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,027 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@215a34b4%7b/executors,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@215a34b4{/executors,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,027 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@35d3ab60%7b/executors/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@35d3ab60{/executors/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,028 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@71870da7%7b/executors/threadDump,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@71870da7{/executors/threadDump,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,028 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@45792847%7b/executors/threadDump/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@45792847{/executors/threadDump/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,034 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@4e25147a%7b/static,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@4e25147a{/static,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,034 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@5f303ecd%7b/,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@5f303ecd{/,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,035 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@25a73de1%7b/api,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@25a73de1{/api,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,035 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@260f2144%7b/jobs/job/kill,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@260f2144{/jobs/job/kill,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,036 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@51827393%7b/stages/stage/kill,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@51827393{/stages/stage/kill,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,038 INFO handler.ContextHandler: Started
<a href="mailto:o.s.j.s.ServletContextHandler@7ec58feb%7b/metrics/json,null,AVAILABLE,@Spark%7d">
o.s.j.s.ServletContextHandler@7ec58feb{/metrics/json,null,AVAILABLE,@Spark}</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,038 INFO cluster.StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,302 INFO spark.SparkContext: Starting job: reduce at SparkPi.scala:38<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,311 INFO scheduler.DAGScheduler: Got job 0 (reduce at SparkPi.scala:38) with 2 output partitions<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,311 INFO scheduler.DAGScheduler: Final stage: ResultStage 0 (reduce at SparkPi.scala:38)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,311 INFO scheduler.DAGScheduler: Parents of final stage: List()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,312 INFO scheduler.DAGScheduler: Missing parents: List()<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,314 INFO scheduler.DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1] at map at SparkPi.scala:34), which has no missing parents<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,349 INFO memory.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 4.0 KiB, free 408.9 MiB)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,365 INFO memory.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 2.3 KiB, free 408.9 MiB)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,367 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on mpi4spark003:36911 (size: 2.3 KiB, free: 408.9 MiB)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,368 INFO spark.SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1429<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,376 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at map at SparkPi.scala:34) (first 15 tasks
are for partitions Vector(0, 1))<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:37:54,377 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 2 tasks resource profile 0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:38:09,388 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:38:24,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:38:39,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:38:54,388 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:39:09,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:39:24,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:39:39,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:39:54,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:40:09,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:40:24,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:40:39,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:40:54,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:41:09,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black">2024-05-22 16:41:24,388 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
and have sufficient resources<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black;border:none windowtext 1.0pt;padding:0in">2024-05-22 16:41:39,387 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster
UI to ensure that workers are registered and have sufficient resources</span><span style="font-size:10.5pt;font-family:"Tahoma",sans-serif;color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>