<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi, can you run any parallel MR-CI job at all? Just something small
on a few cores on one node?<br>
<br>
-Felix<br>
<br>
<div class="moz-cite-prefix">On 12/09/2015 05:55 PM, Max Pinheiro Jr
wrote:<br>
</div>
<blockquote
cite="mid:CAMpKNf_U1+=zBYUqoMD==PN2GaFh+jRvVUJ0+Cm9532VkWKECw@mail.gmail.com"
type="cite">
<div dir="ltr"><font color="#000000">Dear Columbus team,</font>
<div><font color="#000000"><br>
</font></div>
<div>
<div><font color="#000000">I am trying to run some parallel
calculations with Columbus in a Cray linux cluster. I have
compiled the parallel version of the code using the ga-5.0
libraries and the intel compiler 16.0 with success.
However, when I try to run the set of parallel tests of
Columbus during the pciudg.x routine printing out the
following error message in the runc.error file:</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000"> starting ci iteration 1</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">0:0:ga_ndim: ARRAY NOT ACTIVE:: 0</font></div>
<div><font color="#000000">(rank:0 hostname:eos01
pid:128811):ARMCI DASSERT fail.
src/armci.c:ARMCI_Error():276 cond:0</font></div>
<div><font color="#000000">Last System Error Message from Task
0:: Inappropriate ioctl for device</font></div>
<div><font color="#000000">application called
MPI_Abort(MPI_COMM_WORLD, 0) - process 0</font></div>
<div><font color="#000000">Last System Error Message from Task
1:: Inappropriate ioctl for device</font></div>
<div><font color="#000000">application called
MPI_Abort(MPI_COMM_WORLD, 2) - process 1</font></div>
</div>
<div><font color="#000000"><br>
</font></div>
<div>
<div><font color="#000000">According to the mailing lists of
other codes (NWchem and Molpro), this problem seems to be
related with the compilation of the global array libraries
but I haven't found a clear solution for this error so
far. The configure instructions I have used to compile the
global array 5.0 is in line with that suggested by other
codes in the case of intel and impi but I am not sure if
these flags work also for Columbus. The configure
parameters are:</font></div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">./configure --with-tcgmsg
--prefix=/u/maxjr/codes/ga-5.0 CC=icc F77=ifort
--enable-i8 --enable-autodetect --enable-armci-group
--with-mpi=/afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64</font></div>
</div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">For the Columbus compilation, I am
using the following config file:</font></div>
<div><font color="#000000"><br>
</font></div>
<div>
<div><font color="#000000">GREP /bin/grep</font></div>
<div><font color="#000000">GMAKE make</font></div>
<div><font color="#000000">TAR /bin/tar</font></div>
<div><font color="#000000">RANLIB /usr/bin/ranlib</font></div>
<div><font color="#000000">CPPL cpp -C -E -traditional</font></div>
<div><font color="#000000">PERL /usr/bin/perl</font></div>
<div><font color="#000000">BLASLIBRARY -Wl,--start-group
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_intel_ilp64.a
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_core.a
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_sequential.a
-Wl,--end-group -lpthread -lm</font></div>
<div><font color="#000000">LAPACKLIBRARY -Wl,--start-group
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_intel_ilp64.a
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_core.a
/afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_sequential.a
-Wl,--end-group -lpthread -lm</font></div>
<div><font color="#000000">MACHINEID linux64.ifc.byterecl</font></div>
<div><font color="#000000">GUNZIP /bin/gunzip</font></div>
<div><font color="#000000">GACOMMUNICATION MPI</font></div>
<div><font color="#000000">MPI_MAINDIR
/afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/lib</font></div>
<div><font color="#000000">MPI_CC mpiicc</font></div>
<div><font color="#000000">MPI_LD mpiifort -Wl,-z,muldefs</font></div>
<div><font color="#000000">MPI_FC mpiifort -Wl,-z,muldefs</font></div>
<div><font color="#000000">PSCRIPT </font></div>
<div><font color="#000000">GAVERSION GA50</font></div>
<div><font color="#000000">MPI_LIBS
/afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/lib/libmpi_ilp64.so.4.1
-libverbs -libumad</font></div>
<div><font color="#000000">MPI_STARTUP
/afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/bin/mpirun
-v -np _NPROC_ _EXE_ _EXEOPTS_</font></div>
<div><font color="#000000">COLUMBUS
/u/maxjr/codes/columbus-7.0/Columbus</font></div>
<div><font color="#000000">DALTON
/u/maxjr/codes/columbus-7.0/Columbus/source/dalton</font></div>
<div><font color="#000000">COLUMBUSVERSION 7.0</font></div>
<div><font color="#000000">INSTALLOPTION NOGA_INSTALL</font></div>
<div><font color="#000000">cpan </font></div>
<div><font color="#000000">standard </font></div>
<div><font color="#000000">grad </font></div>
<div><font color="#000000">parallel</font></div>
</div>
<div><font color="#000000"><br>
</font></div>
<div><font color="#000000">The computer system where I am try to
run the calculations has 32 <span
style="font-family:Verdana,Geneva,Arial,sans-serif;font-size:12.8px;line-height:19.2px;background-color:rgb(248,248,248)">Intel
Xeon E5-2698 cpus per node and ~120 GB of ram memory per
node. So this probably is not a problem of memory
restrictions. Anyway, </span></font><font face="Verdana,
Geneva, Arial, sans-serif" color="#000000"><span
style="font-size:12.8px;line-height:19.2px">I
have tried to run a parallel calculation increasing </span></font><span
style="color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif;font-size:12.8px;line-height:19.2px">the
maximum ARMCI shared memory by setting the flag
ARMCI_DEFAULT_SHMMAX in the submission script, as suggested
in </span><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif">some
forum discussions of other codes with similar problems, but
the calculations fail again in the same step (pciudg.x).</span></div>
<div><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif"><br>
</span></div>
<div><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif">Please,
anybody has a suggestion to fix this problem? I will greatly
appreciate to resolve this problem.</span></div>
<div><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif"><br>
</span></div>
<div><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif">Thanks
& All the best,</span></div>
<div><span
style="font-size:12.8px;line-height:19.2px;color:rgb(0,0,0);font-family:Verdana,Geneva,Arial,sans-serif"><br>
</span></div>
<div>Max Pinheiro Jr</div>
<div>Pos-doctoral researcher at Instituto Tecnológico de
Aeronáutica - ITA</div>
<div><font face="Verdana, Geneva, Arial, sans-serif"
color="#000000"><span
style="font-size:12.8px;line-height:19.2px"><br>
</span></font></div>
<div><font face="Verdana, Geneva, Arial, sans-serif"
color="#000000"><span
style="font-size:12.8px;line-height:19.2px"><br>
</span></font></div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Columbus mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Columbus@lists.osc.edu">Columbus@lists.osc.edu</a>
<a class="moz-txt-link-freetext" href="https://lists.osu.edu/mailman/listinfo/columbus">https://lists.osu.edu/mailman/listinfo/columbus</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Felix Plasser
Institute for Theoretical Chemistry
University of Vienna
Währingerstr. 17/404
1090 Wien
<a class="moz-txt-link-abbreviated" href="mailto:felix.plasser@univie.ac.at">felix.plasser@univie.ac.at</a>
<a class="moz-txt-link-freetext" href="http://homepage.univie.ac.at/felix.plasser/">http://homepage.univie.ac.at/felix.plasser/</a>
Tel.: +43-1-4277-52757</pre>
</body>
</html>