[Columbus] Compilation problem of parallel Columbus (ga-5.0)

[Felix Plasser]

Hi, can you run any parallel MR-CI job at all? Just something small on a 
few cores on one node?

-Felix

On 12/09/2015 05:55 PM, Max Pinheiro Jr wrote:
> Dear Columbus team,
>
> I am trying to run some parallel calculations with Columbus in a Cray 
> linux cluster. I have compiled the parallel version of the code using 
> the ga-5.0 libraries and the intel compiler 16.0 with success. 
> However, when I try to run the set of parallel tests of Columbus 
> during the pciudg.x routine printing out the following error message 
> in the runc.error file:
>
>           starting ci iteration   1
>
> 0:0:ga_ndim: ARRAY NOT ACTIVE:: 0
> (rank:0 hostname:eos01 pid:128811):ARMCI DASSERT fail. 
> src/armci.c:ARMCI_Error():276 cond:0
> Last System Error Message from Task 0:: Inappropriate ioctl for device
> application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0
> Last System Error Message from Task 1:: Inappropriate ioctl for device
> application called MPI_Abort(MPI_COMM_WORLD, 2) - process 1
>
> According to the mailing lists of other codes (NWchem and Molpro), 
> this problem seems to be related with the compilation of the global 
> array libraries but I haven't found a clear solution for this error so 
> far. The configure instructions I have used to compile the global 
> array 5.0 is in line with that suggested by other codes in the case of 
> intel and impi but I am not sure if these flags work also for 
> Columbus. The configure parameters are:
>
> ./configure --with-tcgmsg --prefix=/u/maxjr/codes/ga-5.0 CC=icc 
> F77=ifort --enable-i8 --enable-autodetect --enable-armci-group 
> --with-mpi=/afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64
>
> For the Columbus compilation, I am using the following config file:
>
> GREP /bin/grep
> GMAKE make
> TAR /bin/tar
> RANLIB /usr/bin/ranlib
> CPPL cpp -C -E -traditional
> PERL /usr/bin/perl
> BLASLIBRARY -Wl,--start-group 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_intel_ilp64.a 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_core.a 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_sequential.a 
> -Wl,--end-group -lpthread -lm
> LAPACKLIBRARY  -Wl,--start-group 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_intel_ilp64.a 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_core.a 
> /afs/@cell/common/soft/intel/ics2016/mkl/lib/intel64_lin/libmkl_sequential.a 
> -Wl,--end-group -lpthread -lm
> MACHINEID linux64.ifc.byterecl
> GUNZIP /bin/gunzip
> GACOMMUNICATION MPI
> MPI_MAINDIR /afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/lib
> MPI_CC mpiicc
> MPI_LD mpiifort -Wl,-z,muldefs
> MPI_FC mpiifort -Wl,-z,muldefs
> PSCRIPT
> GAVERSION GA50
> MPI_LIBS 
> /afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/lib/libmpi_ilp64.so.4.1 
> -libverbs -libumad
> MPI_STARTUP 
> /afs/@cell/common/soft/intel/ics2016/impi/5.1.1/intel64/bin/mpirun -v 
> -np _NPROC_  _EXE_ _EXEOPTS_
> COLUMBUS /u/maxjr/codes/columbus-7.0/Columbus
> DALTON /u/maxjr/codes/columbus-7.0/Columbus/source/dalton
> COLUMBUSVERSION 7.0
> INSTALLOPTION NOGA_INSTALL
> cpan
> standard
> grad
> parallel
>
> The computer system where I am try to run the calculations has 32 
> Intel Xeon E5-2698 cpus per node and ~120 GB of ram memory per node. 
> So this probably is not a problem of memory restrictions. Anyway, I 
> have tried to run a parallel calculation increasing the maximum ARMCI 
> shared memory by setting the flag ARMCI_DEFAULT_SHMMAX in the 
> submission script, as suggested in some forum discussions of other 
> codes with similar problems, but the calculations fail again in the 
> same step (pciudg.x).
>
> Please, anybody has a suggestion to fix this problem? I will greatly 
> appreciate to resolve this problem.
>
> Thanks & All the best,
>
> Max Pinheiro Jr
> Pos-doctoral researcher at Instituto Tecnológico de Aeronáutica - ITA
>
>
>
>
> _______________________________________________
> Columbus mailing list
> Columbus at lists.osc.edu
> https://lists.osu.edu/mailman/listinfo/columbus

-- 
Felix Plasser
Institute for Theoretical Chemistry
University of Vienna
Währingerstr. 17/404
1090 Wien
felix.plasser at univie.ac.at
http://homepage.univie.ac.at/felix.plasser/
Tel.: +43-1-4277-52757

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/columbus/attachments/20151209/aa3f8ca9/attachment.html>