[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version

Thu Mar 13 09:52:10 EDT 2014

Thanks for bringing this to our attention.  We haven't noticed a
problem here ourselves but your explanation leads me to believe that
you can try the work around.

We'll try to reproduce this problem and, if successful, see how best
to resolve it.

On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
> MVAPICH List,
>
> This is a bit of an odd issue, but I'm hoping someone here can help. In the
> code I work with, we use f2py. Due to the fact that some sub-libraries (HDF5
> and netCDF) were compiled with parallel support, that means the compiler
> they were technically built with was "mpif90" (and/or "mpif77"). So, when we
> run f2py, we have to run:
>
>    $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>
> so that it can pick up the MPI libraries when it compiles.
>
> The compilers I use are, mainly, Intel and PGI, and since I maintain some
> GPU code, PGI is often my focus. That said, I'm having an issue trying to
> use PGI and MVAPICH2 in combination that prevents me from exploring the GPU
> enhancements of MVAPICH2.
>
> I think the issue boils down to behavior upon returning the version for the
> compiler through the MPI wrapper. When I traced through f2py's automatic
> compiler detection, it seems to depend on a successful return code from a
> version check to say "Yes, the --f90exec is installed and valid." (I'm
> pretty sure of this, but it took a while to trace through.)
>
> To wit, when I just use PGI 14.1:
>
>> $ pgfortran -V
>>
>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> $ echo $?
>> $ pgfortran --version
>>
>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> $ echo $?
>> 0
>
>
> So, if you use -V or --version, you get the usual spiel and a return code of
> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the version
> under mpif90, I get this behavior:
>
>> $ mpif90 -V
>>
>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> (1427) $ echo $?
>> 0
>> $ mpif90 --version
>>
>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> $ echo $?
>> 0
>
>
> Again, I get the same behavior as the bare compiler with return codes of 0.
>
> Now, PGI 14.1 and MVAPICH2 2.0b:
>
>> $ mpif90 -V
>>
>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>> $ echo $?
>> 2
>> $ mpif90 --version
>>
>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>> $ echo $?
>> 2
>
>
> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with f2py,
> when you run mpif90 --version/-V with MVAPICH2, it returns an error code of
> 2, not 0.
>
> Is there a reason for this? I'm especially surprised at the --version
> behavior. Are there compilers/situations that when you do "$FC --version"
> one should expect a "main" program to link?
>
> The one simple workaround I can think of is to change the mpif90 script
> itself:
>
>> --- mpif90_withchange   2014-03-13 09:25:24.496774411 -0400
>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>> 2014-01-31 13:32:39.033436000 -0500
>> @@ -151,7 +151,7 @@
>>      case "$arg" in
>>         # ----------------------------------------------------------------
>>         # Compiler options that affect whether we are linking or no
>> -    -c|-S|-E|-M|-MM|-V|--version)
>> +    -c|-S|-E|-M|-MM)
>>      # The compiler links by default
>>      linking=no
>>      ;;
>
>
> but as that script (and mpif77) is fairly fundamental to MVAPICH2, well, I'm
> a bit hesitant to have my sysadmins patch it every time they install for
> fear of breaking...everything! (Of course, I might have them try it to see
> if it helps.)
>
> Would you have an objection (if I find it works) to adding these as
> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve the PGI
> ability to change compilers on the fly with -V<ver> since, in bash, "-V14.3"
> does not equal "-V".
>
> Thanks,
> Matt
>
> --
> Matt Thompson, PhD     SSAI, Sr Software Test Engr
> NASA GSFC, Global Modeling and Assimilation Office
> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
> Phone: 301-614-6712              Fax: 301-614-6246
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo