[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version

Thu Mar 13 10:25:25 EDT 2014

Hi, I haven't had a chance to verify that there aren't any side
effects with the -V option yet but this is the patch that resolves the
issue for us.  You can try this out and let us know if it works for
you.

--- mpif90.old    2014-03-13 10:21:53.085899344 -0400
+++ mpif90.new    2014-03-13 10:21:31.928917828 -0400
@@ -190,7 +190,7 @@
     Show=echo
     addarg=no
     ;;
-    -v)
+    -v|-V|--version)
     # Pass this argument to the compiler as well.
     echo "mpif90 for MVAPICH2 version $MVAPICH2_VERSION"
     # if there is only 1 argument, it must be -v.

On Thu, Mar 13, 2014 at 9:52 AM, Jonathan Perkins
<perkinjo at cse.ohio-state.edu> wrote:
> Thanks for bringing this to our attention.  We haven't noticed a
> problem here ourselves but your explanation leads me to believe that
> you can try the work around.
>
> We'll try to reproduce this problem and, if successful, see how best
> to resolve it.
>
> On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>> MVAPICH List,
>>
>> This is a bit of an odd issue, but I'm hoping someone here can help. In the
>> code I work with, we use f2py. Due to the fact that some sub-libraries (HDF5
>> and netCDF) were compiled with parallel support, that means the compiler
>> they were technically built with was "mpif90" (and/or "mpif77"). So, when we
>> run f2py, we have to run:
>>
>>    $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>>
>> so that it can pick up the MPI libraries when it compiles.
>>
>> The compilers I use are, mainly, Intel and PGI, and since I maintain some
>> GPU code, PGI is often my focus. That said, I'm having an issue trying to
>> use PGI and MVAPICH2 in combination that prevents me from exploring the GPU
>> enhancements of MVAPICH2.
>>
>> I think the issue boils down to behavior upon returning the version for the
>> compiler through the MPI wrapper. When I traced through f2py's automatic
>> compiler detection, it seems to depend on a successful return code from a
>> version check to say "Yes, the --f90exec is installed and valid." (I'm
>> pretty sure of this, but it took a while to trace through.)
>>
>> To wit, when I just use PGI 14.1:
>>
>>> $ pgfortran -V
>>>
>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> $ echo $?
>>> $ pgfortran --version
>>>
>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> $ echo $?
>>> 0
>>
>>
>> So, if you use -V or --version, you get the usual spiel and a return code of
>> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the version
>> under mpif90, I get this behavior:
>>
>>> $ mpif90 -V
>>>
>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> (1427) $ echo $?
>>> 0
>>> $ mpif90 --version
>>>
>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> $ echo $?
>>> 0
>>
>>
>> Again, I get the same behavior as the bare compiler with return codes of 0.
>>
>> Now, PGI 14.1 and MVAPICH2 2.0b:
>>
>>> $ mpif90 -V
>>>
>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>> $ echo $?
>>> 2
>>> $ mpif90 --version
>>>
>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>> $ echo $?
>>> 2
>>
>>
>> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with f2py,
>> when you run mpif90 --version/-V with MVAPICH2, it returns an error code of
>> 2, not 0.
>>
>> Is there a reason for this? I'm especially surprised at the --version
>> behavior. Are there compilers/situations that when you do "$FC --version"
>> one should expect a "main" program to link?
>>
>> The one simple workaround I can think of is to change the mpif90 script
>> itself:
>>
>>> --- mpif90_withchange   2014-03-13 09:25:24.496774411 -0400
>>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>>> 2014-01-31 13:32:39.033436000 -0500
>>> @@ -151,7 +151,7 @@
>>>      case "$arg" in
>>>         # ----------------------------------------------------------------
>>>         # Compiler options that affect whether we are linking or no
>>> -    -c|-S|-E|-M|-MM|-V|--version)
>>> +    -c|-S|-E|-M|-MM)
>>>      # The compiler links by default
>>>      linking=no
>>>      ;;
>>
>>
>> but as that script (and mpif77) is fairly fundamental to MVAPICH2, well, I'm
>> a bit hesitant to have my sysadmins patch it every time they install for
>> fear of breaking...everything! (Of course, I might have them try it to see
>> if it helps.)
>>
>> Would you have an objection (if I find it works) to adding these as
>> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve the PGI
>> ability to change compilers on the fly with -V<ver> since, in bash, "-V14.3"
>> does not equal "-V".
>>
>> Thanks,
>> Matt
>>
>> --
>> Matt Thompson, PhD     SSAI, Sr Software Test Engr
>> NASA GSFC, Global Modeling and Assimilation Office
>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>> Phone: 301-614-6712              Fax: 301-614-6246
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>
>
>
> --
> Jonathan Perkins
> http://www.cse.ohio-state.edu/~perkinjo



-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo
