[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version

Thu Mar 13 15:31:06 EDT 2014

Thanks for the info Matt.  I think that this works well for the PGI
compilers and should be safe for you guys to use.  I'll try to get
something like this into one of our upcoming releases once we are able
to verify that there are not side effects on other compilers.  Thanks
again.

On Thu, Mar 13, 2014 at 3:26 PM, Thompson, Matt (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
> Jonathan,
>
> It turns out this doesn't actually work. The issue seem to be that aside
> from the non-zero error return code, f2py doesn't like the extra information
> about mvapich2 that the "echo" is returning about the MPI version:
>
>> $ mpif90 -V
>> mpif90 for MVAPICH2 version 2.0b
>>
>>
>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>> The Portland Group - PGI Compilers and Tools
>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>
>
> I did something like this:
>
>> (1057) $ diff -u mpif90.old  mpif90.new.sepV
>> [discover29:~ 3:20pm]
>> --- mpif90.old  2014-03-13 15:19:59.743561044 -0400
>> +++ mpif90.new.sepV     2014-03-13 15:20:43.483557824 -0400
>> @@ -201,6 +201,12 @@
>>          linking=no
>>      fi
>>      ;;
>> +    -V|--version)
>> +    # For -V and --version, do not echo the MVAPICH2 version
>> +    if [ "$#" -eq "1" ] ; then
>> +        linking=no
>> +    fi
>> +    ;;
>>      -profile=*)
>>      # Pass the name of a profiling configuration.  As
>>      # a special case, lib<name>.so or lib<name>.la may be used
>
>
> and I confirmed that this *does* work. Apparently like my scripts, f2py is
> expecting "mpif90 -V" to return identically what "pgf90 -V" does to the
> character.
>
>
> Matt
>
>
> On 03/13/2014 10:25 AM, Jonathan Perkins wrote:
>>
>> Hi, I haven't had a chance to verify that there aren't any side
>> effects with the -V option yet but this is the patch that resolves the
>> issue for us.  You can try this out and let us know if it works for
>> you.
>>
>> --- mpif90.old    2014-03-13 10:21:53.085899344 -0400
>> +++ mpif90.new    2014-03-13 10:21:31.928917828 -0400
>> @@ -190,7 +190,7 @@
>>       Show=echo
>>       addarg=no
>>       ;;
>> -    -v)
>> +    -v|-V|--version)
>>       # Pass this argument to the compiler as well.
>>       echo "mpif90 for MVAPICH2 version $MVAPICH2_VERSION"
>>       # if there is only 1 argument, it must be -v.
>>
>> On Thu, Mar 13, 2014 at 9:52 AM, Jonathan Perkins
>> <perkinjo at cse.ohio-state.edu> wrote:
>>>
>>> Thanks for bringing this to our attention.  We haven't noticed a
>>> problem here ourselves but your explanation leads me to believe that
>>> you can try the work around.
>>>
>>> We'll try to reproduce this problem and, if successful, see how best
>>> to resolve it.
>>>
>>> On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
>>> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>>>>
>>>> MVAPICH List,
>>>>
>>>> This is a bit of an odd issue, but I'm hoping someone here can help. In
>>>> the
>>>> code I work with, we use f2py. Due to the fact that some sub-libraries
>>>> (HDF5
>>>> and netCDF) were compiled with parallel support, that means the compiler
>>>> they were technically built with was "mpif90" (and/or "mpif77"). So,
>>>> when we
>>>> run f2py, we have to run:
>>>>
>>>>     $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>>>>
>>>> so that it can pick up the MPI libraries when it compiles.
>>>>
>>>> The compilers I use are, mainly, Intel and PGI, and since I maintain
>>>> some
>>>> GPU code, PGI is often my focus. That said, I'm having an issue trying
>>>> to
>>>> use PGI and MVAPICH2 in combination that prevents me from exploring the
>>>> GPU
>>>> enhancements of MVAPICH2.
>>>>
>>>> I think the issue boils down to behavior upon returning the version for
>>>> the
>>>> compiler through the MPI wrapper. When I traced through f2py's automatic
>>>> compiler detection, it seems to depend on a successful return code from
>>>> a
>>>> version check to say "Yes, the --f90exec is installed and valid." (I'm
>>>> pretty sure of this, but it took a while to trace through.)
>>>>
>>>> To wit, when I just use PGI 14.1:
>>>>
>>>>> $ pgfortran -V
>>>>>
>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> $ echo $?
>>>>> $ pgfortran --version
>>>>>
>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> $ echo $?
>>>>> 0
>>>>
>>>>
>>>>
>>>> So, if you use -V or --version, you get the usual spiel and a return
>>>> code of
>>>> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the version
>>>> under mpif90, I get this behavior:
>>>>
>>>>> $ mpif90 -V
>>>>>
>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> (1427) $ echo $?
>>>>> 0
>>>>> $ mpif90 --version
>>>>>
>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> $ echo $?
>>>>> 0
>>>>
>>>>
>>>>
>>>> Again, I get the same behavior as the bare compiler with return codes of
>>>> 0.
>>>>
>>>> Now, PGI 14.1 and MVAPICH2 2.0b:
>>>>
>>>>> $ mpif90 -V
>>>>>
>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>> `main':
>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>> $ echo $?
>>>>> 2
>>>>> $ mpif90 --version
>>>>>
>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>> The Portland Group - PGI Compilers and Tools
>>>>> Copyright (c) 2014, NVIDIA CORPORATION.  All rights reserved.
>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>> `main':
>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>> $ echo $?
>>>>> 2
>>>>
>>>>
>>>>
>>>> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with
>>>> f2py,
>>>> when you run mpif90 --version/-V with MVAPICH2, it returns an error code
>>>> of
>>>> 2, not 0.
>>>>
>>>> Is there a reason for this? I'm especially surprised at the --version
>>>> behavior. Are there compilers/situations that when you do "$FC
>>>> --version"
>>>> one should expect a "main" program to link?
>>>>
>>>> The one simple workaround I can think of is to change the mpif90 script
>>>> itself:
>>>>
>>>>> --- mpif90_withchange   2014-03-13 09:25:24.496774411 -0400
>>>>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>>>>> 2014-01-31 13:32:39.033436000 -0500
>>>>> @@ -151,7 +151,7 @@
>>>>>       case "$arg" in
>>>>>          #
>>>>> ----------------------------------------------------------------
>>>>>          # Compiler options that affect whether we are linking or no
>>>>> -    -c|-S|-E|-M|-MM|-V|--version)
>>>>> +    -c|-S|-E|-M|-MM)
>>>>>       # The compiler links by default
>>>>>       linking=no
>>>>>       ;;
>>>>
>>>>
>>>>
>>>> but as that script (and mpif77) is fairly fundamental to MVAPICH2, well,
>>>> I'm
>>>> a bit hesitant to have my sysadmins patch it every time they install for
>>>> fear of breaking...everything! (Of course, I might have them try it to
>>>> see
>>>> if it helps.)
>>>>
>>>> Would you have an objection (if I find it works) to adding these as
>>>> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve the
>>>> PGI
>>>> ability to change compilers on the fly with -V<ver> since, in bash,
>>>> "-V14.3"
>>>> does not equal "-V".
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> --
>>>> Matt Thompson, PhD     SSAI, Sr Software Test Engr
>>>> NASA GSFC, Global Modeling and Assimilation Office
>>>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>>>> Phone: 301-614-6712              Fax: 301-614-6246
>>>>
>>>> _______________________________________________
>>>> mvapich-discuss mailing list
>>>> mvapich-discuss at cse.ohio-state.edu
>>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Jonathan Perkins
>>> http://www.cse.ohio-state.edu/~perkinjo
>>
>>
>>
>>
>
>
> --
> Matt Thompson          SSAI, Sr Software Test Engr
>
> NASA GSFC, Global Modeling and Assimilation Office
> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
> Phone: 301-614-6712              Fax: 301-614-6246
>
>

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo