[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version
Jonathan Perkins
perkinjo at cse.ohio-state.edu
Thu May 29 10:08:02 EDT 2014
Hi Matt. Thanks for the follow up. I was not able to find a portable
way to handle -V for the various compilers which is likely the reason
why the patch was never applied. What I think we can do is at least
apply the portion of the patch which will all --version to avoid
linking. Do you think this will be enough for your use case?
On Thu, May 29, 2014 at 9:15 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
> Jonathan,
>
> Has there been any progress with this on your end? I looked at MVAPICH2
> 2.0rc2 today and saw it still had the same issue as before (-V and --version
> lead to linking).
>
> Matt
>
>
> On 03/13/2014 03:31 PM, Jonathan Perkins wrote:
>>
>> Thanks for the info Matt. I think that this works well for the PGI
>> compilers and should be safe for you guys to use. I'll try to get
>> something like this into one of our upcoming releases once we are able
>> to verify that there are not side effects on other compilers. Thanks
>> again.
>>
>> On Thu, Mar 13, 2014 at 3:26 PM, Thompson, Matt (GSFC-610.1)[SCIENCE
>> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>>>
>>> Jonathan,
>>>
>>> It turns out this doesn't actually work. The issue seem to be that aside
>>> from the non-zero error return code, f2py doesn't like the extra
>>> information
>>> about mvapich2 that the "echo" is returning about the MPI version:
>>>
>>>> $ mpif90 -V
>>>> mpif90 for MVAPICH2 version 2.0b
>>>>
>>>>
>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>
>>>
>>>
>>> I did something like this:
>>>
>>>> (1057) $ diff -u mpif90.old mpif90.new.sepV
>>>> [discover29:~ 3:20pm]
>>>> --- mpif90.old 2014-03-13 15:19:59.743561044 -0400
>>>> +++ mpif90.new.sepV 2014-03-13 15:20:43.483557824 -0400
>>>> @@ -201,6 +201,12 @@
>>>> linking=no
>>>> fi
>>>> ;;
>>>> + -V|--version)
>>>> + # For -V and --version, do not echo the MVAPICH2 version
>>>> + if [ "$#" -eq "1" ] ; then
>>>> + linking=no
>>>> + fi
>>>> + ;;
>>>> -profile=*)
>>>> # Pass the name of a profiling configuration. As
>>>> # a special case, lib<name>.so or lib<name>.la may be used
>>>
>>>
>>>
>>> and I confirmed that this *does* work. Apparently like my scripts, f2py
>>> is
>>> expecting "mpif90 -V" to return identically what "pgf90 -V" does to the
>>> character.
>>>
>>>
>>> Matt
>>>
>>>
>>> On 03/13/2014 10:25 AM, Jonathan Perkins wrote:
>>>>
>>>>
>>>> Hi, I haven't had a chance to verify that there aren't any side
>>>> effects with the -V option yet but this is the patch that resolves the
>>>> issue for us. You can try this out and let us know if it works for
>>>> you.
>>>>
>>>> --- mpif90.old 2014-03-13 10:21:53.085899344 -0400
>>>> +++ mpif90.new 2014-03-13 10:21:31.928917828 -0400
>>>> @@ -190,7 +190,7 @@
>>>> Show=echo
>>>> addarg=no
>>>> ;;
>>>> - -v)
>>>> + -v|-V|--version)
>>>> # Pass this argument to the compiler as well.
>>>> echo "mpif90 for MVAPICH2 version $MVAPICH2_VERSION"
>>>> # if there is only 1 argument, it must be -v.
>>>>
>>>> On Thu, Mar 13, 2014 at 9:52 AM, Jonathan Perkins
>>>> <perkinjo at cse.ohio-state.edu> wrote:
>>>>>
>>>>>
>>>>> Thanks for bringing this to our attention. We haven't noticed a
>>>>> problem here ourselves but your explanation leads me to believe that
>>>>> you can try the work around.
>>>>>
>>>>> We'll try to reproduce this problem and, if successful, see how best
>>>>> to resolve it.
>>>>>
>>>>> On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
>>>>> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>>>>>>
>>>>>>
>>>>>> MVAPICH List,
>>>>>>
>>>>>> This is a bit of an odd issue, but I'm hoping someone here can help.
>>>>>> In
>>>>>> the
>>>>>> code I work with, we use f2py. Due to the fact that some sub-libraries
>>>>>> (HDF5
>>>>>> and netCDF) were compiled with parallel support, that means the
>>>>>> compiler
>>>>>> they were technically built with was "mpif90" (and/or "mpif77"). So,
>>>>>> when we
>>>>>> run f2py, we have to run:
>>>>>>
>>>>>> $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>>>>>>
>>>>>> so that it can pick up the MPI libraries when it compiles.
>>>>>>
>>>>>> The compilers I use are, mainly, Intel and PGI, and since I maintain
>>>>>> some
>>>>>> GPU code, PGI is often my focus. That said, I'm having an issue trying
>>>>>> to
>>>>>> use PGI and MVAPICH2 in combination that prevents me from exploring
>>>>>> the
>>>>>> GPU
>>>>>> enhancements of MVAPICH2.
>>>>>>
>>>>>> I think the issue boils down to behavior upon returning the version
>>>>>> for
>>>>>> the
>>>>>> compiler through the MPI wrapper. When I traced through f2py's
>>>>>> automatic
>>>>>> compiler detection, it seems to depend on a successful return code
>>>>>> from
>>>>>> a
>>>>>> version check to say "Yes, the --f90exec is installed and valid." (I'm
>>>>>> pretty sure of this, but it took a while to trace through.)
>>>>>>
>>>>>> To wit, when I just use PGI 14.1:
>>>>>>
>>>>>>> $ pgfortran -V
>>>>>>>
>>>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> $ echo $?
>>>>>>> $ pgfortran --version
>>>>>>>
>>>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> $ echo $?
>>>>>>> 0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> So, if you use -V or --version, you get the usual spiel and a return
>>>>>> code of
>>>>>> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the
>>>>>> version
>>>>>> under mpif90, I get this behavior:
>>>>>>
>>>>>>> $ mpif90 -V
>>>>>>>
>>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> (1427) $ echo $?
>>>>>>> 0
>>>>>>> $ mpif90 --version
>>>>>>>
>>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> $ echo $?
>>>>>>> 0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Again, I get the same behavior as the bare compiler with return codes
>>>>>> of
>>>>>> 0.
>>>>>>
>>>>>> Now, PGI 14.1 and MVAPICH2 2.0b:
>>>>>>
>>>>>>> $ mpif90 -V
>>>>>>>
>>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>>>> `main':
>>>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>>>> $ echo $?
>>>>>>> 2
>>>>>>> $ mpif90 --version
>>>>>>>
>>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>>>> `main':
>>>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>>>> $ echo $?
>>>>>>> 2
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with
>>>>>> f2py,
>>>>>> when you run mpif90 --version/-V with MVAPICH2, it returns an error
>>>>>> code
>>>>>> of
>>>>>> 2, not 0.
>>>>>>
>>>>>> Is there a reason for this? I'm especially surprised at the --version
>>>>>> behavior. Are there compilers/situations that when you do "$FC
>>>>>> --version"
>>>>>> one should expect a "main" program to link?
>>>>>>
>>>>>> The one simple workaround I can think of is to change the mpif90
>>>>>> script
>>>>>> itself:
>>>>>>
>>>>>>> --- mpif90_withchange 2014-03-13 09:25:24.496774411 -0400
>>>>>>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>>>>>>> 2014-01-31 13:32:39.033436000 -0500
>>>>>>> @@ -151,7 +151,7 @@
>>>>>>> case "$arg" in
>>>>>>> #
>>>>>>> ----------------------------------------------------------------
>>>>>>> # Compiler options that affect whether we are linking or no
>>>>>>> - -c|-S|-E|-M|-MM|-V|--version)
>>>>>>> + -c|-S|-E|-M|-MM)
>>>>>>> # The compiler links by default
>>>>>>> linking=no
>>>>>>> ;;
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> but as that script (and mpif77) is fairly fundamental to MVAPICH2,
>>>>>> well,
>>>>>> I'm
>>>>>> a bit hesitant to have my sysadmins patch it every time they install
>>>>>> for
>>>>>> fear of breaking...everything! (Of course, I might have them try it to
>>>>>> see
>>>>>> if it helps.)
>>>>>>
>>>>>> Would you have an objection (if I find it works) to adding these as
>>>>>> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve
>>>>>> the
>>>>>> PGI
>>>>>> ability to change compilers on the fly with -V<ver> since, in bash,
>>>>>> "-V14.3"
>>>>>> does not equal "-V".
>>>>>>
>>>>>> Thanks,
>>>>>> Matt
>>>>>>
>>>>>> --
>>>>>> Matt Thompson, PhD SSAI, Sr Software Test Engr
>>>>>> NASA GSFC, Global Modeling and Assimilation Office
>>>>>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>>>>>> Phone: 301-614-6712 Fax: 301-614-6246
>>>>>>
>>>>>> _______________________________________________
>>>>>> mvapich-discuss mailing list
>>>>>> mvapich-discuss at cse.ohio-state.edu
>>>>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jonathan Perkins
>>>>> http://www.cse.ohio-state.edu/~perkinjo
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Matt Thompson SSAI, Sr Software Test Engr
>>>
>>> NASA GSFC, Global Modeling and Assimilation Office
>>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>>> Phone: 301-614-6712 Fax: 301-614-6246
>>>
>>>
>>
>>
>>
>
>
> --
> Matt Thompson SSAI, Sr Software Test Engr
> NASA GSFC, Global Modeling and Assimilation Office
> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
> Phone: 301-614-6712 Fax: 301-614-6246
>
>
--
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo
More information about the mvapich-discuss
mailing list