[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version
Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
matthew.thompson at nasa.gov
Thu May 29 09:15:21 EDT 2014
Jonathan,
Has there been any progress with this on your end? I looked at MVAPICH2
2.0rc2 today and saw it still had the same issue as before (-V and
--version lead to linking).
Matt
On 03/13/2014 03:31 PM, Jonathan Perkins wrote:
> Thanks for the info Matt. I think that this works well for the PGI
> compilers and should be safe for you guys to use. I'll try to get
> something like this into one of our upcoming releases once we are able
> to verify that there are not side effects on other compilers. Thanks
> again.
>
> On Thu, Mar 13, 2014 at 3:26 PM, Thompson, Matt (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>> Jonathan,
>>
>> It turns out this doesn't actually work. The issue seem to be that aside
>> from the non-zero error return code, f2py doesn't like the extra information
>> about mvapich2 that the "echo" is returning about the MPI version:
>>
>>> $ mpif90 -V
>>> mpif90 for MVAPICH2 version 2.0b
>>>
>>>
>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>> The Portland Group - PGI Compilers and Tools
>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>
>>
>> I did something like this:
>>
>>> (1057) $ diff -u mpif90.old mpif90.new.sepV
>>> [discover29:~ 3:20pm]
>>> --- mpif90.old 2014-03-13 15:19:59.743561044 -0400
>>> +++ mpif90.new.sepV 2014-03-13 15:20:43.483557824 -0400
>>> @@ -201,6 +201,12 @@
>>> linking=no
>>> fi
>>> ;;
>>> + -V|--version)
>>> + # For -V and --version, do not echo the MVAPICH2 version
>>> + if [ "$#" -eq "1" ] ; then
>>> + linking=no
>>> + fi
>>> + ;;
>>> -profile=*)
>>> # Pass the name of a profiling configuration. As
>>> # a special case, lib<name>.so or lib<name>.la may be used
>>
>>
>> and I confirmed that this *does* work. Apparently like my scripts, f2py is
>> expecting "mpif90 -V" to return identically what "pgf90 -V" does to the
>> character.
>>
>>
>> Matt
>>
>>
>> On 03/13/2014 10:25 AM, Jonathan Perkins wrote:
>>>
>>> Hi, I haven't had a chance to verify that there aren't any side
>>> effects with the -V option yet but this is the patch that resolves the
>>> issue for us. You can try this out and let us know if it works for
>>> you.
>>>
>>> --- mpif90.old 2014-03-13 10:21:53.085899344 -0400
>>> +++ mpif90.new 2014-03-13 10:21:31.928917828 -0400
>>> @@ -190,7 +190,7 @@
>>> Show=echo
>>> addarg=no
>>> ;;
>>> - -v)
>>> + -v|-V|--version)
>>> # Pass this argument to the compiler as well.
>>> echo "mpif90 for MVAPICH2 version $MVAPICH2_VERSION"
>>> # if there is only 1 argument, it must be -v.
>>>
>>> On Thu, Mar 13, 2014 at 9:52 AM, Jonathan Perkins
>>> <perkinjo at cse.ohio-state.edu> wrote:
>>>>
>>>> Thanks for bringing this to our attention. We haven't noticed a
>>>> problem here ourselves but your explanation leads me to believe that
>>>> you can try the work around.
>>>>
>>>> We'll try to reproduce this problem and, if successful, see how best
>>>> to resolve it.
>>>>
>>>> On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
>>>> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>>>>>
>>>>> MVAPICH List,
>>>>>
>>>>> This is a bit of an odd issue, but I'm hoping someone here can help. In
>>>>> the
>>>>> code I work with, we use f2py. Due to the fact that some sub-libraries
>>>>> (HDF5
>>>>> and netCDF) were compiled with parallel support, that means the compiler
>>>>> they were technically built with was "mpif90" (and/or "mpif77"). So,
>>>>> when we
>>>>> run f2py, we have to run:
>>>>>
>>>>> $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>>>>>
>>>>> so that it can pick up the MPI libraries when it compiles.
>>>>>
>>>>> The compilers I use are, mainly, Intel and PGI, and since I maintain
>>>>> some
>>>>> GPU code, PGI is often my focus. That said, I'm having an issue trying
>>>>> to
>>>>> use PGI and MVAPICH2 in combination that prevents me from exploring the
>>>>> GPU
>>>>> enhancements of MVAPICH2.
>>>>>
>>>>> I think the issue boils down to behavior upon returning the version for
>>>>> the
>>>>> compiler through the MPI wrapper. When I traced through f2py's automatic
>>>>> compiler detection, it seems to depend on a successful return code from
>>>>> a
>>>>> version check to say "Yes, the --f90exec is installed and valid." (I'm
>>>>> pretty sure of this, but it took a while to trace through.)
>>>>>
>>>>> To wit, when I just use PGI 14.1:
>>>>>
>>>>>> $ pgfortran -V
>>>>>>
>>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> $ echo $?
>>>>>> $ pgfortran --version
>>>>>>
>>>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> $ echo $?
>>>>>> 0
>>>>>
>>>>>
>>>>>
>>>>> So, if you use -V or --version, you get the usual spiel and a return
>>>>> code of
>>>>> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the version
>>>>> under mpif90, I get this behavior:
>>>>>
>>>>>> $ mpif90 -V
>>>>>>
>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> (1427) $ echo $?
>>>>>> 0
>>>>>> $ mpif90 --version
>>>>>>
>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> $ echo $?
>>>>>> 0
>>>>>
>>>>>
>>>>>
>>>>> Again, I get the same behavior as the bare compiler with return codes of
>>>>> 0.
>>>>>
>>>>> Now, PGI 14.1 and MVAPICH2 2.0b:
>>>>>
>>>>>> $ mpif90 -V
>>>>>>
>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>>> `main':
>>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>>> $ echo $?
>>>>>> 2
>>>>>> $ mpif90 --version
>>>>>>
>>>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>>>> The Portland Group - PGI Compilers and Tools
>>>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function
>>>>>> `main':
>>>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>>>> $ echo $?
>>>>>> 2
>>>>>
>>>>>
>>>>>
>>>>> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with
>>>>> f2py,
>>>>> when you run mpif90 --version/-V with MVAPICH2, it returns an error code
>>>>> of
>>>>> 2, not 0.
>>>>>
>>>>> Is there a reason for this? I'm especially surprised at the --version
>>>>> behavior. Are there compilers/situations that when you do "$FC
>>>>> --version"
>>>>> one should expect a "main" program to link?
>>>>>
>>>>> The one simple workaround I can think of is to change the mpif90 script
>>>>> itself:
>>>>>
>>>>>> --- mpif90_withchange 2014-03-13 09:25:24.496774411 -0400
>>>>>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>>>>>> 2014-01-31 13:32:39.033436000 -0500
>>>>>> @@ -151,7 +151,7 @@
>>>>>> case "$arg" in
>>>>>> #
>>>>>> ----------------------------------------------------------------
>>>>>> # Compiler options that affect whether we are linking or no
>>>>>> - -c|-S|-E|-M|-MM|-V|--version)
>>>>>> + -c|-S|-E|-M|-MM)
>>>>>> # The compiler links by default
>>>>>> linking=no
>>>>>> ;;
>>>>>
>>>>>
>>>>>
>>>>> but as that script (and mpif77) is fairly fundamental to MVAPICH2, well,
>>>>> I'm
>>>>> a bit hesitant to have my sysadmins patch it every time they install for
>>>>> fear of breaking...everything! (Of course, I might have them try it to
>>>>> see
>>>>> if it helps.)
>>>>>
>>>>> Would you have an objection (if I find it works) to adding these as
>>>>> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve the
>>>>> PGI
>>>>> ability to change compilers on the fly with -V<ver> since, in bash,
>>>>> "-V14.3"
>>>>> does not equal "-V".
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>>
>>>>> --
>>>>> Matt Thompson, PhD SSAI, Sr Software Test Engr
>>>>> NASA GSFC, Global Modeling and Assimilation Office
>>>>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>>>>> Phone: 301-614-6712 Fax: 301-614-6246
>>>>>
>>>>> _______________________________________________
>>>>> mvapich-discuss mailing list
>>>>> mvapich-discuss at cse.ohio-state.edu
>>>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Jonathan Perkins
>>>> http://www.cse.ohio-state.edu/~perkinjo
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Matt Thompson SSAI, Sr Software Test Engr
>>
>> NASA GSFC, Global Modeling and Assimilation Office
>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>> Phone: 301-614-6712 Fax: 301-614-6246
>>
>>
>
>
>
--
Matt Thompson SSAI, Sr Software Test Engr
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
More information about the mvapich-discuss
mailing list