[mvapich-discuss] MVAPICH2, Open MPI, PGI, f2py and behavior of -V/--version
Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
matthew.thompson at nasa.gov
Thu Mar 13 15:26:20 EDT 2014
Jonathan,
It turns out this doesn't actually work. The issue seem to be that aside
from the non-zero error return code, f2py doesn't like the extra
information about mvapich2 that the "echo" is returning about the MPI
version:
> $ mpif90 -V
> mpif90 for MVAPICH2 version 2.0b
>
> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
> The Portland Group - PGI Compilers and Tools
> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
I did something like this:
> (1057) $ diff -u mpif90.old mpif90.new.sepV [discover29:~ 3:20pm]
> --- mpif90.old 2014-03-13 15:19:59.743561044 -0400
> +++ mpif90.new.sepV 2014-03-13 15:20:43.483557824 -0400
> @@ -201,6 +201,12 @@
> linking=no
> fi
> ;;
> + -V|--version)
> + # For -V and --version, do not echo the MVAPICH2 version
> + if [ "$#" -eq "1" ] ; then
> + linking=no
> + fi
> + ;;
> -profile=*)
> # Pass the name of a profiling configuration. As
> # a special case, lib<name>.so or lib<name>.la may be used
and I confirmed that this *does* work. Apparently like my scripts, f2py
is expecting "mpif90 -V" to return identically what "pgf90 -V" does to
the character.
Matt
On 03/13/2014 10:25 AM, Jonathan Perkins wrote:
> Hi, I haven't had a chance to verify that there aren't any side
> effects with the -V option yet but this is the patch that resolves the
> issue for us. You can try this out and let us know if it works for
> you.
>
> --- mpif90.old 2014-03-13 10:21:53.085899344 -0400
> +++ mpif90.new 2014-03-13 10:21:31.928917828 -0400
> @@ -190,7 +190,7 @@
> Show=echo
> addarg=no
> ;;
> - -v)
> + -v|-V|--version)
> # Pass this argument to the compiler as well.
> echo "mpif90 for MVAPICH2 version $MVAPICH2_VERSION"
> # if there is only 1 argument, it must be -v.
>
> On Thu, Mar 13, 2014 at 9:52 AM, Jonathan Perkins
> <perkinjo at cse.ohio-state.edu> wrote:
>> Thanks for bringing this to our attention. We haven't noticed a
>> problem here ourselves but your explanation leads me to believe that
>> you can try the work around.
>>
>> We'll try to reproduce this problem and, if successful, see how best
>> to resolve it.
>>
>> On Thu, Mar 13, 2014 at 9:41 AM, Thompson, Matt (GSFC-610.1)[SCIENCE
>> SYSTEMS AND APPLICATIONS INC] <matthew.thompson at nasa.gov> wrote:
>>> MVAPICH List,
>>>
>>> This is a bit of an odd issue, but I'm hoping someone here can help. In the
>>> code I work with, we use f2py. Due to the fact that some sub-libraries (HDF5
>>> and netCDF) were compiled with parallel support, that means the compiler
>>> they were technically built with was "mpif90" (and/or "mpif77"). So, when we
>>> run f2py, we have to run:
>>>
>>> $ f2py --f77exec=mpif77 --f90exec=mpif90 -c ...
>>>
>>> so that it can pick up the MPI libraries when it compiles.
>>>
>>> The compilers I use are, mainly, Intel and PGI, and since I maintain some
>>> GPU code, PGI is often my focus. That said, I'm having an issue trying to
>>> use PGI and MVAPICH2 in combination that prevents me from exploring the GPU
>>> enhancements of MVAPICH2.
>>>
>>> I think the issue boils down to behavior upon returning the version for the
>>> compiler through the MPI wrapper. When I traced through f2py's automatic
>>> compiler detection, it seems to depend on a successful return code from a
>>> version check to say "Yes, the --f90exec is installed and valid." (I'm
>>> pretty sure of this, but it took a while to trace through.)
>>>
>>> To wit, when I just use PGI 14.1:
>>>
>>>> $ pgfortran -V
>>>>
>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> $ echo $?
>>>> $ pgfortran --version
>>>>
>>>> pgfortran 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> $ echo $?
>>>> 0
>>>
>>>
>>> So, if you use -V or --version, you get the usual spiel and a return code of
>>> 0. Now, when I use PGI 14.1 and Open MPI 1.7.3 and look for the version
>>> under mpif90, I get this behavior:
>>>
>>>> $ mpif90 -V
>>>>
>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> (1427) $ echo $?
>>>> 0
>>>> $ mpif90 --version
>>>>
>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp nehalem
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> $ echo $?
>>>> 0
>>>
>>>
>>> Again, I get the same behavior as the bare compiler with return codes of 0.
>>>
>>> Now, PGI 14.1 and MVAPICH2 2.0b:
>>>
>>>> $ mpif90 -V
>>>>
>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>> $ echo $?
>>>> 2
>>>> $ mpif90 --version
>>>>
>>>> pgf90 14.1-0 64-bit target on x86-64 Linux -tp sandybridge
>>>> The Portland Group - PGI Compilers and Tools
>>>> Copyright (c) 2014, NVIDIA CORPORATION. All rights reserved.
>>>> /usr/local/sles11/pgi/linux86-64/14.1/libso/f90main.o: In function `main':
>>>> f90main.c:(.text+0x40): undefined reference to `MAIN_'
>>>> $ echo $?
>>>> 2
>>>
>>>
>>> And herein lies the reason, I think, I can't use MVAPICH2 and PGI with f2py,
>>> when you run mpif90 --version/-V with MVAPICH2, it returns an error code of
>>> 2, not 0.
>>>
>>> Is there a reason for this? I'm especially surprised at the --version
>>> behavior. Are there compilers/situations that when you do "$FC --version"
>>> one should expect a "main" program to link?
>>>
>>> The one simple workaround I can think of is to change the mpif90 script
>>> itself:
>>>
>>>> --- mpif90_withchange 2014-03-13 09:25:24.496774411 -0400
>>>> +++ /usr/local/other/SLES11.1/mvapich2/2.0b/pgi-14.1.0/bin/mpif90
>>>> 2014-01-31 13:32:39.033436000 -0500
>>>> @@ -151,7 +151,7 @@
>>>> case "$arg" in
>>>> # ----------------------------------------------------------------
>>>> # Compiler options that affect whether we are linking or no
>>>> - -c|-S|-E|-M|-MM|-V|--version)
>>>> + -c|-S|-E|-M|-MM)
>>>> # The compiler links by default
>>>> linking=no
>>>> ;;
>>>
>>>
>>> but as that script (and mpif77) is fairly fundamental to MVAPICH2, well, I'm
>>> a bit hesitant to have my sysadmins patch it every time they install for
>>> fear of breaking...everything! (Of course, I might have them try it to see
>>> if it helps.)
>>>
>>> Would you have an objection (if I find it works) to adding these as
>>> "linking=no" flags for mpif90/f77/cc/cxx? I think this does preserve the PGI
>>> ability to change compilers on the fly with -V<ver> since, in bash, "-V14.3"
>>> does not equal "-V".
>>>
>>> Thanks,
>>> Matt
>>>
>>> --
>>> Matt Thompson, PhD SSAI, Sr Software Test Engr
>>> NASA GSFC, Global Modeling and Assimilation Office
>>> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>>> Phone: 301-614-6712 Fax: 301-614-6246
>>>
>>> _______________________________________________
>>> mvapich-discuss mailing list
>>> mvapich-discuss at cse.ohio-state.edu
>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>
>>>
>>
>>
>>
>> --
>> Jonathan Perkins
>> http://www.cse.ohio-state.edu/~perkinjo
>
>
>
--
Matt Thompson SSAI, Sr Software Test Engr
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
More information about the mvapich-discuss
mailing list