[mvapich-discuss] Problem Building MVAPICH-1.0-beta with PGI Compilers

[Peter Cebull]

Jonathan L. Perkins wrote:
> Jonathan Perkins wrote:
>> Peter Cebull wrote:
>>> I'm trying to build MVAPICH with PGI compilers (version 7.1-3) and 
>>> am running into an error. Our system is running SLES 10. Log files 
>>> are attached. Has anyone seen this before?
>>>
>>> Thanks,
>>> Peter
>>>
>>>
>>> ------------------------------------------------------------------------ 
>>>
>>>
>>> _______________________________________________
>>> mvapich-discuss mailing list
>>> mvapich-discuss at cse.ohio-state.edu
>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>> Peter:
>> We're sorry that you're having trouble with the PGI compiler.  We'll 
>> take a look into this issue and get back to you as soon as we can.
>>
>>
>
> Peter:
> Can you send a bit more information related to this error.  Can you 
> tell us which variables you set in order to use the pgi compiler.  
> Also, are you using the make.mvapich.gen2 script for your build process?
>
> I haven't been able to reproduce this error using SuSE 10 and both PGI 
> 7.0-7 and PGI 7.1-3.
>
I've attached the make.mvapich.gen2 file I was using. I think all I 
changed were these variables

IBHOME=${IBHOME:-/usr}
IBHOME_LIB=${IBHOME_LIB:-/usr/lib64}
PREFIX=${PREFIX:-/usr/local/mvapich/mvapich-1.0-beta/pgi-opt}
export CC=${CC:-pgcc}
export CXX=${CXX:-pgCC}
export F77=${F77:-pgf77}
export F90=${F90:-pgf90}

and added some options to configure:

./configure --enable-shared-lib --with-device=ch_gen2 --with-arch=LINUX 
-prefix=${PREFIX} \
        --enable-f77 --enable-f90 --enable-f90modules $ROMIO 
-lib="$LIBS" 2>&1 |tee config-mine.log

Then I just executed the script. Let me know if you need any more info.

Thanks,
Peter

-- 
Peter Cebull
HPC User Consultant
Idaho National Laboratory
P.O. Box 1625, MS3605
Idaho Falls, ID 83415
Phone: 208-526-1909
Email: Peter.Cebull at inl.gov

-------------- next part --------------
#!/bin/bash

# Most variables here can be overridden by exporting them in the environment
# before running this script.  Default values have been provided if the
# environment variable is not already set.

source ./make.mvapich.def

# The target architecture.  If not exported outside of this script,
# it will be found automatically or prompted for if necessary.
# Supported: "_IA32_", "_IA64_", "_EM64T_", "_X86_64_"
#
if [ -z "$ARCH" ]; then
    arch
fi

# This is the compatibility mode. If not exported outside this
# script, the user is prompted to select either mode. In
# autodetection mode, different types of IB hardware can
# be detected and optimized for. In compatibility mode a
# common set of parameters is chosen by MVAPICH.
# Supported: "AUTO_DETECT" and "COMPAT_MODE"
#
if [ -z "$COMPAT" ]; then
    prompt_compat_mode
fi

# Mandatory variables.  All are checked except CXX and F90.
IBHOME=${IBHOME:-/usr}
IBHOME_LIB=${IBHOME_LIB:-/usr/lib64}
PREFIX=${PREFIX:-/usr/local/mvapich/mvapich-1.0-beta/pgi-opt}
export CC=${CC:-pgcc}
export CXX=${CXX:-pgCC}
export F77=${F77:-pgf77}
export F90=${F90:-pgf90}

if [ $ARCH = "SOLARIS" ]; then
    die_setup "MVAPICH GEN2 is not supported on Solaris."
elif [ $ARCH = "MAC_OSX" ]; then
    die_setup "MVAPICH GEN2 is not supported on MacOS."
fi

#
# Compiler specific flags. If you are using
# ICC on IA64 platform, please set COMPILER_FLAG
# to "icc"
#

COMPILER_FLAG=${COMPILER_FLAG:-}

if [ "$COMPILER_FLAG" == "icc" ]; then
       COMPILER_FLAG="-D_ICC_"
else
       COMPILER_FLAG=""
fi

# Check mandatory variable settings.
if [ -z "$IBHOME" ] || [ -z "$PREFIX" ] || [ -z "$CC" ] || [ -z "$F77" ]; then
    die_setup "Please set mandatory variables in this script."
elif [ ! -d $IBHOME ]; then
    die_setup "IBHOME directory $IBHOME does not exist."
fi

# Optional variables.
#

# Whether to enable ROMIO support.  This is necessary if building the
# F90 modules.
if [ -n "$F90" ]; then
    ROMIO="--with-romio"
else
    ROMIO=${ROMIO:---without-romio}
fi

# PTMALLOC support for MVAPICH2 memory hooks.  Enabling this will allow
# MVAPICH2 to release memory to the Operating System (when registration
# cache is enabled).  Enabled by default.  Disable with "no".
PTMALLOC=${PTMALLOC:-}

if [ "$PTMALLOC" = "no" ]; then
	PTMALLOC="-DDISABLE_PTMALLOC"
else
	PTMALLOC=""
fi


# Whether to use an optimized queue pair exchange scheme.  This is not
# checked for a setting in in the script.  It must be set here explicitly.
# Supported: "-DUSE_MPD_RING", "-DUSE_MPD_BASIC" and "" (to disable)
HAVE_MPD_RING=${HAVE_MPD_RING:-}

# Set this to override automatic optimization setting (-03).
OPT_FLAG=${OPT_FLAG:--O3}

export LIBS=${LIBS:--L${IBHOME_LIB} -Wl,-rpath=${IBHOME_LIB} -libverbs -libumad -lpthread}
export FFLAGS=${FFLAGS:--L${IBHOME_LIB}}
export CFLAGS=${CFLAGS:--D${ARCH} -D${COMPAT} ${PTMALLOC} -DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE ${COMPILER_FLAG} ${HAVE_MPD_RING} -I${IBHOME}/include $OPT_FLAG}

export MPIRUN_CFLAGS="${MPIRUN_CFLAGS} -DLD_LIBRARY_PATH_MPI=\\\"${PREFIX}/lib/shared\\\" -DPARAM_GLOBAL=\\\"${PREFIX}/etc/mvapich.conf\\\""

# Prelogue
make distclean &>/dev/null
set -o pipefail

# Configure MVAPICH

echo "Configuring MVAPICH..."

./configure --enable-shared-lib --with-device=ch_gen2 --with-arch=LINUX -prefix=${PREFIX} \
	--enable-f77 --enable-f90 --enable-f90modules $ROMIO -lib="$LIBS" 2>&1 |tee config-mine.log
ret=$?
test $ret = 0 ||  die "configuration."

# Build MVAPICH
echo "Building MVAPICH..."
make 2>&1 |tee make-mine.log 
ret=$?
test $ret = 0 ||  die "building MVAPICH."

# Install MVAPICH
echo "MVAPICH installation..."
rm -f install-mine.log 
make install 2>&1 |tee install-mine.log
ret=$?
test $ret = 0 ||  die "installing MVAPICH."