[mvapich-discuss] Bug in MVAPICH2 SVN Trunk Configure Script

[Tom Crockett]

Pavan Balaji wrote:
> This was a bug in MPICH2 that was fixed in r1000 shortly after the 1.0.7 
> release. But, this error should only mean that the Fortran bindings are 
> not compiled with the internal optimizations; so there shouldn't be any 
> noticeable performance impact.

Pavan,

As it turns out, the impact was more severe than that.  When I was 
attempting a 32-bit build of MVAPICH2, gfortran apparently defaulted to 
a 64-bit compilation because it didn't get the "-m32" option that I had 
specified in F90FLAGS, and that caused one of the configuration tests to 
fail because it was trying to link a 64-bit object file against the 
32-bit OFED libraries.


>> We'll also take a look at this.  We may still need to reset these flags
>> but maybe it should still include various 'internal' flags that the user
>> has specified.
> 
> The CFLAGS will be included within mpicc; they won't show up externally. 

The configuration test that's blowing up isn't using mpicc, it's 
invoking gcc directly.  Here's the relevant output from "configure":

-----------------------------------------------------------------------
RUNNING CONFIGURE FOR MPI TESTS
checking for make... make
checking whether clock skew breaks make... no
checking whether make supports include... yes
checking whether make allows comments in actions... yes
checking for virtual path format... VPATH
checking whether make sets CFLAGS... no
checking for gcc... /usr/bin/gcc
checking for C compiler default output file name... configure: error: C 
compiler cannot create executables
See `config.log' for more details.
configure: error: /bin/sh './configure' failed for test/mpi
-----------------------------------------------------------------------

And here's an excerpt from test/mpi/config.log which shows the specific 
test that blows up.  It's essentially the same problem that gfortran ran 
into in src/binding/f90/configure:  gcc doesn't get the right set of 
compiler flags, so it defaults to 64-bit mode and fails when it tries to 
link against the 32-bit OFED libraries:

-----------------------------------------------------------------------
configure:2304: checking for C compiler default output file name
configure:2307: /usr/bin/gcc  -I/usr/local/OFED-1.3/include 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/include 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/include 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/common/datatype 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/common/datatype 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/channels/mrail/include 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/channels/mrail/include 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/channels/mrail/src/gen2 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/ch3/channels/mrail/src/gen2 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/common/locks 
-I/usr/local/src/mvapich2-trunk-2008-09-09/src/mpid/common/locks 
-L/usr/local/OFED-1.3/lib  conftest.c -lpthread  -lrdmacm -libverbs 
-libumad    -lrt    >&5
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../x86_64-suse-linux/bin/ld: 
skipping incompatible /usr/local/OFED-1.3/lib/librdmacm.so when 
searching for -lrdmacm
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../x86_64-suse-linux/bin/ld: 
skipping incompatible /usr/local/OFED-1.3/lib/librdmacm.a when searching 
for -lrdmacm
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../x86_64-suse-linux/bin/ld: 
cannot find -lrdmacm
collect2: ld returned 1 exit status
configure:2310: $? = 1
configure: failed program was:
| /* confdefs.h.  */
|
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:2349: error: C compiler cannot create executables
See `config.log' for more details.
-----------------------------------------------------------------------

 > What exact CFLAGS and MPICH2LIB_CFLAGS did you use? Note that these
 > are exclusive. CFLAGS sets flags that are used to compile
 > MPICH2/MVAPICH2 as well as show up in mpicc. MPICH2LIB_CFLAGS only
 > sets flags that are used to compile MPICH2/MVAPICH2.

Here are the configuration parameters (from test/mpi/config.log):

-----------------------------------------------------------------------
./configure --prefix=/local/scr/root/mvapich2-trunk-2008-09-09-ib 
--prefix=/local/scr/root/mvapich2-trunk-2008-09-09-ib --with-rdma=gen2 
--with-pm=mpirun --with-ib-include=/usr/local/OFED-1.3/include 
--with-ib-libpath=/usr/local/OFED-1.3/lib --enable-romio 
--with-file-system=ufs+nfs CC=/usr/bin/gcc CFLAGS=-march=k8 -m32 
CXXFLAGS=-march=k8 -m32 CXX=/usr/bin/g++ F77=/usr/bin/gfortran 
F90FLAGS=-march=k8 -m32 F90=/usr/bin/gfortran FFLAGS=-march=k8 -m32 
MPICH2LIB_CFLAGS=-O3 -march=k8 -msse -msse2 -msse3 -m32 
MPICH2LIB_CXXFLAGS=-O3 -march=k8 -msse -msse2 -msse3 -m32 
MPICH2LIB_F90FLAGS=-O3 -march=k8 -msse -msse2 -msse3 -m32 
MPICH2LIB_FFLAGS=-O3 -march=k8 -msse -msse2 -msse3 -m32 
--enable-header-caching --cache-file=/dev/null --srcdir=.
-----------------------------------------------------------------------

Here's the output from "mpicc -show":

-----------------------------------------------------------------------
/usr/bin/gcc -march=k8 -m32 -L/usr/local/OFED-1.3/lib 
-I/local/scr/root/mvapich2-trunk-2008-09-09-ib/include 
-L/local/scr/root/mvapich2-trunk-2008-09-09-ib/lib -lmpich -lpthread 
-lrdmacm -libverbs -libumad -lrt
-----------------------------------------------------------------------

Does this clarify things?

-Tom

-- 
Tom Crockett

College of William and Mary               email:  twcroc at wm.edu
IT/High Performance Computing Group       phone:  (757) 221-2762
Savage House                              fax:    (757) 221-2023
P.O. Box 8795
Williamsburg, VA  23187-8795