[mvapich-discuss] MVAPICH2 1.4rc1 with CR on RHEL4 AS

[Constantinos Evangelinos]

Another problem that I'm facing with MVAPICH2 1.4rc1 on RHEL4 Intel compilers 
10.1. I'm trying to build it (without Limic2) with support for CR. I've 
installed the latest BLCR rpms etc.

This is my configure line:
./configure --with-rdma=gen2 --enable-blcr --enable-romio --with-file-system=ufs+pvfs2 
CPPFLAGS="-I/opt/pvfs-2.8.1/include" LDFLAGS=-L/opt/pvfs-2.8.1/lib 
LIBS="-lpvfs2 -lcr -pthread" MPICH2LIB_CFLAGS="-xT -fPIC -D_GNU_SOURCE" 
MPICH2LIB_CXXFLAGS="-xT -fPIC" MPICH2LIB_FFLAGS="-xT -fPIC" 
MPICH2LIB_F90FLAGS="-xT -fPIC" CC=icc CXX=icpc F77=ifort 
F90=ifort --prefix=/opt/mvapich2-1.4rc1/icc

In successfully building MVAPICH2 1.2 on the very same platform with the same 
compilers I discovered I had to (a) add a -lcr to LIBS and (b) add a -pthread 
to LIBS (-lpthread will not do). I've also had to add -D_GNU_SOURCE as 
already discussed here. 

However I hit a snag:

make[8]: Entering directory 
`/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2'
icc -DHAVE_CONFIG_H -I. -I. -I/root/build/mvapich2-1.4rc1/src/include -I../../../../../../include -DNDEBUG -O2 -xT -fPIC -D_GNU_SOURCE -D_GNU_SOURCE -I/opt/pvfs-2.8.1/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -c 
cr.c
cr.c(277): error: struct "_pthread_rwlock_t" has no field "__data"
      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
                        ^

cr.c(289): error: struct "_pthread_rwlock_t" has no field "__data"
      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
                        ^

cr.c(963): warning #167: argument of type "struct _IO_FILE *" is incompatible 
with parameter of type "const char *"
          MPIU_Error_printf(stderr, "rdma_open_hca failed\n");
                            ^

cr.c(1005): warning #167: argument of type "void *" is incompatible with 
parameter of type "void *(*)(void *)"
                  (void*) async_thread,
                  ^

cr.c(924): warning #589: transfer of control bypasses initialization of:
            variable "pg" (declared at line 938)
            variable "pg_rank" (declared at line 939)
            variable "pg_size" (declared at line 940)
            variable "ud_qpn_all" (declared at line 942)
            variable "lid_all" (declared at line 943)
            variable "vc" (declared at line 950)
            variable "i" (declared at line 951)
        MPIU_ERR_SETFATALANDJUMP1(
        ^

cr.c(1268): warning #144: a value of type "MPIDI_CH3_PktGeneric_t *" cannot be 
used to initialize an entity of type "MPIDI_CH3_Pkt_t *"
                  MPIDI_CH3_Pkt_t *upkt = &(req->dev.pending_pkt);
                                          ^

compilation aborted for cr.c (code 2)

at which point I cannot proceed any further. Any ideas on what can be done?

Also in the past there was a performance penalty for shared memory operations 
when using CR - am I to understand that is not the case anymore?

Thanks in advance,

Constantinos
-- 
Dr. Constantinos Evangelinos                    Room 54-1518, EAPS/MIT
Earth, Atmospheric and Planetary Sciences       77 Massachusetts Avenue
Massachusetts Institute of Technology           Cambridge, MA 02139
+1-617-324-3386/+1-617-253-4464 (fax)           USA