[mvapich-discuss] MVAPICH2 1.4rc1 with CR on RHEL4 AS

[Karthik Gopalakrishnan]

Hi Constantinos.

2009/6/18 Constantinos Evangelinos <ce107 at mit.edu>:
> Another problem that I'm facing with MVAPICH2 1.4rc1 on RHEL4 Intel compilers
> 10.1. I'm trying to build it (without Limic2) with support for CR. I've
> installed the latest BLCR rpms etc.
>
> This is my configure line:
> ./configure --with-rdma=gen2 --enable-blcr --enable-romio --with-file-system=ufs+pvfs2
> CPPFLAGS="-I/opt/pvfs-2.8.1/include" LDFLAGS=-L/opt/pvfs-2.8.1/lib
> LIBS="-lpvfs2 -lcr -pthread" MPICH2LIB_CFLAGS="-xT -fPIC -D_GNU_SOURCE"
> MPICH2LIB_CXXFLAGS="-xT -fPIC" MPICH2LIB_FFLAGS="-xT -fPIC"
> MPICH2LIB_F90FLAGS="-xT -fPIC" CC=icc CXX=icpc F77=ifort
> F90=ifort --prefix=/opt/mvapich2-1.4rc1/icc
>
> In successfully building MVAPICH2 1.2 on the very same platform with the same
> compilers I discovered I had to (a) add a -lcr to LIBS and (b) add a -pthread
> to LIBS (-lpthread will not do). I've also had to add -D_GNU_SOURCE as
> already discussed here.
>
> However I hit a snag:
>
> make[8]: Entering directory
> `/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2'
> icc -DHAVE_CONFIG_H -I. -I. -I/root/build/mvapich2-1.4rc1/src/include -I../../../../../../include -DNDEBUG -O2 -xT -fPIC -D_GNU_SOURCE -D_GNU_SOURCE -I/opt/pvfs-2.8.1/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -c
> cr.c
> cr.c(277): error: struct "_pthread_rwlock_t" has no field "__data"
>      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
>                        ^
>
> cr.c(289): error: struct "_pthread_rwlock_t" has no field "__data"
>      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
>                        ^
>
> cr.c(963): warning #167: argument of type "struct _IO_FILE *" is incompatible
> with parameter of type "const char *"
>          MPIU_Error_printf(stderr, "rdma_open_hca failed\n");
>                            ^
>
> cr.c(1005): warning #167: argument of type "void *" is incompatible with
> parameter of type "void *(*)(void *)"
>                  (void*) async_thread,
>                  ^
>
> cr.c(924): warning #589: transfer of control bypasses initialization of:
>            variable "pg" (declared at line 938)
>            variable "pg_rank" (declared at line 939)
>            variable "pg_size" (declared at line 940)
>            variable "ud_qpn_all" (declared at line 942)
>            variable "lid_all" (declared at line 943)
>            variable "vc" (declared at line 950)
>            variable "i" (declared at line 951)
>        MPIU_ERR_SETFATALANDJUMP1(
>        ^
>
> cr.c(1268): warning #144: a value of type "MPIDI_CH3_PktGeneric_t *" cannot be
> used to initialize an entity of type "MPIDI_CH3_Pkt_t *"
>                  MPIDI_CH3_Pkt_t *upkt = &(req->dev.pending_pkt);
>                                          ^
>
> compilation aborted for cr.c (code 2)
>
> at which point I cannot proceed any further. Any ideas on what can be done?
>
Thank You for your report. We are working on a fix for the compilation
error. I will provide you with a patch ASAP.

> Also in the past there was a performance penalty for shared memory operations
> when using CR - am I to understand that is not the case anymore?
>
Your understanding is correct. CR of the intra-node shared memory
channel is supported as of MVAPICH2 1.2. So there is no longer a
performance penalty.

Thanks & Regards,
Karthik

> Thanks in advance,
>
> Constantinos
> --
> Dr. Constantinos Evangelinos                    Room 54-1518, EAPS/MIT
> Earth, Atmospheric and Planetary Sciences       77 Massachusetts Avenue
> Massachusetts Institute of Technology           Cambridge, MA 02139
> +1-617-324-3386/+1-617-253-4464 (fax)           USA
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>