[mvapich-discuss] MVAPICH2 1.4rc1 with CR on RHEL4 AS

[Karthik Gopalakrishnan]

Hi Constantinos.

Please check if the attached patch fixes the CR build issue on RHEL4.

Regards,
Karthik

On Fri, Jun 19, 2009 at 10:22 AM, Karthik
Gopalakrishnan<gopalakk at cse.ohio-state.edu> wrote:
> Hi Constantinos.
>
> 2009/6/18 Constantinos Evangelinos <ce107 at mit.edu>:
>> Another problem that I'm facing with MVAPICH2 1.4rc1 on RHEL4 Intel compilers
>> 10.1. I'm trying to build it (without Limic2) with support for CR. I've
>> installed the latest BLCR rpms etc.
>>
>> This is my configure line:
>> ./configure --with-rdma=gen2 --enable-blcr --enable-romio --with-file-system=ufs+pvfs2
>> CPPFLAGS="-I/opt/pvfs-2.8.1/include" LDFLAGS=-L/opt/pvfs-2.8.1/lib
>> LIBS="-lpvfs2 -lcr -pthread" MPICH2LIB_CFLAGS="-xT -fPIC -D_GNU_SOURCE"
>> MPICH2LIB_CXXFLAGS="-xT -fPIC" MPICH2LIB_FFLAGS="-xT -fPIC"
>> MPICH2LIB_F90FLAGS="-xT -fPIC" CC=icc CXX=icpc F77=ifort
>> F90=ifort --prefix=/opt/mvapich2-1.4rc1/icc
>>
>> In successfully building MVAPICH2 1.2 on the very same platform with the same
>> compilers I discovered I had to (a) add a -lcr to LIBS and (b) add a -pthread
>> to LIBS (-lpthread will not do). I've also had to add -D_GNU_SOURCE as
>> already discussed here.
>>
>> However I hit a snag:
>>
>> make[8]: Entering directory
>> `/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2'
>> icc -DHAVE_CONFIG_H -I. -I. -I/root/build/mvapich2-1.4rc1/src/include -I../../../../../../include -DNDEBUG -O2 -xT -fPIC -D_GNU_SOURCE -D_GNU_SOURCE -I/opt/pvfs-2.8.1/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/include -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/datatype -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/include -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/ch3/channels/mrail/src/gen2 -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -I/root/build/mvapich2-1.4rc1/src/mpid/common/locks -c
>> cr.c
>> cr.c(277): error: struct "_pthread_rwlock_t" has no field "__data"
>>      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
>>                        ^
>>
>> cr.c(289): error: struct "_pthread_rwlock_t" has no field "__data"
>>      if (MPICR_cs_lock.__data.__writer == syscall(SYS_gettid))
>>                        ^
>>
>> cr.c(963): warning #167: argument of type "struct _IO_FILE *" is incompatible
>> with parameter of type "const char *"
>>          MPIU_Error_printf(stderr, "rdma_open_hca failed\n");
>>                            ^
>>
>> cr.c(1005): warning #167: argument of type "void *" is incompatible with
>> parameter of type "void *(*)(void *)"
>>                  (void*) async_thread,
>>                  ^
>>
>> cr.c(924): warning #589: transfer of control bypasses initialization of:
>>            variable "pg" (declared at line 938)
>>            variable "pg_rank" (declared at line 939)
>>            variable "pg_size" (declared at line 940)
>>            variable "ud_qpn_all" (declared at line 942)
>>            variable "lid_all" (declared at line 943)
>>            variable "vc" (declared at line 950)
>>            variable "i" (declared at line 951)
>>        MPIU_ERR_SETFATALANDJUMP1(
>>        ^
>>
>> cr.c(1268): warning #144: a value of type "MPIDI_CH3_PktGeneric_t *" cannot be
>> used to initialize an entity of type "MPIDI_CH3_Pkt_t *"
>>                  MPIDI_CH3_Pkt_t *upkt = &(req->dev.pending_pkt);
>>                                          ^
>>
>> compilation aborted for cr.c (code 2)
>>
>> at which point I cannot proceed any further. Any ideas on what can be done?
>>
> Thank You for your report. We are working on a fix for the compilation
> error. I will provide you with a patch ASAP.
>
>> Also in the past there was a performance penalty for shared memory operations
>> when using CR - am I to understand that is not the case anymore?
>>
> Your understanding is correct. CR of the intra-node shared memory
> channel is supported as of MVAPICH2 1.2. So there is no longer a
> performance penalty.
>
> Thanks & Regards,
> Karthik
>
>> Thanks in advance,
>>
>> Constantinos
>> --
>> Dr. Constantinos Evangelinos                    Room 54-1518, EAPS/MIT
>> Earth, Atmospheric and Planetary Sciences       77 Massachusetts Avenue
>> Massachusetts Institute of Technology           Cambridge, MA 02139
>> +1-617-324-3386/+1-617-253-4464 (fax)           USA
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cr_rhel4.patch
Type: application/octet-stream
Size: 3495 bytes
Desc: not available
Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20090621/b9504a47/cr_rhel4.obj