[mvapich-discuss] Patch to mvapich spec file to improve performance when using the QLogic PSM interface

[Mike Heinz]

Hey,

This patch to the spec file changes the compile options when building against the QLogic PSM library instead of verbs, I hope you find it useful:

Index: mvapich.spec
===================================================================
RCS file: /cvs/vendor/OFA/Ofed/mvapich/mvapich.spec,v
retrieving revision 1.1.1.9
retrieving revision 1.34
diff -w -u -d -r1.1.1.9 -r1.34
--- mvapich.spec        29 Apr 2010 21:11:46 -0000      1.1.1.9
+++ mvapich.spec        15 Nov 2011 17:40:17 -0000      1.34
@@ -245,7 +245,11 @@
 IB_INCLUDE=%{openib_prefix}/include
 IB_LIB=%{openib_prefix}/lib${BIT}

+if [ "%{mpi_device}" == "ch_psm" ] ; then
+export CFLAGS="$CFLAGS $OPTIMIZATION_FLAG -g -D_GNU_SOURCE -DCH_PSM -D_AFFINITY_ -D_SMP_ -DXRC $ARCH_NAME -I$IB_INCLUDE"
+else
 export CFLAGS="$CFLAGS $OPTIMIZATION_FLAG -g -D_GNU_SOURCE -DCH_GEN2 -D_AFFINITY_ -DMEMORY_SCALE -D_SMP_ -D_SMP_RNDV_ -DVIADEV_RPUT_SUPPORT -DEARLY_SEND_COMPLETION -DXRC $ARCH_NAME -I$IB_INCLUDE"
+fi

 # Mvapich configure is not functional, so I must to add "manual check" for LINK LAYER defenition
 if ( grep IBV_LINK_LAYER_ETHERNET %{openib_prefix}/include/infiniband/verbs.h &> /dev/null ) ; then
@@ -257,11 +261,19 @@
 export MPE_CFLAGS
 export LDFLAGS
 export CXXFLAGS="$CXXFLAGS"
+if [ "%{mpi_device}" == "ch_psm" ] ; then
+export FFLAGS="$FFLAGS $EXTRA_CFLAG"
+else
 export FFLAGS="$FFLAGS -L$IB_LIB $EXTRA_CFLAG"
+fi
 export F90FLAGS="$F90FLAGS $EXTRA_CFLAG"
 export CONFIG_FLAGS
 export MPIRUN_CFLAGS="$MPIRUN_CFLAGS -DPARAM_GLOBAL=\\\"%{_prefix}/etc/$conffile\\\" -DLD_LIBRARY_PATH_MPI=\\\"%{_prefix}/lib/shared\\\" -DMPI_PREFIX=\\\"%{_prefix}/\\\" $DEF_BUILDID"
+if [ "%{mpi_device}" == "ch_psm" ] ; then
+./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=$RPM_BUILD_ROOT%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -lpsm_infinipath -libverbs -libumad -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS
+else
 ./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=$RPM_BUILD_ROOT%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS
+fi
 %{__make}

 #############################################################################

This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.