[mvapich-discuss] Error trying to build mvapich2-1.0 with PGI 7.0-7

[Peter Thompson]

Seeing this thread, and the mention of TotalView, caused me to take a
closer look at the earlier messages in the thread.  If you notice the
compile line when building for TotalView, you will see an issue that we
first noticed with MPICH2

gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -m64
-D_GNU_SOURCE -fPIC -fPIC -I/usr/include/python2.3 -c mtv.c -o
/home/7/rowland/work/mvapich2-trunk/src/pm/mpd/mtv.o

The combination of -O2 and -g will often cause issues with TotalView
support, as some symbols get optimized away.  We found that you could
add some arguments to the mtv_setup.py script in src/pm/mpd/ so that
this gets corrected.  This patch is currently in the 1.0.5p4 and 1.0.6
versions of MPICH2, but don't seem to have been picked up here for
mvapich2-1.0.  I'm attaching the patch we added to MPICH2 to get that to
work a bit more seamlessly with TotalView.

Your interest, or mileage, may vary.

Regards,
Peter Thompson


Shaun Rowland wrote:
> David Gunter wrote:
>> Thanks for pointing out the Python distutils connection to me.  The
>> hint about the change from PGI 6 to PGI 7 was a clue as well.  I was
>> able to add the flag "-noswitcherror" to my compile scripts which
>> tells PGI 7 to only print warnings if unsupported flags are encountered.
>>
>> So for now, I have been able to compile the 1.0 release with PGI.
>>
>> Thanks for the help.
> 
> I was unaware of the -noswitcherror flag for PGI 7. Thanks! I was just
> getting ready to see if such a flag existed.
-------------- next part --------------
diff -rc src.orig/Makefile.in src/Makefile.in
*** src.orig/Makefile.in	2006-08-11 09:54:48.000000000 -0400
--- src/Makefile.in	2006-09-21 11:09:55.331187000 -0400
***************
*** 114,125 ****
  	if [ ! -d ${DESTDIR}${htmldir} ] ; then $(MKDIR_P) ${DESTDIR}${htmldir} ; fi
  	if [ ! -d ${DESTDIR}${mandir} ] ; then $(MKDIR_P) ${DESTDIR}${mandir} ; fi
  	if [ ! -d ${DESTDIR}${includedir} ] ; then $(MKDIR_P) ${DESTDIR}${includedir} ; fi
! 	${MAKE} install-local
! 	$(INSTALL_DATA) src/include/mpi.h ${DESTDIR}${includedir}/mpi.h
  	if [ ! -d ${DESTDIR}${exec_prefix} ] ; then $(MKDIR_P) ${DESTDIR}${exec_prefix} ; fi
  	if [ ! -d ${DESTDIR}${libdir} ] ; then $(MKDIR_P) ${DESTDIR}${libdir} ; fi
  	$(INSTALL_DATA) lib/lib${MPILIBNAME}.a ${DESTDIR}${libdir}/lib${MPILIBNAME}.a
- 	if [ ! -d ${DESTDIR}${docdir} ] ; then $(MKDIR_P) ${DESTDIR}${docdir} ; fi
  	@if test -d $(srcdir)/www && cd $(srcdir)/www ; then \
  	 for name in * ; do \
  	  if [ -f $$name ] ; then \
--- 114,125 ----
  	if [ ! -d ${DESTDIR}${htmldir} ] ; then $(MKDIR_P) ${DESTDIR}${htmldir} ; fi
  	if [ ! -d ${DESTDIR}${mandir} ] ; then $(MKDIR_P) ${DESTDIR}${mandir} ; fi
  	if [ ! -d ${DESTDIR}${includedir} ] ; then $(MKDIR_P) ${DESTDIR}${includedir} ; fi
! 	if [ ! -d ${DESTDIR}${docdir} ] ; then $(MKDIR_P) ${DESTDIR}${docdir} ; fi
  	if [ ! -d ${DESTDIR}${exec_prefix} ] ; then $(MKDIR_P) ${DESTDIR}${exec_prefix} ; fi
  	if [ ! -d ${DESTDIR}${libdir} ] ; then $(MKDIR_P) ${DESTDIR}${libdir} ; fi
+ 	${MAKE} install-local
+ 	$(INSTALL_DATA) src/include/mpi.h ${DESTDIR}${includedir}/mpi.h
  	$(INSTALL_DATA) lib/lib${MPILIBNAME}.a ${DESTDIR}${libdir}/lib${MPILIBNAME}.a
  	@if test -d $(srcdir)/www && cd $(srcdir)/www ; then \
  	 for name in * ; do \
  	  if [ -f $$name ] ; then \
diff -rc src.orig/src/pm/mpd/mpiexec.py src/src/pm/mpd/mpiexec.py
*** src.orig/src/pm/mpd/mpiexec.py	2006-04-14 22:25:11.000000000 -0400
--- src/src/pm/mpd/mpiexec.py	2006-09-21 10:17:42.221863000 -0400
***************
*** 451,463 ****
                  outECs += 'jobid=%s\n' % (jobid.strip())
          # print 'mpiexec: job %s started' % (jobid)
          if parmdb['MPIEXEC_TOTALVIEW']:
!             if not mpd_which('totalview'):
                  print 'cannot find "totalview" in your $PATH:'
                  print '    ', os.environ['PATH']
                  sys.exit(-1)
              import mtv
              tv_cmd = 'dattach python ' + `os.getpid()` + '; dgo; dassign MPIR_being_debugged 1'
!             os.system('totalview -e "%s" &' % (tv_cmd) )
              mtv.wait_for_debugger()
              mtv.allocate_proctable(parmdb['nprocs'])
              # extract procinfo (rank,hostname,exec,pid) tuples from msg
--- 451,466 ----
                  outECs += 'jobid=%s\n' % (jobid.strip())
          # print 'mpiexec: job %s started' % (jobid)
          if parmdb['MPIEXEC_TOTALVIEW']:
!             tvname = 'totalview'
!             if os.environ.has_key('TOTALVIEW'):
!                 tvname = os.environ['TOTALVIEW']
!             if not mpd_which(((tvname.strip()).split()[0])):
                  print 'cannot find "totalview" in your $PATH:'
                  print '    ', os.environ['PATH']
                  sys.exit(-1)
              import mtv
              tv_cmd = 'dattach python ' + `os.getpid()` + '; dgo; dassign MPIR_being_debugged 1'
!             os.system(tvname + ' -e "%s" &' % (tv_cmd) )
              mtv.wait_for_debugger()
              mtv.allocate_proctable(parmdb['nprocs'])
              # extract procinfo (rank,hostname,exec,pid) tuples from msg
diff -rc src.orig/src/pm/mpd/mtv_setup.py src/src/pm/mpd/mtv_setup.py
*** src.orig/src/pm/mpd/mtv_setup.py	2005-01-29 11:37:14.000000000 -0500
--- src/src/pm/mpd/mtv_setup.py	2006-09-20 14:59:57.037187000 -0400
***************
*** 6,11 ****
  
  from distutils.core import setup, Extension
  
! mtv = Extension("mtv",["mtv.c"])
  
  setup(name="mtv", version="1.0", ext_modules=[mtv])
--- 6,14 ----
  
  from distutils.core import setup, Extension
  
! mtv = Extension("mtv",
!                 sources = ["mtv.c"],
!                 extra_compile_args = ["-O0", "-g"],
!                 extra_link_args = ["-O0", "-g"])
  
  setup(name="mtv", version="1.0", ext_modules=[mtv])