[mvapich-discuss] SDR or DDR
Steve Jones
stevejones at stanford.edu
Wed Feb 7 19:41:25 EST 2007
Hi.
I didn't immediately notice the Topspin roll was being used here.
We have PATH set in the Cisco Topspin Roll to include the Topspin MPI
wrapper scripts, but it looks like I'll have to write better documentation
on using it as Rocks provides MPI wrapper scripts as well.
You can also post to the Rocks list with questions about the Topspin Roll.
Be sure to include Topspin in the subject line so it stands out to me.
Steve
Quoting "Scott Weitzenkamp (sweitzen)" <sweitzen at cisco.com>:
> Yes, .p scripts are for PGI compiler. We support Intel C/C++/F77/F90,
> and PGI F77/F90 (no PGI C/C++ support) with the binary RPMs.
>
> Scott
>
>
> ________________________________
>
> From: mvapich-discuss-bounces at cse.ohio-state.edu
> [mailto:mvapich-discuss-bounces at cse.ohio-state.edu] On Behalf Of Mike
> Hanby
> Sent: Wednesday, February 07, 2007 2:02 PM
> To: mvapich-discuss at cse.ohio-state.edu
> Subject: RE: [mvapich-discuss] SDR or DDR
>
>
>
> Thanks Scott, I wasn't aware of that. It looks like there's also
> mpicc.p (pgi?).
>
>
>
> I recompiled Amber9 using /usr/local/topspin/mpi/mpich/mpicc.i
> (and CC.i and f77.i), and now it appears to running using mpirun_ssh (at
> least I'm not getting a bunch of segfaults).
>
>
>
> Thanks, again
>
>
>
> Mike
>
>
>
>
> ________________________________
>
>
> From: Scott Weitzenkamp (sweitzen) [mailto:sweitzen at cisco.com]
> Sent: Wednesday, February 07, 2007 12:48
> To: Mike Hanby
> Subject: RE: [mvapich-discuss] SDR or DDR
>
>
>
> The Topspin roll should include Intel compiler support, I will
> admit this is not well documented, we are working to correct this. Look
> for mpicc.i, mpiCC.i, mpif77.i, and mpif90.i in
> /usr/local/topspin/mpi/mpich/bin/.
>
>
>
> Scott Weitzenkamp
>
> SQA and Release Manager
>
> Server Virtualization Business Unit
>
> Cisco Systems
>
>
>
>
>
>
> ________________________________
>
>
> From: mvapich-discuss-bounces at cse.ohio-state.edu
> [mailto:mvapich-discuss-bounces at cse.ohio-state.edu] On Behalf Of Mike
> Hanby
> Sent: Wednesday, February 07, 2007 10:30 AM
> To: mvapich-discuss at cse.ohio-state.edu
> Subject: RE: [mvapich-discuss] SDR or DDR
>
> Thanks, I feel like a buffoon :-)
>
>
>
> I compiled MVAPICH 0.9.8 on a x86_64 Rocks 4.2.1 Cluster
> system using Intel 9.1 compilers. I have the Topspin roll installed on
> the cluster, where /usr/local/topspin contains the libraries and
> binaries for Infiniband. I could use the mvapich included with the
> Topspin roll, however my users want their applications compiled using
> the Intel compilers, and the mvapich on the roll is compiled with GNU.
>
>
>
> If I compile a simple helloworld mpi c program using
> mpicc and then run it using the command I get a Segmentation Fault:
>
> $ mpirun_rsh -np 1 node1 ~/mpi_hello
>
> bash: line 1: 12801 Segmentation fault /usr/bin/env
> MPIRUN_MPD=0 MPIRUN_HOST=headnode MPIRUN_PORT=41013
> MPIRUN_PROCESSES=node1:' MPIRUN_RANK=0 MPIRUN_NPROCS=1 MPIRUN_ID=12669
> /home/makeuser/mpi_hello
>
>
>
> I looked through the make log and don't see any errors,
> just a bunch of warnings like:
>
> graph_nbr.c(83): warning #187: use of "=" where "==" may
> have been intended
>
> ( (topo->type != MPI_GRAPH) && (mpi_errno =
> MPI_ERR_TOPOLOGY))
>
>
>
> I've also compiled Amber9 using mpicc, mpixx and mpif77
> and also get a segmentation fault when I attempt to run sander.MPI (an
> Amber9 binary).
>
>
>
> Something tells me I'm doing something wrong.
>
>
>
> Here are the steps I followed to compile:
>
> I edit the file make.mvapich.vapi as follows:
>
> MTHOME=/usr/local/topspin
>
> PREFIX=/share/apps/mvapich/intel/mvapich-0.9.8-64
>
> export CC =icc
>
> export CXX=icpc
>
> export F77=ifort
>
> export F90=ifort
>
> IO_BUS=_PCI_EX_ # For PCI Express
>
> LINKS=_SDR_
>
> export CFLAGS="-D${ARCH} -DUSE_INLINE
> -DEARLY_SEND_COMPLETION -DRDMA_FAST_PATH \
>
> -DVIADEV_RPUT_SUPPORT
> -DLAZY_MEM_UNREGISTER -D_SMP_ -D_SMP_RNDV_ \
>
> $SUPPRESS -D${IO_BUS} -D${LINKS} \
>
> ${HAVE_MPD_RING} -I${MTHOME}/include
> -I${MTHOME}/include/vapi $OPT_FLAG"
>
>
>
> I also have to edit mpid/vapi/viainit.c based on an
> error I received:
>
> case VAPI_PORT_ACTIVE:
>
> #ifdef VAPI_VERSION_CODE
>
> #if 0
>
> #if VAPI_VERSION_CODE >= VAPI_VERSION(4,1,0)
>
> case VAPI_CLIENT_REREGISTER:
>
> case VAPI_RECEIVE_QUEUE_DRAINED:
>
> case VAPI_ECC_DETECT:
>
> case VAPI_PATH_MIG_ARMED:
>
> #endif
>
> #endif
>
> #endif
>
>
>
> ...and...
>
>
>
> case VAPI_PORT_ERROR:
>
> #ifdef VAPI_VERSION_CODE
>
> #if 0
>
> #if VAPI_VERSION_CODE >= VAPI_VERSION(4,1,0)
>
> case VAPI_SRQ_CATASTROPHIC_ERROR:
>
> #endif
>
> #endif
>
> #endif
>
>
>
> I then just run:
>
> ./make.mvapich.vapi
>
>
>
> It appears to succeed and the directories and files get
> created in the --prefix location. Does anyone see anything glaringly
> wrong here?
>
>
>
> Thanks, Mike
>
>
>
>
> ________________________________
>
>
> From: Gilad Shainer [mailto:Shainer at mellanox.com]
> Sent: Wednesday, February 07, 2007 11:36
> To: Scott Weitzenkamp (sweitzen); Mike Hanby;
> mvapich-discuss at cse.ohio-state.edu
> Subject: RE: [mvapich-discuss] SDR or DDR
>
>
>
> The DDR or SDR is the link speed, 10Gb/s or 20Gb/s, and
> not the RAM.
>
>
>
> Gilad.
>
>
>
>
> ________________________________
>
>
> From: mvapich-discuss-bounces at cse.ohio-state.edu
> [mailto:mvapich-discuss-bounces at cse.ohio-state.edu] On Behalf Of Scott
> Weitzenkamp (sweitzen)
> Sent: Wednesday, February 07, 2007 8:57 AM
> To: Mike Hanby; mvapich-discuss at cse.ohio-state.edu
> Subject: RE: [mvapich-discuss] SDR or DDR
>
> Your HCA is SDR, with DDR you will see DDR in the
> tvflash -i output.
>
>
>
> Scott
>
>
>
>
> ________________________________
>
>
> From: mvapich-discuss-bounces at cse.ohio-state.edu
> [mailto:mvapich-discuss-bounces at cse.ohio-state.edu] On Behalf Of Mike
> Hanby
> Sent: Wednesday, February 07, 2007 8:36 AM
> To: mvapich-discuss at cse.ohio-state.edu
> Subject: [mvapich-discuss] SDR or DDR
>
> I need to fill in the value for LINKS=_DDR_ or
> _SDR_
>
> Does anyone know how I tell whether my
> Infiniband cards have DDR or SDR ram?
>
>
>
> Also, my cards are PCI Express. For IO_BUS, I
> would choose _PCI_EX_, correct?
>
>
>
> tvflash -i command reports the following:
>
>
>
> HCA #0: MT25208 Tavor Compat, Lion Cub, revision
> A0
>
> Primary image is v4.7.600 build 3.2.0.110,
> with label 'HCA.LionCub.A0'
>
> Secondary image is v4.6.000 build 3.1.0.113,
> with label 'HCA.LionCub.A0'
>
>
>
> Vital Product Data
>
> Product Name: Lion cub
>
> P/N: 99-00026-01
>
> E/C: Rev: B04
>
> S/N: TS0548X03797
>
> Freq/Power: PW=10W;PCIe 8X
>
> Date Code: 0548
>
> Checksum: Ok
>
>
>
> Thanks, Mike
>
>
More information about the mvapich-discuss
mailing list