[mvapich-discuss] Re: MVAPICH2 for rnfs capable 2.6.17.10 kernel
Sundeep Narravula
narravul at cse.ohio-state.edu
Thu Nov 30 16:33:59 EST 2006
> > Please note that you will need to download and install the iwarp branch
> > from the OpenFabrics svn located at
> > https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
>
> How do I compile and install the userspace libraries and utilities after I download
> the iwarp branch without replacing the stack of the running kernel and modules?
In theory, you should be able to configure all the userspace libraries
with an installation target directory specified other than the default
(where your existing installation would be).
$ ./autogen.sh ; ./configure --help
....
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc. You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
....
However, we have not tried this kind of setup of the iwarp branch.
Hope that helps.
--Sundeep.
>
> Thanks,
> Helen
> -----Original Message-----
> From: Sundeep Narravula [mailto:narravul at cse.ohio-state.edu]
> Sent: Thu 11/30/2006 12:56 PM
> To: Chen, Helen Y
> Cc: panda at cse.ohio-state.edu; mvapich-discuss at cse.ohio-state.edu; Fischer, Noah
> Subject: Re: [mvapich-discuss] Re: MVAPICH2 for rnfs capable 2.6.17.10 kernel
>
> Hi Helen,
>
> The instructions for building and installing MVAPICH2 using iWARP are
> available in our user guide at
> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html#x1-110004.4.2
> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html
>
> I am listing the basic instructions here for your conveniance.
>
> -------------------------------------------
> Using Default build configuration:
>
> Go to the mvapich2-0.9.8 directory. We have included a single build script
> for Gen2-iWARP (make.mvapich2.iwarp) that takes care of different
> platforms, compilers and architectures. By default, the compilation script
> uses gcc. In order to use a specific compiler, set the CC, CXX, F77, and
> F90 variables to the appropriate commands. MVAPICH2 currently supports the
> GCC, Intel and PGI compiler suites for Gen2-iWARP.
>
> Please note that you will need to download and install the iwarp branch
> from the OpenFabrics svn located at
> https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
>
> After setting all the required parameters, the script make.mvapich2.iwarp
> configures, builds and installs the entire package in the directory
> specified by the variable PREFIX.
>
>
>
> Running MPI applications with Gen2-iWARP support:
>
> In MVAPICH2, Gen2-iWARP support is enabled with the two CFLAGS: (i)
> -DRDMA_CM and (ii) -DRDMA_CM_RNIC. These can be enabled from our build
> script (make.mvapich2.iwarp or make.mvapich2.gen2).
>
> In addition to these, the system needs further one time setup for enabling
> RDMA_CM usage. The required steps are as follows:
>
> 1. Setup the OpenFabrics iWARP Branch: The OpenFabrics iwarp branch
> located at https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable needs
> to be installed. Detailed instructions for this installation are
> available on the OpenFabrics Wiki at
> https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Chelsio+T3
>
> 2. Setup the RDMA_CM device: RDMA_CM device needs to be setup, configured
> with an IP address and connected to the network.
>
> 3. Setup the Local Address File: Create the file (/etc/mv2.conf) with the
> local IP address to be used by RDMA_CM. Example:
> $ echo 10.1.1.1 >> /etc/mv2.conf
>
> -------------------------------------------
>
> Further troubleshooting instructions are present at:
>
> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html#x1-400007.3
>
> Regards,
> --Sundeep.
>
>
>
>
> On Thu, 30 Nov 2006, Chen, Helen Y wrote:
>
> > Hi,
> >
> > We are running a 2.6.17.10 kernel with rnfs patch and the stable
> > iwarp-stack from openib. We'd like to installed MVAPICH2 that can use
> > either IB or iWARP RDMA. What are the exact instructions for building
> > the package? We are new at this so any help from you will be
> > appreciated.
> >
> > Thanks,
> > Helen
> >
> >
> >
> > -----Original Message-----
> > From: Dhabaleswar Panda [mailto:panda at cse.ohio-state.edu]
> > Sent: Thu 11/30/2006 12:13 PM
> > To: Chen, Helen Y
> > Cc: panda at cse.ohio-state.edu; Chen, Helen Y
> > Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel
> >
> > Hi Helen,
> >
> > > Thanks for your information. Actually my student had tried to install =
> > > MVAPICH2 and failed, which I believe is because we are new at this. So =
> > > my questions remains how to keep my iwarp-satck intact and install =
> > > MVAPICH2 on i
> > > the rnfs capable 2.6.17.10 kernel? =20
> >
> > Thanks for your prompt reply.
> >
> > Sorry to know that you are experiencing problem here. We are able to
> > run it sucessfully in our lab. Steve Wise is also able to run it.
> >
> > I believe you are trying to use it with Chelsio adapter. I am
> > wondering whether your student took a detailed look at the MVAPICH2
> > 0.9.8 user guide (available from mvapich page). It has a detailed set
> > of instructions on how to install and run MVAPICH2 with the iWARP
> > stack.
> >
> > We will be happy to extend help here so that you can use MVAPICH2 with
> > the iWARP stack. May I request your student to send out a note to us
> > using our public `mvapich-discuss at cse.ohio-state.edu' mailing list
> > with the problems he is encountering and we will be happy to help.
> >
> > Best Regards,
> >
> > DK
> >
> > > Thanks,
> > > Helen
> > >
> > > -----Original Message-----
> > > From: Dhabaleswar Panda [mailto:panda at cse.ohio-state.edu]
> > > Sent: Thu 11/30/2006 10:58 AM
> > > To: Chen, Helen Y
> > > Cc: panda at cse.ohio-state.edu
> > > Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel
> > > =20
> > > Hi Helen,=20
> > >
> > > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and would =
> > > =3D
> > > > like to build Open MPI that uses either IB or iWARP RDMA through this =
> > > =3D
> > > > kernel. Can I achieve this by getting the latest stable git and =
> > > compile =3D
> > > > just the userspace software? If so, how do I go about getting the =
> > > latest =3D
> > > > stable git, and how to compile just the userspace stuff? I am new at =
> > > =3D
> > > > this and would really appreciate your help.
> > >
> > > This is just a private e-mail I am sending to you. Not cc'ing to OpenIB.
> > >
> > > Not sure whether you have noticed that our latest MVAPICH2 0.9.8
> > > release (done immediately before SC '06) supports RDMA CM and can run
> > > on both IB and iWARP hardware. It has been tested with Chelsio T3
> > > adapter with iWARP support and OFED 1.1 stack with 2.6.17 kernel. I
> > > think this is what you are looking for. This release also has added
> > > features like Checkpoint-Restart for IB.
> > >
> > > If you are interested in running a single MPI stack for both IB and
> > > iWARP, you may try this release.
> > >
> > > We will be happy to get your feedbacks.=20
> > >
> > > Best Regards,=20
> > >
> > > DK
> > >
> > >
> > > > Thanks,
> > > > Helen
> > > >=20
> > > > ------_=3D_NextPart_001_01C71415.11E95606
> > > > Content-Type: text/html;
> > > > charset=3Diso-8859-1
> > > > Content-Transfer-Encoding: quoted-printable
> > > >=20
> > > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
> > > > <HTML>
> > > > <HEAD>
> > > > <META HTTP-EQUIV=3D3D"Content-Type" CONTENT=3D3D"text/html; =3D
> > > > charset=3D3Diso-8859-1">
> > > > <META NAME=3D3D"Generator" CONTENT=3D3D"MS Exchange Server version =3D
> > > > 6.5.7638.1">
> > > > <TITLE>Re: openMPI for 2.6.17.10 kernel</TITLE>
> > > > </HEAD>
> > > > <BODY>
> > > > <!-- Converted from text/plain format -->
> > > >=20
> > > > <P><FONT SIZE=3D3D2>Hi,<BR>
> > > > <BR>
> > > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and would =
> > > =3D
> > > > like to build Open MPI that uses either IB or iWARP RDMA through this =
> > > =3D
> > > > kernel. Can I achieve this by getting the latest stable git and =
> > > compile =3D
> > > > just the userspace software? If so, how do I go about getting the =
> > > latest =3D
> > > > stable git, and how to compile just the userspace stuff? I am =
> > > new =3D
> > > > at this and would really appreciate your help.<BR>
> > > > <BR>
> > > > Thanks,<BR>
> > > > Helen<BR>
> > > > </FONT>
> > > > </P>
> > > >=20
> > > > </BODY>
> > > > </HTML>
> > > > ------_=3D_NextPart_001_01C71415.11E95606--
> > > >=20
> > > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D
> > > > MIME-Version: 1.0
> > > > Content-Type: text/plain;
> > > > charset=3Dus-ascii
> > > > Content-Transfer-Encoding: 7bit
> > > > Content-Disposition: inline
> > > >=20
> > > > _______________________________________________
> > > > openib-general mailing list
> > > > openib-general at openib.org
> > > > http://openib.org/mailman/listinfo/openib-general
> > > >=20
> > > > To unsubscribe, please visit =
> > > http://openib.org/mailman/listinfo/openib-general
> > > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D--
> > > >=20
> > >
> > >
> > >
> > > ------_=_NextPart_001_01C714B0.E043C99C
> > > Content-Type: text/html;
> > > charset=iso-8859-1
> > > Content-Transfer-Encoding: quoted-printable
> > >
> > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
> > > <HTML>
> > > <HEAD>
> > > <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
> > > charset=3Diso-8859-1">
> > > <META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
> > > 6.5.7638.1">
> > > <TITLE>RE: [openib-general] openMPI for 2.6.17.10 kernel</TITLE>
> > > </HEAD>
> > > <BODY>
> > > <!-- Converted from text/plain format -->
> > >
> > > <P><FONT SIZE=3D2>Panda,<BR>
> > > <BR>
> > > Thanks for your information. Actually my student had tried to =
> > > install MVAPICH2 and failed, which I believe is because we are new at =
> > > this. So my questions remains how to keep my iwarp-satck intact =
> > > and install MVAPICH2 on i<BR>
> > > the rnfs capable 2.6.17.10 kernel? <BR>
> > > <BR>
> > > Thanks,<BR>
> > > Helen<BR>
> > > <BR>
> > > -----Original Message-----<BR>
> > > From: Dhabaleswar Panda [<A =
> > > HREF=3D"mailto:panda at cse.ohio-state.edu">mailto:panda at cse.ohio-state.edu<=
> > > /A>]<BR>
> > > Sent: Thu 11/30/2006 10:58 AM<BR>
> > > To: Chen, Helen Y<BR>
> > > Cc: panda at cse.ohio-state.edu<BR>
> > > Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel<BR>
> > > <BR>
> > > Hi Helen,<BR>
> > > <BR>
> > > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and =
> > > would =3D<BR>
> > > > like to build Open MPI that uses either IB or iWARP RDMA through =
> > > this =3D<BR>
> > > > kernel. Can I achieve this by getting the latest stable git and =
> > > compile =3D<BR>
> > > > just the userspace software? If so, how do I go about getting the =
> > > latest =3D<BR>
> > > > stable git, and how to compile just the userspace stuff? I am =
> > > new at =3D<BR>
> > > > this and would really appreciate your help.<BR>
> > > <BR>
> > > This is just a private e-mail I am sending to you. Not cc'ing to =
> > > OpenIB.<BR>
> > > <BR>
> > > Not sure whether you have noticed that our latest MVAPICH2 0.9.8<BR>
> > > release (done immediately before SC '06) supports RDMA CM and can =
> > > run<BR>
> > > on both IB and iWARP hardware. It has been tested with Chelsio T3<BR>
> > > adapter with iWARP support and OFED 1.1 stack with 2.6.17 kernel. I<BR>
> > > think this is what you are looking for. This release also has added<BR>
> > > features like Checkpoint-Restart for IB.<BR>
> > > <BR>
> > > If you are interested in running a single MPI stack for both IB and<BR>
> > > iWARP, you may try this release.<BR>
> > > <BR>
> > > We will be happy to get your feedbacks.<BR>
> > > <BR>
> > > Best Regards,<BR>
> > > <BR>
> > > DK<BR>
> > > <BR>
> > > <BR>
> > > > Thanks,<BR>
> > > > Helen<BR>
> > > ><BR>
> > > > ------_=3D_NextPart_001_01C71415.11E95606<BR>
> > > > Content-Type: text/html;<BR>
> > > > charset=3Diso-8859-1<BR>
> > > > Content-Transfer-Encoding: quoted-printable<BR>
> > > ><BR>
> > > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML =
> > > 3.2//EN"><BR>
> > > > <HTML><BR>
> > > > <HEAD><BR>
> > > > <META HTTP-EQUIV=3D3D"Content-Type" =
> > > CONTENT=3D3D"text/html; =3D<BR>
> > > > charset=3D3Diso-8859-1"><BR>
> > > > <META NAME=3D3D"Generator" CONTENT=3D3D"MS =
> > > Exchange Server version =3D<BR>
> > > > 6.5.7638.1"><BR>
> > > > <TITLE>Re: openMPI for 2.6.17.10 kernel</TITLE><BR>
> > > > </HEAD><BR>
> > > > <BODY><BR>
> > > > <!-- Converted from text/plain format --><BR>
> > > ><BR>
> > > > <P><FONT SIZE=3D3D2>Hi,<BR><BR>
> > > > <BR><BR>
> > > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and =
> > > would =3D<BR>
> > > > like to build Open MPI that uses either IB or iWARP RDMA through =
> > > this =3D<BR>
> > > > kernel. Can I achieve this by getting the latest stable git and =
> > > compile =3D<BR>
> > > > just the userspace software? If so, how do I go about getting the =
> > > latest =3D<BR>
> > > > stable git, and how to compile just the userspace stuff?&nbsp; =
> > > I am new =3D<BR>
> > > > at this and would really appreciate your help.<BR><BR>
> > > > <BR><BR>
> > > > Thanks,<BR><BR>
> > > > Helen<BR><BR>
> > > > </FONT><BR>
> > > > </P><BR>
> > > ><BR>
> > > > </BODY><BR>
> > > > </HTML><BR>
> > > > ------_=3D_NextPart_001_01C71415.11E95606--<BR>
> > > ><BR>
> > > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D<BR>
> > > > MIME-Version: 1.0<BR>
> > > > Content-Type: text/plain;<BR>
> > > > charset=3Dus-ascii<BR>
> > > > Content-Transfer-Encoding: 7bit<BR>
> > > > Content-Disposition: inline<BR>
> > > ><BR>
> > > > _______________________________________________<BR>
> > > > openib-general mailing list<BR>
> > > > openib-general at openib.org<BR>
> > > > <A =
> > > HREF=3D"http://openib.org/mailman/listinfo/openib-general">http://openib.=
> > > org/mailman/listinfo/openib-general</A><BR>
> > > ><BR>
> > > > To unsubscribe, please visit <A =
> > > HREF=3D"http://openib.org/mailman/listinfo/openib-general">http://openib.=
> > > org/mailman/listinfo/openib-general</A><BR>
> > > > =
> > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D--<BR>
> > > ><BR>
> > > <BR>
> > > <BR>
> > > </FONT>
> > > </P>
> > >
> > > </BODY>
> > > </HTML>
> > > ------_=_NextPart_001_01C714B0.E043C99C--
> > >
> >
> >
> >
>
>
>
More information about the mvapich-discuss
mailing list