[mvapich-discuss] Re: MVAPICH2 for rnfs capable 2.6.17.10 kernel
Chen, Helen Y
hycsw at sandia.gov
Thu Nov 30 16:05:52 EST 2006
Sundeep,
> Please note that you will need to download and install the iwarp branch
> from the OpenFabrics svn located at
> https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
How do I compile and install the userspace libraries and utilities after I download
the iwarp branch without replacing the stack of the running kernel and modules?
Thanks,
Helen
-----Original Message-----
From: Sundeep Narravula [mailto:narravul at cse.ohio-state.edu]
Sent: Thu 11/30/2006 12:56 PM
To: Chen, Helen Y
Cc: panda at cse.ohio-state.edu; mvapich-discuss at cse.ohio-state.edu; Fischer, Noah
Subject: Re: [mvapich-discuss] Re: MVAPICH2 for rnfs capable 2.6.17.10 kernel
Hi Helen,
The instructions for building and installing MVAPICH2 using iWARP are
available in our user guide at
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html#x1-110004.4.2
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html
I am listing the basic instructions here for your conveniance.
-------------------------------------------
Using Default build configuration:
Go to the mvapich2-0.9.8 directory. We have included a single build script
for Gen2-iWARP (make.mvapich2.iwarp) that takes care of different
platforms, compilers and architectures. By default, the compilation script
uses gcc. In order to use a specific compiler, set the CC, CXX, F77, and
F90 variables to the appropriate commands. MVAPICH2 currently supports the
GCC, Intel and PGI compiler suites for Gen2-iWARP.
Please note that you will need to download and install the iwarp branch
from the OpenFabrics svn located at
https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
After setting all the required parameters, the script make.mvapich2.iwarp
configures, builds and installs the entire package in the directory
specified by the variable PREFIX.
Running MPI applications with Gen2-iWARP support:
In MVAPICH2, Gen2-iWARP support is enabled with the two CFLAGS: (i)
-DRDMA_CM and (ii) -DRDMA_CM_RNIC. These can be enabled from our build
script (make.mvapich2.iwarp or make.mvapich2.gen2).
In addition to these, the system needs further one time setup for enabling
RDMA_CM usage. The required steps are as follows:
1. Setup the OpenFabrics iWARP Branch: The OpenFabrics iwarp branch
located at https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable needs
to be installed. Detailed instructions for this installation are
available on the OpenFabrics Wiki at
https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Chelsio+T3
2. Setup the RDMA_CM device: RDMA_CM device needs to be setup, configured
with an IP address and connected to the network.
3. Setup the Local Address File: Create the file (/etc/mv2.conf) with the
local IP address to be used by RDMA_CM. Example:
$ echo 10.1.1.1 >> /etc/mv2.conf
-------------------------------------------
Further troubleshooting instructions are present at:
http://nowlab.cse.ohio-state.edu/projects/mpi-iba/download-mvapich2/mvapich2_user_guide.html#x1-400007.3
Regards,
--Sundeep.
On Thu, 30 Nov 2006, Chen, Helen Y wrote:
> Hi,
>
> We are running a 2.6.17.10 kernel with rnfs patch and the stable
> iwarp-stack from openib. We'd like to installed MVAPICH2 that can use
> either IB or iWARP RDMA. What are the exact instructions for building
> the package? We are new at this so any help from you will be
> appreciated.
>
> Thanks,
> Helen
>
>
>
> -----Original Message-----
> From: Dhabaleswar Panda [mailto:panda at cse.ohio-state.edu]
> Sent: Thu 11/30/2006 12:13 PM
> To: Chen, Helen Y
> Cc: panda at cse.ohio-state.edu; Chen, Helen Y
> Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel
>
> Hi Helen,
>
> > Thanks for your information. Actually my student had tried to install =
> > MVAPICH2 and failed, which I believe is because we are new at this. So =
> > my questions remains how to keep my iwarp-satck intact and install =
> > MVAPICH2 on i
> > the rnfs capable 2.6.17.10 kernel? =20
>
> Thanks for your prompt reply.
>
> Sorry to know that you are experiencing problem here. We are able to
> run it sucessfully in our lab. Steve Wise is also able to run it.
>
> I believe you are trying to use it with Chelsio adapter. I am
> wondering whether your student took a detailed look at the MVAPICH2
> 0.9.8 user guide (available from mvapich page). It has a detailed set
> of instructions on how to install and run MVAPICH2 with the iWARP
> stack.
>
> We will be happy to extend help here so that you can use MVAPICH2 with
> the iWARP stack. May I request your student to send out a note to us
> using our public `mvapich-discuss at cse.ohio-state.edu' mailing list
> with the problems he is encountering and we will be happy to help.
>
> Best Regards,
>
> DK
>
> > Thanks,
> > Helen
> >
> > -----Original Message-----
> > From: Dhabaleswar Panda [mailto:panda at cse.ohio-state.edu]
> > Sent: Thu 11/30/2006 10:58 AM
> > To: Chen, Helen Y
> > Cc: panda at cse.ohio-state.edu
> > Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel
> > =20
> > Hi Helen,=20
> >
> > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and would =
> > =3D
> > > like to build Open MPI that uses either IB or iWARP RDMA through this =
> > =3D
> > > kernel. Can I achieve this by getting the latest stable git and =
> > compile =3D
> > > just the userspace software? If so, how do I go about getting the =
> > latest =3D
> > > stable git, and how to compile just the userspace stuff? I am new at =
> > =3D
> > > this and would really appreciate your help.
> >
> > This is just a private e-mail I am sending to you. Not cc'ing to OpenIB.
> >
> > Not sure whether you have noticed that our latest MVAPICH2 0.9.8
> > release (done immediately before SC '06) supports RDMA CM and can run
> > on both IB and iWARP hardware. It has been tested with Chelsio T3
> > adapter with iWARP support and OFED 1.1 stack with 2.6.17 kernel. I
> > think this is what you are looking for. This release also has added
> > features like Checkpoint-Restart for IB.
> >
> > If you are interested in running a single MPI stack for both IB and
> > iWARP, you may try this release.
> >
> > We will be happy to get your feedbacks.=20
> >
> > Best Regards,=20
> >
> > DK
> >
> >
> > > Thanks,
> > > Helen
> > >=20
> > > ------_=3D_NextPart_001_01C71415.11E95606
> > > Content-Type: text/html;
> > > charset=3Diso-8859-1
> > > Content-Transfer-Encoding: quoted-printable
> > >=20
> > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
> > > <HTML>
> > > <HEAD>
> > > <META HTTP-EQUIV=3D3D"Content-Type" CONTENT=3D3D"text/html; =3D
> > > charset=3D3Diso-8859-1">
> > > <META NAME=3D3D"Generator" CONTENT=3D3D"MS Exchange Server version =3D
> > > 6.5.7638.1">
> > > <TITLE>Re: openMPI for 2.6.17.10 kernel</TITLE>
> > > </HEAD>
> > > <BODY>
> > > <!-- Converted from text/plain format -->
> > >=20
> > > <P><FONT SIZE=3D3D2>Hi,<BR>
> > > <BR>
> > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and would =
> > =3D
> > > like to build Open MPI that uses either IB or iWARP RDMA through this =
> > =3D
> > > kernel. Can I achieve this by getting the latest stable git and =
> > compile =3D
> > > just the userspace software? If so, how do I go about getting the =
> > latest =3D
> > > stable git, and how to compile just the userspace stuff? I am =
> > new =3D
> > > at this and would really appreciate your help.<BR>
> > > <BR>
> > > Thanks,<BR>
> > > Helen<BR>
> > > </FONT>
> > > </P>
> > >=20
> > > </BODY>
> > > </HTML>
> > > ------_=3D_NextPart_001_01C71415.11E95606--
> > >=20
> > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D
> > > MIME-Version: 1.0
> > > Content-Type: text/plain;
> > > charset=3Dus-ascii
> > > Content-Transfer-Encoding: 7bit
> > > Content-Disposition: inline
> > >=20
> > > _______________________________________________
> > > openib-general mailing list
> > > openib-general at openib.org
> > > http://openib.org/mailman/listinfo/openib-general
> > >=20
> > > To unsubscribe, please visit =
> > http://openib.org/mailman/listinfo/openib-general
> > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D--
> > >=20
> >
> >
> >
> > ------_=_NextPart_001_01C714B0.E043C99C
> > Content-Type: text/html;
> > charset=iso-8859-1
> > Content-Transfer-Encoding: quoted-printable
> >
> > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
> > <HTML>
> > <HEAD>
> > <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
> > charset=3Diso-8859-1">
> > <META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
> > 6.5.7638.1">
> > <TITLE>RE: [openib-general] openMPI for 2.6.17.10 kernel</TITLE>
> > </HEAD>
> > <BODY>
> > <!-- Converted from text/plain format -->
> >
> > <P><FONT SIZE=3D2>Panda,<BR>
> > <BR>
> > Thanks for your information. Actually my student had tried to =
> > install MVAPICH2 and failed, which I believe is because we are new at =
> > this. So my questions remains how to keep my iwarp-satck intact =
> > and install MVAPICH2 on i<BR>
> > the rnfs capable 2.6.17.10 kernel? <BR>
> > <BR>
> > Thanks,<BR>
> > Helen<BR>
> > <BR>
> > -----Original Message-----<BR>
> > From: Dhabaleswar Panda [<A =
> > HREF=3D"mailto:panda at cse.ohio-state.edu">mailto:panda at cse.ohio-state.edu<=
> > /A>]<BR>
> > Sent: Thu 11/30/2006 10:58 AM<BR>
> > To: Chen, Helen Y<BR>
> > Cc: panda at cse.ohio-state.edu<BR>
> > Subject: Re: [openib-general] openMPI for 2.6.17.10 kernel<BR>
> > <BR>
> > Hi Helen,<BR>
> > <BR>
> > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and =
> > would =3D<BR>
> > > like to build Open MPI that uses either IB or iWARP RDMA through =
> > this =3D<BR>
> > > kernel. Can I achieve this by getting the latest stable git and =
> > compile =3D<BR>
> > > just the userspace software? If so, how do I go about getting the =
> > latest =3D<BR>
> > > stable git, and how to compile just the userspace stuff? I am =
> > new at =3D<BR>
> > > this and would really appreciate your help.<BR>
> > <BR>
> > This is just a private e-mail I am sending to you. Not cc'ing to =
> > OpenIB.<BR>
> > <BR>
> > Not sure whether you have noticed that our latest MVAPICH2 0.9.8<BR>
> > release (done immediately before SC '06) supports RDMA CM and can =
> > run<BR>
> > on both IB and iWARP hardware. It has been tested with Chelsio T3<BR>
> > adapter with iWARP support and OFED 1.1 stack with 2.6.17 kernel. I<BR>
> > think this is what you are looking for. This release also has added<BR>
> > features like Checkpoint-Restart for IB.<BR>
> > <BR>
> > If you are interested in running a single MPI stack for both IB and<BR>
> > iWARP, you may try this release.<BR>
> > <BR>
> > We will be happy to get your feedbacks.<BR>
> > <BR>
> > Best Regards,<BR>
> > <BR>
> > DK<BR>
> > <BR>
> > <BR>
> > > Thanks,<BR>
> > > Helen<BR>
> > ><BR>
> > > ------_=3D_NextPart_001_01C71415.11E95606<BR>
> > > Content-Type: text/html;<BR>
> > > charset=3Diso-8859-1<BR>
> > > Content-Transfer-Encoding: quoted-printable<BR>
> > ><BR>
> > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML =
> > 3.2//EN"><BR>
> > > <HTML><BR>
> > > <HEAD><BR>
> > > <META HTTP-EQUIV=3D3D"Content-Type" =
> > CONTENT=3D3D"text/html; =3D<BR>
> > > charset=3D3Diso-8859-1"><BR>
> > > <META NAME=3D3D"Generator" CONTENT=3D3D"MS =
> > Exchange Server version =3D<BR>
> > > 6.5.7638.1"><BR>
> > > <TITLE>Re: openMPI for 2.6.17.10 kernel</TITLE><BR>
> > > </HEAD><BR>
> > > <BODY><BR>
> > > <!-- Converted from text/plain format --><BR>
> > ><BR>
> > > <P><FONT SIZE=3D3D2>Hi,<BR><BR>
> > > <BR><BR>
> > > I am running a 2.6.17.10 kernel with Tom Tucker's rnfs patch and =
> > would =3D<BR>
> > > like to build Open MPI that uses either IB or iWARP RDMA through =
> > this =3D<BR>
> > > kernel. Can I achieve this by getting the latest stable git and =
> > compile =3D<BR>
> > > just the userspace software? If so, how do I go about getting the =
> > latest =3D<BR>
> > > stable git, and how to compile just the userspace stuff?&nbsp; =
> > I am new =3D<BR>
> > > at this and would really appreciate your help.<BR><BR>
> > > <BR><BR>
> > > Thanks,<BR><BR>
> > > Helen<BR><BR>
> > > </FONT><BR>
> > > </P><BR>
> > ><BR>
> > > </BODY><BR>
> > > </HTML><BR>
> > > ------_=3D_NextPart_001_01C71415.11E95606--<BR>
> > ><BR>
> > > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D<BR>
> > > MIME-Version: 1.0<BR>
> > > Content-Type: text/plain;<BR>
> > > charset=3Dus-ascii<BR>
> > > Content-Transfer-Encoding: 7bit<BR>
> > > Content-Disposition: inline<BR>
> > ><BR>
> > > _______________________________________________<BR>
> > > openib-general mailing list<BR>
> > > openib-general at openib.org<BR>
> > > <A =
> > HREF=3D"http://openib.org/mailman/listinfo/openib-general">http://openib.=
> > org/mailman/listinfo/openib-general</A><BR>
> > ><BR>
> > > To unsubscribe, please visit <A =
> > HREF=3D"http://openib.org/mailman/listinfo/openib-general">http://openib.=
> > org/mailman/listinfo/openib-general</A><BR>
> > > =
> > --=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D0218902636=3D=3D--<BR>
> > ><BR>
> > <BR>
> > <BR>
> > </FONT>
> > </P>
> >
> > </BODY>
> > </HTML>
> > ------_=_NextPart_001_01C714B0.E043C99C--
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20061130/758a9fbb/attachment-0001.html
More information about the mvapich-discuss
mailing list