[mvapich-discuss] Re: [openfabrics-ewg] Announcing the release of
MVAPICH2 0.9.8 with Checkpoint/Restart, iWARP,
RDMA CM-based connection manageme
david elsen
elsen_david at yahoo.com
Mon Nov 13 20:57:23 EST 2006
Sundeep,
Finally I was able to build the OSU MPI tools.
Thanks for your help.
Now I have some problem while trying to run it. Please see the following:
[root at ammasso1 bin]# ./mpdtrace
/root/0.9.8-RELEASE/bin/mpdroot: error while loading shared libraries: librdmacm.so: cannot open shared object file: No such file or directory
mpdtrace: mpd_uncaught_except_tb handling:
exceptions.TypeError: not all arguments converted during string formatting
/root/0.9.8-RELEASE/bin/mpdlib.py 899 __init__
mpd_print(1,'forked process failed; status=' % status)
./mpdtrace 46 mpdtrace
conSock = MPDConClientSock(mpdroot=mpdroot,secretword=parmdb['MPD_SECRETWORD'])
./mpdtrace 83 ?
mpdtrace()
[root at ammasso1 bin]#
[root at ammasso1 bin]#
[root at ammasso1 bin]# ./mpiexec -n 1 ./cpi
/root/0.9.8-RELEASE/bin/mpdroot: error while loading shared libraries: librdmacm.so: cannot open shared object file: No such file or directory
mpiexec_ammasso1: mpd_uncaught_except_tb handling:
exceptions.TypeError: not all arguments converted during string formatting
/root/0.9.8-RELEASE/bin/mpdlib.py 899 __init__
mpd_print(1,'forked process failed; status=' % status)
./mpiexec 231 mpiexec
conSock = MPDConClientSock(mpdroot=mpdroot,secretword=parmdb['MPD_SECRETWORD'])
./mpiexec 1509 ?
mpiexec()
[root at ammasso1 bin]#
Can you let me know how to enable the ssh on my nodes?
Thanks,
David
david elsen <elsen_david at yahoo.com> wrote: Sundeep,
See the following error messages:
It looks like something is not set on my computer.
Can someone help me with this?
David
[root at ammasso2 mvapich]# ./make.mvapich2.iwarp
Configuring MVAPICH2...
Configuring MPICH2 version MVAPICH2-0.9.8 with --prefix=/usr/local/mvapich2 --with-device=osu_ch3:mrail --with-rdma=gen2 --with-pm=mpd --disable-romio --without-mpe
sourcing /usr/local/mvapich2/src/pm/mpd/setup_pm
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for type of weak symbol support... pragma weak
checking whether __attribute__ ((weak)) allowed... yes
checking for multiple weak symbol support... yes
checking whether we are using the GNU Fortran 77 compiler... no
checking whether g77 accepts -g... no
checking how to get verbose linking output from g77... configure: WARNING: compilation failed
checking for Fortran libraries of g77...
checking whether C can link with ... yes
checking for linker for Fortran main programs... configure: error: Could not compile Fortran test program
checking for type of weak symbol support... pragma weak
checking whether __attribute__ ((weak)) allowed... yes
checking for multiple weak symbol support... yes
checking whether we are using the GNU Fortran 77 compiler... no
checking whether g77 accepts -g... no
checking how to get verbose linking output from g77... configure: WARNING: compilation failed
checking for Fortran libraries of g77...
checking whether C can link with ... yes
checking for linker for Fortran main programs... configure: error: Could not compile Fortran test program
Building MVAPICH2...
make: *** No targets specified and no makefile found. Stop.
make: *** No targets specified and no makefile found. Stop.
MVAPICH2 installation...
make: *** No rule to make target `install'. Stop.
make: *** No rule to make target `install'. Stop.
Congratulations on successfully building MVAPICH2. Please send your feedback to mvapich-discuss at cse.ohio-state.edu
Sundeep Narravula <narravul at cse.ohio-state.edu> wrote: The executables are usually installed in the installation path. The
default for which is /usr/local/mvapich2. Please check this directory for
any executable files.
After a successful compilation you should have the executables in your
/path-to-the-mvapich2-build-dir/bin directory.
Otherwise, please send me your make logs; i.e. config-mine.log and
make-mine.log.
Thanks,
--Sundeep.
On Mon, 13 Nov 2006, david elsen wrote:
> There is no executable files created in
> /usr/local/bin
>
> There is no bin directory in my MVAPICH2 directory.
>
> Sundeep Narravula wrote:
>
>
> > I tried from the trunk and I could run the makefile for iWARP without any error. But I do not see any executable there.
> > What else do I need to do?
> >
>
> I am not sure which excutable you are refering to? Once you are done with
> compiling mvapich2, mvapich2 is ready for use and you should have all your
> mpi related executables in the install path and the ./bin/ directory of
> your build path.
>
> Information related to running mpi programs, using iWARP device, etc. are
> all detailed in the mvapich2 user guide available on our web-page.
> http://nowlab.cse.ohio-state.edu/projects/mpi-iba/ --> UserGuide
>
> Does this answer your question? Please let me know if you have any further
> questions.
>
> Cheers,
> --Sundeep.
>
> > Sundeep Narravula wrote:
> > Your path should probably be /usr/local/
> >
> > The files required for OSU MVAPICH2 will be in
> > /usr/local/include
> > and
> > /usr/local/lib (or lib64 for 64bit machines)
> >
> > You will have to set the path to /usr/local only.
> >
> > export OPEN_IB_HOME=/usr/local
> >
> > This should let you compile mvapich2.
> >
> >
> >
> > Alternatively, you can also download the latest trunk source for mvapich2
> > from our svn which already has a changed default.
> >
> > svn co https://mvapich.cse.ohio-state.edu/svn/mpi/mvapich2/trunk mvapich2
> >
> > Cheers,
> > --Sundeep.
> >
> >
> > On Mon, 13 Nov 2006, david elsen wrote:
> >
> > > Does it mean that if I have my iWARP code in
> > >
> > > /usr/src/gen2/branches/iwarp
> > >
> > > then
> > > OPEN_IB_HOME = /usr/src/gen2/branches/iwarp
> > >
> > >
> > > Sundeep Narravula wrote:
> > > Hi David,
> > >
> > > > My question is why should I have this reference to /usr/local/ofed
> > > > there if I do not need to download the OFED distribution code to run
> > > > the iWARP.
> > >
> > > The variable OPEN_IB_HOME in make.mvapich2.iwarp sets the path to the Gen2
> > > installation that you intend to use for iwarp. The default in the script
> > > is /usr/local/ofed.
> > >
> > > Based on your installation, please set this variable appropriately.
> > >
> > > export OPEN_IB_HOME=/usr/local
> > >
> > > > Is it possible to add more information in your MVAPICH2 0.9.8 User Guide describing how to build this and what are the dependencies?
> > >
> > > We have updated our userguide with the installation information. The
> > > dependencies that we have are the installation of OF iwarp branch, the
> > > setup of rdma-cm module and the setup of the underlying network.
> > >
> > > Regards,
> > > --Sundeep.
> > >
> > > >
> > > > David
> > > > Sundeep Narravula wrote:
> > > > Hi David,
> > > >
> > > > > iWARP is actually a part of the Open Fabrics SVN. It is available from
> > > > > a different branch.
> > > > >
> > > > > I am cc'ing this note to my group. One of the students (Sundeep) will
> > > > > send you the detailed instructions on which branch of OF to download
> > > > > and use.
> > > >
> > > > The instructions for setting up iwarp on OpenFrabics is avalable on the
> > > > openIB wiki at
> > > > https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Chelsio+T3
> > > > https://openib.org/tiki/tiki-index.php?page=Install+OpenIB+for+Ammasso1100
> > > >
> > > > Further, the branch you can download from the svn is
> > > > https://openib.org/svn/gen2/branches/iwarp/tags/iwarp-2.6.17-stable
> > > >
> > > > Please let us know if you have any further questions.
> > > >
> > > > Regards,
> > > > --Sundeep.
> > > >
> > > >
> > > > > > Is there any document describing the build process of the MVAPICH2
> > > > > > tool? I am going through the MVAPICH2 0.9.8 users guide and that
> > > > > > does not seem to be giving me the detailed information.
> > > > >
> > > > > We will add this additional information on our user guide.
> > > > >
> > > > > > Can you please provide README files for the iWARP which describes
> > > > > > TODO steps?
> > > > >
> > > > > Sundeep's information will help. If you have any additional questions,
> > > > > please feel free to ask us.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > DK
> > > > >
> > > > > > Thanks,
> > > > > > David
> > > > > >
> > > > > > Dhabaleswar Panda
> > > > wrote:
> > > > > > The MVAPICH team is pleased to announce the availability of MVAPICH2
> > > > > > 0.9.8 with the following NEW features:
> > > > > >
> > > > > > - Checkpoint/Restart support for application transparent systems-level
> > > > > > fault tolerance. BLCR-based support using native InfiniBand Gen2
> > > > > > interface is provided. Flexible interface to work with different
> > > > > > file systems. Tested with ext3 (local disk), NFS and PVFS2.
> > > > > >
> > > > > > Performance of sample applications with checkpoint-restart using
> > > > > > PVFS2 and Lustre can be found here:
> > > > > >
> > > > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html
> > > > > >
> > > > > > - iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.
> > > > > > Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and
> > > > > > drivers.
> > > > > >
> > > > > > - RDMA CM-based Connection management support
> > > > > >
> > > > > > - Shared memory optimizations for collective communication operations.
> > > > > > Efficient algorithms and optimizations for barrier, reduce and
> > > > > > all-reduce operations. Exploits the multi-core optimized shared
> > > > > > memory point-to-point communication support introduced in MVAPICH2
> > > > > > 0.9.6.
> > > > > >
> > > > > > Performance of sample collective operations with this new feature
> > > > > > can be found here:
> > > > > >
> > > > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html
> > > > > >
> > > > > > - uDAPL support for NetEffect 10GigE adapter. Tested with
> > > > > > NetEffect NE010 adapter.
> > > > > >
> > > > > > More details on all features and supported platforms can be obtained
> > > > > > by visiting the following URL:
> > > > > >
> > > > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html
> > > > > >
> > > > > > MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It
> > > > > > continues to deliver excellent performance. Sample performance
> > > > > > numbers include:
> > > > > >
> > > > > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:
> > > > > > Two-sided operations:
> > > > > > - 2.81 microsec one-way latency (4 bytes)
> > > > > > - 1561 MB/sec unidirectional bandwidth
> > > > > > - 2935 MB/sec bidirectional bandwidth
> > > > > >
> > > > > > One-sided operations:
> > > > > > - 4.92 microsec Put latency
> > > > > > - 1569 MB/sec unidirectional Put bandwidth
> > > > > > - 2935 MB/sec bidirectional Put bandwidth
> > > > > >
> > > > > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):
> > > > > > Two-sided operations:
> > > > > > - 2.81 microsec one-way latency (4 bytes)
> > > > > > - 3127 MB/sec unidirectional bandwidth
> > > > > > - 5917 MB/sec bidirectional bandwidth
> > > > > >
> > > > > > One-sided operations:
> > > > > > - 4.37 microsec Put latency
> > > > > > - 3137 MB/sec unidirectional Put bandwidth
> > > > > > - 5917 MB/sec bidirectional Put bandwidth
> > > > > >
> > > > > > - OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:
> > > > > > Two-sided operations:
> > > > > > - 3.01 microsec one-way latency (4 bytes)
> > > > > > - 1402 MB/sec unidirectional bandwidth
> > > > > > - 2238 MB/sec bidirectional bandwidth
> > > > > >
> > > > > > One-sided operations:
> > > > > > - 4.65 microsec Put latency
> > > > > > - 1402 MB/sec unidirectional Put bandwidth
> > > > > > - 2238 MB/sec bidirectional Put bandwidth
> > > > > >
> > > > > > Performance numbers for all other platforms, system configurations and
> > > > > > operations can be viewed by visiting `Performance' section of the
> > > > > > project's web page.
> > > > > >
> > > > > > With the ADI-3-level design, MVAPICH2 0.9.8 delivers similar
> > > > > > performance for two-sided operations compared to MVAPICH 0.9.8.
> > > > > > Organizations and users interested in getting the best performance for
> > > > > > both two-sided and one-sided operations and also want to exploit
> > > > > > advanced features (such as fault tolerance with checkpoint/restart,
> > > > > > iWARP, RDMA CM connection management, multi-threading, integrated
> > > > > > multi-rail, multi-core optimization, memory hook support and optimized
> > > > > > collectives) may migrate from MVAPICH code base to MVAPICH2 code base.
> > > > > >
> > > > > > For downloading MVAPICH2 0.9.8 package and accessing the anonymous
> > > > > > SVN, please visit the following URL:
> > > > > >
> > > > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/
> > > > > >
> > > > > > A stripped down version of this release is also available at the
> > > > > > OpenFabrics SVN.
> > > > > >
> > > > > > All feedbacks, including bug reports and hints for performance tuning,
> > > > > > are welcome. Please post it to the mvapich-discuss mailing list.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > MVAPICH Team at OSU/NBCL
> > > > > >
> > > > > > ======================================================================
> > > > > > MVAPICH/MVAPICH2 project is currently supported with funding from
> > > > > > U.S. National Science Foundation, U.S. DOE Office of Science,
> > > > > > Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;
> > > > > > and with equipment support from Advanced Clustering, AMD, Apple,
> > > > > > Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm
> > > > > > and Sun Microsystems. Other technology partner includes Etnus.
> > > > > > ======================================================================
> > > > > >
> > > > > > _______________________________________________
> > > > > > openfabrics-ewg mailing list
> > > > > > openfabrics-ewg at openib.org
> > > > > > http://openib.org/mailman/listinfo/openfabrics-ewg
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > ---------------------------------
> > > > > > Sponsored Link
> > > > > >
> > > > > > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?
> > > > > > --0-119921422-1163376356=:19890
> > > > > > Content-Type: text/html; charset=iso-8859-1
> > > > > > Content-Transfer-Encoding: 8bit
> > > > > >
> > > > > > I am trying to use the OSC MPI tool for the iWARP and quite new to the open fabrics tools.
> > > >
> > > > If I know it correctly, iWARP is not yet part of the OFED release.
> > > > But the iWARP makefile has reference to ofed code. Is it really required.
> > > >
> > > > Is there any document describing the build process of the MVAPICH2 tool? I am going through the MVAPICH2 0.9.8 users guide and that does not seem to be giving me the detailed information.
> > > >
> > > > Can you please provide README files for the iWARP which describes TODO steps?
> > > >
> > > > Thanks,
> > > > David
> > > >
> > > > Dhabaleswar Panda
> > > wrote:
> > > > The MVAPICH team is pleased to announce the availability of MVAPICH2
> > > > 0.9.8 with the following NEW features:
> > > >
> > > > -
> > > > > > Checkpoint/Restart support for application transparent systems-level
> > > > fault tolerance. BLCR-based support using native InfiniBand Gen2
> > > > interface is provided. Flexible interface to work with different
> > > > file systems. Tested with ext3 (local disk), NFS and PVFS2.
> > > >
> > > > Performance of sample applications with checkpoint-restart using
> > > > PVFS2 and Lustre can be found here:
> > > >
> > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/performance/mvapich2/application/MVAPICH2-ckpt.html
> > > >
> > > > - iWARP support: Incorporates the support for OpenFabrics/Gen2-iWARP.
> > > > Tested with Chelsio T3 (10GigE) and Ammasso iWARP adapters and
> > > > drivers.
> > > >
> > > > - RDMA CM-based Connection management support
> > > >
> > > > - Shared memory optimizations for collective communication operations.
> > > > Efficient algorithms and optimizations for barrier, reduce and
> > > > all-reduce operations. Exploits the multi-core optimized shared
> > > > memory point-to-point communication support introduced in
> > > > > > MVAPICH2
> > > > 0.9.6.
> > > >
> > > > Performance of sample collective operations with this new feature
> > > > can be found here:
> > > >
> > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/perf-coll.html
> > > >
> > > > - uDAPL support for NetEffect 10GigE adapter. Tested with
> > > > NetEffect NE010 adapter.
> > > >
> > > > More details on all features and supported platforms can be obtained
> > > > by visiting the following URL:
> > > >
> > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/mvapich2_features.html
> > > >
> > > > MVAPICH2 0.9.8 release is tested with the latest OFED 1.1 stack. It
> > > > continues to deliver excellent performance. Sample performance
> > > > numbers include:
> > > >
> > > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR:
> > > > Two-sided operations:
> > > > - 2.81 microsec one-way latency (4 bytes)
> > > > - 1561 MB/sec unidirectional bandwidth
> > > > - 2935 MB/sec bidirectional bandwidth
> > > >
> > > > One-sided operations:
> > > > - 4.92 microsec Put latency
> > > > - 1569 MB/sec unidirectional Put bandwidth
> > > > - 2935 MB/sec
> > > > > > bidirectional Put bandwidth
> > > >
> > > > - OpenFabrics/Gen2 on EM64T dual-core with PCI-Ex and IBA-DDR (Dual-rail):
> > > > Two-sided operations:
> > > > - 2.81 microsec one-way latency (4 bytes)
> > > > - 3127 MB/sec unidirectional bandwidth
> > > > - 5917 MB/sec bidirectional bandwidth
> > > >
> > > > One-sided operations:
> > > > - 4.37 microsec Put latency
> > > > - 3137 MB/sec unidirectional Put bandwidth
> > > > - 5917 MB/sec bidirectional Put bandwidth
> > > >
> > > > - OpenFabrics/Gen2 on Opteron single-core with PCI-Ex and IBA-DDR:
> > > > Two-sided operations:
> > > > - 3.01 microsec one-way latency (4 bytes)
> > > > - 1402 MB/sec unidirectional bandwidth
> > > > - 2238 MB/sec bidirectional bandwidth
> > > >
> > > > One-sided operations:
> > > > - 4.65 microsec Put latency
> > > > - 1402 MB/sec unidirectional Put bandwidth
> > > > - 2238 MB/sec bidirectional Put bandwidth
> > > >
> > > > Performance numbers for all other platforms, system configurations and
> > > > operations can be viewed by visiting `Performance' section of the
> > > > project's web page.
> > > >
> > > > With the
> > > > > > ADI-3-level design, MVAPICH2 0.9.8 delivers similar
> > > > performance for two-sided operations compared to MVAPICH 0.9.8.
> > > > Organizations and users interested in getting the best performance for
> > > > both two-sided and one-sided operations and also want to exploit
> > > > advanced features (such as fault tolerance with checkpoint/restart,
> > > > iWARP, RDMA CM connection management, multi-threading, integrated
> > > > multi-rail, multi-core optimization, memory hook support and optimized
> > > > collectives) may migrate from MVAPICH code base to MVAPICH2 code base.
> > > >
> > > > For downloading MVAPICH2 0.9.8 package and accessing the anonymous
> > > > SVN, please visit the following URL:
> > > >
> > > > http://nowlab.cse.ohio-state.edu/projects/mpi-iba/
> > > >
> > > > A stripped down version of this release is also available at the
> > > > OpenFabrics SVN.
> > > >
> > > > All feedbacks, including bug reports and hints for performance tuning,
> > > > are welcome. Please post it to the mvapich-discuss mailing list.
> > > >
> > > > Thanks,
> > > > > >
> > > >
> > > > MVAPICH Team at OSU/NBCL
> > > >
> > > > ======================================================================
> > > > MVAPICH/MVAPICH2 project is currently supported with funding from
> > > > U.S. National Science Foundation, U.S. DOE Office of Science,
> > > > Mellanox, Intel, Cisco Systems, Sun Microsystems and Linux Networx;
> > > > and with equipment support from Advanced Clustering, AMD, Apple,
> > > > Appro, Dell, IBM, Intel, Mellanox, Microway, PathScale, SilverStorm
> > > > and Sun Microsystems. Other technology partner includes Etnus.
> > > > ======================================================================
> > > >
> > > > _______________________________________________
> > > > openfabrics-ewg mailing list
> > > > openfabrics-ewg at openib.org
> > > > http://openib.org/mailman/listinfo/openfabrics-ewg
> > > >
> > > >
> > > >
> > > > > >
> > > > > >
> > > > > >
> > > > ---------------------------------
> > > > Sponsored Link
> > > >
> > > >
> > > > > > Try Netflix today! With plans starting at only $5.99 a month what are you waiting for?
> > > > > > --0-119921422-1163376356=:19890--
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------
> > > > Sponsored Link
> > > >
> > > > $200,000 mortgage for $660/mo - 30/15 yr fixed, reduce debt, home equity - Click now for info
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------
> > > Want to start your own business? Learn how on Yahoo! Small Business.
> >
> >
> >
> >
> > ---------------------------------
> > Everyone is raving about the all-new Yahoo! Mail beta.
>
>
>
>
> ---------------------------------
> Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates.
---------------------------------
Sponsored Link
Degrees online in as fast as 1 Yr - MBA, Bachelor's, Master's, Associate - Click now to apply
---------------------------------
Access over 1 million songs - Yahoo! Music Unlimited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20061113/ccaec814/attachment-0001.html
More information about the mvapich-discuss
mailing list