[mvapich-discuss] mvapich2 installation with BLCR support

[wei huang]

Hi Yiannis,

You need to increase the memlock limit (the size of memory that a user
level program is allowed to lock) on your system. Please refer to Section
8.2.4 in our user guide to change it:

http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2.html#x1-480008.2.4

Let us know if it works for you.

Thanks.

Regards,
Wei Huang

774 Dreese Lab, 2015 Neil Ave,
Dept. of Computer Science and Engineering
Ohio State University
OH 43210
Tel: (614)292-8501


On Tue, 29 Apr 2008, yiannis georgiou wrote:

> Hello and thanks for the answer it helped a lot!!!
>
> I finally installed the ibverbs libraries using the debian packages
> and ibumad using the source files from openfabrics...
> I also had to put --disable-f77 --disable-f90 for the configure into
> the make.mvapich2.ofa script...
>
> The installation finished succesfully!!
>
> I now have another problem...:
>
> After the initialization phase I try to run the test ./cpi and I get
> the following errors:
>
> -----------------
> g5k at bordeplage-9:~/mvapich2-1.0.2/examples$ mpdboot -n 2 -f ~/nodes
> g5k at bordeplage-9:~/mvapich2-1.0.2/examples$ mpdtrace
> bordeplage-9
> bordeplage-7
> g5k at bordeplage-9:~/mvapich2-1.0.2/examples$
> /usr/local/mvapich2/bin/mpirun -machinefile ~/nodes -n 2 ./cpi
> libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
>      This will severely limit memory registrations.
> Fatal error in MPI_Init:
> Other MPI error, error stack:
> MPIR_Init_thread(259)...........: Initialization failed
> MPID_Init(102)..................: channel initialization failed
> MPIDI_CH3_Init(178).............:
> MPIDI_CH3I_CM_Init(878).........: rdma_get_control_parameters
> rdma_get_control_parameters(434):
> rdma_open_hca(382)..............: Failed to open HCA number 0
> rank 1 in job 1  bordeplage-9.bordeaux.grid5000.fr_34110   caused
> collective abort of all ranks
>    exit status of rank 1: return code 1
> ----------------
>
> I also try to run another program previously compiled using
> /usr/local/mvapich2/bin/mpicc
>
> and I get the following:
>
> ----------------
> g5k at bordeplage-9:~$ /usr/local/mvapich2/bin/mpirun -machinefile
> ~/nodes -n 2 BENCHS/esp-2.1.1/src-mvapich2/pchksum -t 5
> Attempting to use an MPI routine before initializing MPICH
> Attempting to use an MPI routine before initializing MPICH
>
> ----------------
>
> Any ideas where those problems may come from?
>
> Regards,
> yiannis
>
>
> Quoting "Jonathan L. Perkins" <perkinjo at cse.ohio-state.edu>:
>
> > On Mon, Apr 28, 2008 at 05:20:55PM +0200, Yiannis Georgiou wrote:
> >> Hello,
> >>
> >> I'm trying to install MVAPICH2 with BLCR support upon an infiniband cluster
> >> and I would like to ask if it is possible to do this installation wihout
> >> the OFED packages installed?
> >
> > Yes, if you have the corresponding packages installed from other sources.
> >
> >>
> >> Some details for the system:
> >>
> >> Debian kernel 2.6.22-3-amd64
> >> gcc version 4.2.3 (Debian 4.2.3-2)
> >> InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor
> >> compatibility mode) (rev a0)
> >> ib_mthca: Mellanox InfiniBand HCA driver v0.08 (February 14, 2006)
> >> blcr version 0.6.5
> >>
> >> I am trying for mvapich2 version 1.0.2p1
> >>
> >> The problem is that I cannot install successfully the official OFED
> >> distribution, needed to use the script "make.mvapich2.ofa" ....but I think
> >> that a lot of the OFED packages can be found as debian packages....That's
> >> why I would like to ask which are the OFED packages that are needed for the
> >> installation and if there is a different procedure that I could follow
> >> other than changing the script "make.mvapich2.ofa" to point to the correct
> >> values, for the installation-configuration....
> >
> > You'll need to have ibumad and ibverbs installed.  Of course you'll also need
> > the BLCR toolkit installed.  Please see section 4.4.1 of our
> > userguide for more
> > information in regard to building MVAPICH2.
> >
> > http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2.html#x1-100004.4.1
> >
> > Hope this helps.
> >
> >>
> >> Thank you in advance for your help...
> >> Regards,
> >>
> >> yiannis
> >>
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Yiannis Georgiou                LIG Laboratory / MESCAL Project
> >> Yiannis.Georgiou at imag.fr        http://mescal.imag.fr/ +33 (0)4.76.61.20.33
> >>            FRANCE
> >> _______________________________________________
> >> mvapich-discuss mailing list
> >> mvapich-discuss at cse.ohio-state.edu
> >> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
> >
> > --
> > Jonathan Perkins
> > http://www.cse.ohio-state.edu/~perkinjo
> > _______________________________________________
> > mvapich-discuss mailing list
> > mvapich-discuss at cse.ohio-state.edu
> > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
> >
>
>
>
> --
>
> Yiannis Georgiou                LIG Laboratory / MESCAL Project
> Yiannis.Georgiou at imag.fr        http://mescal.imag.fr/
> +33 (0)4.76.61.20.33            FRANCE
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>