[mvapich-discuss] building mvapich2 on a large SMP

[Hari Subramoni]

Hello Dr. Sankaran,

There are no maximum limits set for the number of ranks that can be run on
a node.

If you do not want to install OFED libraries, then yes,
setting --with-device=ch3:sock during configuring will help you avoid the
issues you are seeing currently. However, we would strongly recommend
against doing this. The default CH3 channel in MVAPICH2 has
several optimized designs for intra-node (intra-socket and inter-socket)
pt-to-pt and collectives. These will not be available if Ch3:sock is used.

Regards,
Hari.

On Fri, Oct 3, 2014 at 12:05 AM, Sankaran, Logan (Strategic Solutions
Architecure (SSA)) <logan.sankaran at hp.com> wrote:

>  Hello Hari,
>
>
>
> Thanks for your prompt response. I have asked our system admin to install
> OFED (it is not installed now).  I have two more questions to clarify:
>
> 1.      What is the largest no. of ranks one can run on a single server
> ?  Is there a maximum set in the source ?
>
> 2.      Will setting --with-device=ch3:sock during configuring help for a
> SMP build ?
>
> Thanks and Regards,
>
> Logan
>
> *From:* hari.subramoni at gmail.com [mailto:hari.subramoni at gmail.com] *On
> Behalf Of *Hari Subramoni
> *Sent:* Thursday, October 02, 2014 6:30 PM
> *To:* Sankaran, Logan (Strategic Solutions Architecure (SSA))
> *Cc:* mvapich-discuss at cse.ohio-state.edu
> *Subject:* Re: [mvapich-discuss] building mvapich2 on a large SMP
>
>
>
> Hi Dr. Sankaran,
>
> The only requirement for building MVAPICH is that the system should have
> OFED libraries installed. As long as the OFED libraries are present, you
> should be able to compile and build MVAPICH2. Could you please install OFED
> libraries and retry? The following section of the userguide gives some more
> information about specific config options required if OFED libraries are in
> non standard locations
>
>
> http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.1a-userguide.html#x1-140004.6
>
> To the best of our knowledge there should not be any other issues that
> will prevent you from running MVAPICH2 on this system. However, please note
> that we have not optimized MVAPICH2 for performance or memory footprint on
> such a large system before. So please let us know if you have any such
> issues and we will be glad to work with you.
>
>
>
> Regards,
>
> Hari.
>
>
>
> On Thu, Oct 2, 2014 at 4:23 PM, Sankaran, Logan (Strategic Solutions
> Architecure (SSA)) <logan.sankaran at hp.com> wrote:
>
>  Hello,
>
>
>
> I am trying to build the latest version, 2.1a on a large SMP.  I am
> getting errors such as:
>
> configure: error: 'libibverbs not found. Did you specify
> --with-ib-libpath=?'
>
>
>
> Tried --without-ibverbs-lib and it doesn’t help.
>
>
>
> Could you recommend configure options for a large SMP with 500 cores ?  Is
> there limit on the no. of ranks you can build and run on an SMP ?  How to
> increase this number ?
>
> Thanks.
>
> Logan
>
> Logan Sankaran Ph.D.
>
> HPC Strategist
>
> *S*trategic *S*olutions *A*rchitecture (*SSA*)
>
> *Hewlett-Packard Company *
>
> Building/Floor/Post:    H4-GF-36
>
> *5400  Legacy Drive*
>
> *PLANO, Texas 75024*
>
> *USA*
>
>
>
> Ph:   *972–605-5714 <972%E2%80%93605-5714>*
>
>  logan.sankaran at hp.com
>
>
>
> Passion for Performance
>
>
>
> *Engage our services by: *
>
> *Web:         **http://engage.presales.hp.com*
> <http://engage.presales.hp.com/> (select STS/Solution Center Team)
>
> *E-mail*:         STS.central at hp.com
>
>
>
> HP and SSA – The Difference for Success
>
>
>
> [image: email_footer_tree_text]
>
>
>
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20141004/69d06ac9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 1814 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20141004/69d06ac9/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 2310 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20141004/69d06ac9/attachment-0001.gif>