[mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

[Novosielski, Ryan]

OK, that's what I thought. I just wanted to make sure everyone was on the same page. Yes, that's the error. No big deal -- I recompiled it and it went away, I just thought there was some confusion around whether it is checked for by configure vs. at runtime. Thanks for clearing that up.
--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
 || \\UTGERS      |---------------------*O*---------------------
 ||_// Biomedical | Ryan Novosielski - Senior Technologist
 || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
 ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
      `'
________________________________________
From: perkinjo at cse.ohio-state.edu [perkinjo at cse.ohio-state.edu]
Sent: Saturday, August 02, 2014 11:21 PM
To: Novosielski, Ryan
Cc: Jerome Vienne; mvapich-discuss at mailman.cse.ohio-state.edu
Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

Just to clarify.  CMA support is enabled by default.  If it is detected
at runtime that the underlying kernel does not support it, the following
message is printed:

    CMA is not available. Set MV2_SMP_USE_CMA=0 to disable CMA.

Based on your mail I believe this is the message that you've
encountered.  Is this correct?

On Sat, Aug 02, 2014 at 10:19:27PM -0400, Novosielski, Ryan wrote:
> Thanks. I guess I just assumed that there were some processor requirements too.
>
> And actually, I built 2.0 GA recently and it enabled CMA even though my kernel did not support it. ;)
>
> --
> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
>  || \\UTGERS      |---------------------*O*---------------------
>  ||_// Biomedical | Ryan Novosielski - Senior Technologist
>  || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
>  ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
>       `'
> ________________________________________
> From: Jerome Vienne [viennej at tacc.utexas.edu]
> Sent: Saturday, August 02, 2014 7:29 PM
> To: Novosielski, Ryan
> Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
>
> CMA is a kernel feature that only require a recent kernel version. It has been introduced with Kernel 3.2.
> If you want more details, I recommend you to read the following paper:
> http://dl.acm.org/citation.cfm?id=2616532
> Please note that MVAPICH2 2.0 GA automatically enable CMA if your kernel support it. For 1.9 and previous 2.0 versions, you will need to use a configuration flag.
>
> Thanks,
> Jerome
>
>
> On Sat, Aug 2, 2014 at 12:24 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>> wrote:
> Can anyone briefly or generally summarize the hardware requirements to take advantage of this (minimum chipset, for example)?
>
>
> ----- Original Message -----
> From: Panda, Dhabaleswar [mailto:panda at cse.ohio-state.edu<mailto:panda at cse.ohio-state.edu>]
> Sent: Saturday, August 02, 2014 11:36 AM
> To: Mehmet Belgin <mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>>; mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu> <mvapich-discuss at mailman.cse.ohio-state.edu<mailto:mvapich-discuss at mailman.cse.ohio-state.edu>>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu> <viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>>
> Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
>
> Hi Mehmet,
>
> Thanks for your note. CMA support has been available in MVAPICH2 since 1.9 version. Prior to this support, equivalent
> kernel-based support (LiMIC2 module) has been there in MVAPICH2 for many years.  In the MVAPICH2 2.0 GA-version,
> CMA support has been made default if the underlying kernel supports it. The GA version also has the flexibility
> for LiMIC2 support.
>
> We strongly suggest users to use CMA/LiMIC2 support to extract maximum intra-node communication
> performance with MVAPICH2 on modern multi-core platforms.
>
> Thanks,
>
> DK
>
> ________________________________________
> From: mvapich-discuss-bounces at cse.ohio-state.edu<mailto:mvapich-discuss-bounces at cse.ohio-state.edu> on behalf of Mehmet Belgin [mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>]
> Sent: Saturday, August 02, 2014 11:11 AM
> To: mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>
> Subject: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
>
> Hi Everyone,
>
> I stumbled upon a poster (by Jerome Vienne) during the XSEDE14
> conference, which explained CMA-related improvements with the latest
> version of MVAPICH2 . It looked too good to be true, so I gave it a shot
> myself on our 64core Interlagos nodes. The latency and bandwidth
> improvements I observed also confirm the significant improvements over
> rc1 (my findings are attached, sorry for the ugly format). The results
> are mind blowing, especially given that they are available to users with
> no extra effort (no special kernel modules, extra runtime parameters, etc).
>
> For once, I wanted to write the list to NOT complain about bugs or
> problems, but rather appreciate the good work the developers are doing.
> I hope this encourages others who still have not given the latest
> version a try to do so :)
>
> Our big applause goes to all the MVAPICH2 developers, and please keep up
> the good work!
>
>   -Mehmet
>
> --
> =========================================
> Mehmet Belgin, Ph.D. (mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>)
> Scientific Computing Consultant | OIT - Academic and Research Technologies
> Georgia Institute of Technology
> 258 Fourth Street, Rich Building, Room 326
> Atlanta, GA  30332-0700
> Office: (404) 385-0665
>
>
>
>
> --
> Jerome Vienne, Ph.D
> Research Associate
> Texas Advanced Computing Center
> The University of Texas at Austin
> (512) 475-9322
> viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>
>
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss

--
Jonathan Perkins