[mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

Sat Aug 2 23:21:07 EDT 2014

Just to clarify.  CMA support is enabled by default.  If it is detected
at runtime that the underlying kernel does not support it, the following
message is printed:

    CMA is not available. Set MV2_SMP_USE_CMA=0 to disable CMA.

Based on your mail I believe this is the message that you've
encountered.  Is this correct?

On Sat, Aug 02, 2014 at 10:19:27PM -0400, Novosielski, Ryan wrote:
> Thanks. I guess I just assumed that there were some processor requirements too. 
> 
> And actually, I built 2.0 GA recently and it enabled CMA even though my kernel did not support it. ;)
> 
> --
> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
>  || \\UTGERS      |---------------------*O*---------------------
>  ||_// Biomedical | Ryan Novosielski - Senior Technologist
>  || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
>  ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
>       `'
> ________________________________________
> From: Jerome Vienne [viennej at tacc.utexas.edu]
> Sent: Saturday, August 02, 2014 7:29 PM
> To: Novosielski, Ryan
> Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
> 
> CMA is a kernel feature that only require a recent kernel version. It has been introduced with Kernel 3.2.
> If you want more details, I recommend you to read the following paper:
> http://dl.acm.org/citation.cfm?id=2616532
> Please note that MVAPICH2 2.0 GA automatically enable CMA if your kernel support it. For 1.9 and previous 2.0 versions, you will need to use a configuration flag.
> 
> Thanks,
> Jerome
> 
> 
> On Sat, Aug 2, 2014 at 12:24 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>> wrote:
> Can anyone briefly or generally summarize the hardware requirements to take advantage of this (minimum chipset, for example)?
> 
> 
> ----- Original Message -----
> From: Panda, Dhabaleswar [mailto:panda at cse.ohio-state.edu<mailto:panda at cse.ohio-state.edu>]
> Sent: Saturday, August 02, 2014 11:36 AM
> To: Mehmet Belgin <mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>>; mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu> <mvapich-discuss at mailman.cse.ohio-state.edu<mailto:mvapich-discuss at mailman.cse.ohio-state.edu>>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu> <viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>>
> Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
> 
> Hi Mehmet,
> 
> Thanks for your note. CMA support has been available in MVAPICH2 since 1.9 version. Prior to this support, equivalent
> kernel-based support (LiMIC2 module) has been there in MVAPICH2 for many years.  In the MVAPICH2 2.0 GA-version,
> CMA support has been made default if the underlying kernel supports it. The GA version also has the flexibility
> for LiMIC2 support.
> 
> We strongly suggest users to use CMA/LiMIC2 support to extract maximum intra-node communication
> performance with MVAPICH2 on modern multi-core platforms.
> 
> Thanks,
> 
> DK
> 
> ________________________________________
> From: mvapich-discuss-bounces at cse.ohio-state.edu<mailto:mvapich-discuss-bounces at cse.ohio-state.edu> on behalf of Mehmet Belgin [mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>]
> Sent: Saturday, August 02, 2014 11:11 AM
> To: mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>
> Subject: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)
> 
> Hi Everyone,
> 
> I stumbled upon a poster (by Jerome Vienne) during the XSEDE14
> conference, which explained CMA-related improvements with the latest
> version of MVAPICH2 . It looked too good to be true, so I gave it a shot
> myself on our 64core Interlagos nodes. The latency and bandwidth
> improvements I observed also confirm the significant improvements over
> rc1 (my findings are attached, sorry for the ugly format). The results
> are mind blowing, especially given that they are available to users with
> no extra effort (no special kernel modules, extra runtime parameters, etc).
> 
> For once, I wanted to write the list to NOT complain about bugs or
> problems, but rather appreciate the good work the developers are doing.
> I hope this encourages others who still have not given the latest
> version a try to do so :)
> 
> Our big applause goes to all the MVAPICH2 developers, and please keep up
> the good work!
> 
>   -Mehmet
> 
> --
> =========================================
> Mehmet Belgin, Ph.D. (mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>)
> Scientific Computing Consultant | OIT - Academic and Research Technologies
> Georgia Institute of Technology
> 258 Fourth Street, Rich Building, Room 326
> Atlanta, GA  30332-0700
> Office: (404) 385-0665
> 
> 
> 
> 
> --
> Jerome Vienne, Ph.D
> Research Associate
> Texas Advanced Computing Center
> The University of Texas at Austin
> (512) 475-9322
> viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>
> 
> 
> 
> 
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss

-- 
Jonathan Perkins