[mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

[Novosielski, Ryan]

Thanks. I guess I just assumed that there were some processor requirements too. 

And actually, I built 2.0 GA recently and it enabled CMA even though my kernel did not support it. ;)

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
 || \\UTGERS      |---------------------*O*---------------------
 ||_// Biomedical | Ryan Novosielski - Senior Technologist
 || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
 ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
      `'
________________________________________
From: Jerome Vienne [viennej at tacc.utexas.edu]
Sent: Saturday, August 02, 2014 7:29 PM
To: Novosielski, Ryan
Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

CMA is a kernel feature that only require a recent kernel version. It has been introduced with Kernel 3.2.
If you want more details, I recommend you to read the following paper:
http://dl.acm.org/citation.cfm?id=2616532
Please note that MVAPICH2 2.0 GA automatically enable CMA if your kernel support it. For 1.9 and previous 2.0 versions, you will need to use a configuration flag.

Thanks,
Jerome


On Sat, Aug 2, 2014 at 12:24 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>> wrote:
Can anyone briefly or generally summarize the hardware requirements to take advantage of this (minimum chipset, for example)?


----- Original Message -----
From: Panda, Dhabaleswar [mailto:panda at cse.ohio-state.edu<mailto:panda at cse.ohio-state.edu>]
Sent: Saturday, August 02, 2014 11:36 AM
To: Mehmet Belgin <mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>>; mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu> <mvapich-discuss at mailman.cse.ohio-state.edu<mailto:mvapich-discuss at mailman.cse.ohio-state.edu>>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu> <viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>>
Subject: Re: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

Hi Mehmet,

Thanks for your note. CMA support has been available in MVAPICH2 since 1.9 version. Prior to this support, equivalent
kernel-based support (LiMIC2 module) has been there in MVAPICH2 for many years.  In the MVAPICH2 2.0 GA-version,
CMA support has been made default if the underlying kernel supports it. The GA version also has the flexibility
for LiMIC2 support.

We strongly suggest users to use CMA/LiMIC2 support to extract maximum intra-node communication
performance with MVAPICH2 on modern multi-core platforms.

Thanks,

DK

________________________________________
From: mvapich-discuss-bounces at cse.ohio-state.edu<mailto:mvapich-discuss-bounces at cse.ohio-state.edu> on behalf of Mehmet Belgin [mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>]
Sent: Saturday, August 02, 2014 11:11 AM
To: mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>; viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>
Subject: [mvapich-discuss] Enjoying the CMA-aware mvapich2 :)

Hi Everyone,

I stumbled upon a poster (by Jerome Vienne) during the XSEDE14
conference, which explained CMA-related improvements with the latest
version of MVAPICH2 . It looked too good to be true, so I gave it a shot
myself on our 64core Interlagos nodes. The latency and bandwidth
improvements I observed also confirm the significant improvements over
rc1 (my findings are attached, sorry for the ugly format). The results
are mind blowing, especially given that they are available to users with
no extra effort (no special kernel modules, extra runtime parameters, etc).

For once, I wanted to write the list to NOT complain about bugs or
problems, but rather appreciate the good work the developers are doing.
I hope this encourages others who still have not given the latest
version a try to do so :)

Our big applause goes to all the MVAPICH2 developers, and please keep up
the good work!

  -Mehmet

--
=========================================
Mehmet Belgin, Ph.D. (mehmet.belgin at oit.gatech.edu<mailto:mehmet.belgin at oit.gatech.edu>)
Scientific Computing Consultant | OIT - Academic and Research Technologies
Georgia Institute of Technology
258 Fourth Street, Rich Building, Room 326
Atlanta, GA  30332-0700
Office: (404) 385-0665




--
Jerome Vienne, Ph.D
Research Associate
Texas Advanced Computing Center
The University of Texas at Austin
(512) 475-9322
viennej at tacc.utexas.edu<mailto:viennej at tacc.utexas.edu>