[mvapich-discuss] LiMIC2/KNEM

[Dhabaleswar Panda]

Hi,

> The discussion raised a few thoughts/questions of my own:
>
> On Jul 12, 2013, at 8:34 AM, Mark Dixon <m.c.dixon at leeds.ac.uk> wrote:
>> ... it all depends on context, doesn't it? Some production clusters (of whatever size) can cope with vulnerabilities, some cannot. But is there a vulnerability here?
>
> I think a large mitigating factor is that nodes in these large limic 
> deployments are not shared; at worst, users would be able to shoot 
> themselves in the foot or cause kernel panics.

Thanks for providing your perspective here.

>> ...
>>> Recent kernels provide support for kernel-based memory copy directly. One such support is called Cross Memory Attach (CMA). Thus, you do not need any separate Kernel module. MVAPICH2 1.9 provides support for CMA, in addition to LiMIC2.
>> ...
>>
>> Brilliant! Hadn't heard of that: I will certainly look out for it hitting our preferred server OS :)
>
> It looks like CMA was included in 3.2 kernels.  From the MVAPICH2 
> release notes though, it wasn't clear to me: is there CMA support for 
> collective operations, or just point-to-point?  And are there any 
> performance comparison data for CMA vs. limic available to the public?

As you know, many of the collective operations are designed on top of 
pt-to-pt operations. So if CMA support is activated for intra-node 
operations, you will see benefits for collectives also (like LiMIC2). For 
LiMIC2, some additional collective-level optimizations and tunings are 
available in MVAPICH2 1.9. Similar support with CMA are currently being 
worked out and will be available in future MVAPICH2 releases.

As per the performance comparison between LiMIC2 and CMA, some studies are 
being carried out. It will be available in the near future.

Hope this helps.

Thanks,

DK

> Thanks
>
> Glenn
>
> --
> Glenn K. Lockwood, Ph.D.
> SDSC User Services
> glock at sdsc.edu
> (858) 246-1075
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>