[mvapich-discuss] regestration cache limits in mvapich 0.9.7

[Sayantan Sur]

Hello Pasha,

Pavel Shamis (Pasha) wrote:

> Hi,
> In mvapich-0.9.7 you have viadev_dreg_cache_limit limit that should 
> limit the size of registration cache. From review in dreg.* stuff I see
> that actually this parameter limits only the maximum number of pages 
> per _single_ buffer registration (see dreg_new_entry()) and not total 
> number of registered pages in the cache.  Can you please review this 
> point ? Because I'm not sure that we want limit # of pages per single 
> buffer registration.

Thanks for looking at the code in detail. Here's my explanation of the code:

1) dreg cache interface is called using dreg_register()
2) If no cache entry is found, dreg_new_entry() is called
3) dreg_new_entry() checks if the single registration size itself will 
violate the cache limit. It doesn't allow a single registration to go 
past the user defined limit.
4) If it is found that the single registration is below the user defined 
limit, then the flow proceeds to dreg_insert()
5) In dreg_insert(), we call vma_new() to insert a new virtual memory region
6) In vma_new() the number of pages requested is added to a variable 
called `pinned_pages_count'. If this pinned_pages_count exceeds the user 
set limit `viadev_dreg_cache_limit', then NULL is returned and this is 
propagated to the upper layer to report a failed registration. If not, 
then the `pinned_pages_count' variable is incremented accordingly.
7) `pinned_pages_count' is decremented when a memory area is removed 
using vma_remove()

I hope this helps.

Thanks,
Sayantan.

>
> Thanks,
> Pavel Shamis (Pasha)
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss



-- 
http://www.cse.ohio-state.edu/~surs