[mvapich-discuss] MVAPICH and HugeTLB

[Matthew Koop]

Andrey,

As you found, for optimal performance MVAPICH caches memory registrations.
Unfortunately, due to inherent issues in current InfiniBand drivers, to do
this properly we need to override malloc and free.

MVAPICH currently will automatically turn off the registration cache if
other malloc hooks are detected. We have found this to be sufficient for
other applications that override malloc. Does simply turning off the
registration cache (VIADEV_USE_DREG_CACHE=0) work for you and not
compiling with PTMALLOC off?

If not, can you send more information on your compilation parameters for
the applications as well as MPI library?

Thanks,

Matt

On Mon, 2 Jul 2007, Andrey Slepuhin wrote:

> Dear folks,
>
> I am playing with libhugetlb to see the performance difference with
> different application, but in order get it work correctly I need
> 1) to disable PTMALLOC (otherwise malloc hooks from libhugetlb do not
> work and the application uses standard pages)
> 2) to turn off memory registration cache (otherwise I got incorrect
> results or application simpy fails)
> Does anybody else tried libhugetlb with MVAPICH? Are there any plans to
> implement "transparent" support for libhugetlb that will not require any
> special configuration for MVAPICH?
>
> Thanks,
> Andrey
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>