[mvapich-discuss] Are mallopt calls needed here

[Subramoni, Hari]

Hi, Mark.

No, the mallopt calls are not strictly necessary here.

However, the issue with registration cache is quite surprising. We have not had any of our users report such a behavior before. Would it be possible to have a reproducer so that we can attempt to reproduce it locally? We would like to identify and fix the issue.

Thx,
Hari.

-----Original Message-----
From: mvapich-discuss <mvapich-discuss-bounces at cse.ohio-state.edu> On Behalf Of Mark Beall
Sent: Tuesday, January 1, 2019 10:34 AM
To: mvapich-discuss at cse.ohio-state.edu <mvapich-discuss at mailman.cse.ohio-state.edu>
Subject: [mvapich-discuss] Are mallopt calls needed here

Hi,

In ch3_init.c (and similarly in ib_init.c) there is the following code:

#if !defined(DISABLE_PTMALLOC)
        if (mvapich2_minit()) {
            if (pg_rank == 0) {
                MPL_error_printf("WARNING: Error in initializing MVAPICH2 ptmalloc library."
                "Continuing without InfiniBand registration cache support.\n");
            }
            mv2_MPIDI_CH3I_RDMA_Process.has_lazy_mem_unregister = 0;
        }
#else /* !defined(DISABLE_PTMALLOC) */
        mallopt(M_TRIM_THRESHOLD, -1);
        mallopt(M_MMAP_MAX, 0);
        mv2_MPIDI_CH3I_RDMA_Process.has_lazy_mem_unregister = 0; #endif /* !defined(DISABLE_PTMALLOC) */


The question is whether it is ok to not have the mallopt() calls there. 

Presumably everything works without them since mallopt() is not called if mvapich2_minit() returns an error.

We must build with -–disable-registration-cache to disable ptmalloc since it causes one part of our code to take 10 hours vs 30 seconds (I’m not exaggerating).

We would also rather that the mallopt call wasn’t here since it causes undesirable behavior for our application. 

I’m guessing that at one point, DISABLE_PTMALLOC didn’t also imply disabling the registration cache (and then would need the mallopt() calls), now it does (since it sets has_lazy_mem_unregister = 0) and thus they really aren’t needed anymore.

Thanks,

mark


_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss