[mvapich-discuss] problem w/MVAPICH in the frames of Gen1

[Sayantan Sur]

Hi Mikhail,

> I applied strace output in my 1st message here,
> and now (below) apply strace for more simple perf_main: as I wrote in 
> previous message, the problem isn't only for mvapich.

Thanks for verifying all the details and posting the strace output. Yes, 
it seems that the problem stems from something else with system and VAPI 
libraries than just MVAPICH.

Observing this run for `perf_main' with strace,

> strace on server part (strace perf_main -trc ...)           
> -------------------------------------------------------------
>
[...]

> ioctl(6, 0x80287801, 0x7fbfffe800)      = 0
> mlock(0x627000, 2150135809)             = -1 EPERM (Operation not 
> permitted)
> write(1, "Error: Allocating PD : Invalid V"..., 47Error: Allocating PD 
> : Invalid Virtual Address) = 47
> --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> +++ killed by SIGSEGV +++


I am particularly disturbed noticing that the mlock() call requires 
locking 2150135809 bytes of memory. That's almost 2GB! Going back a 
little in this thread, it seems that you have given max lockable memory 
to 1.95GB memory with ssh and only 0.8GB with rsh. Ofcourse, the mlock() 
will fail :-)

I think there are two questions now:

1) Can you run perf_main as user if you give mlock() permissions for 
more than 2GB memory (provided the system has that much)?
2) Why is perf_main requiring 2GB to be locked? My hunch is that there 
is some build or other system issues which results in such behavior. On 
our systems, definitely perf_main does not require that much memory.

Thanks,
Sayantan.

-- 
http://www.cse.ohio-state.edu/~surs