[mvapich-discuss] problem w/MVAPICH in the frames of Gen1

[Mikhail Kuzminsky]

In message from Jimmy Tang <jtang at tchpc.tcd.ie> (Fri, 4 Aug 2006 
17:35:13 +0100):
>Hi,
>
>I think we had a similar problem to this a while ago, if I remember
>correctly, what we did was run this on all the nodes that wishes to
>participate in a calculation with mvapich...
>
>	sysctl -w vm.disable_cap_mlock=1
>
>Jimmy.

Jimmy,
thanks for your idea !

Unfortunately there is no /proc/sys/vm/disable_cap_mlock file
for my 2.4.21 SuSE kernel, and sysctl -w doesn't work :-(

Yours
Mikhail

>
>On Fri, Aug 04, 2006 at 08:05:58PM +0400, Mikhail Kuzminsky wrote:
>> In message from Sayantan Sur <surs at cse.ohio-state.edu> (Fri, 04 Aug 
>> 2006 09:20:56 -0500):
>> >Mikail,
>> <skipped>
>> 
>> >Thanks for trying out our latest version on your cluster. Clearly, 
>> >the problem in both versions stems from the inability of the VAPI 
>> >(underlying IB layer) to create a protection domain. The error 
>> >indicates that when MVAPICH calls the function VAPI_alloc_pd(), the 
>> >function doesn't return success.
>> 
>> :-(
>> 
>> >
>> >My hunch is that your IB installation is not proper. In particular, 
>> >the kernel modules which support IB, might not be working well with 
>> >your kernel. Could you please verify from Mellanox that the kernel 
>> >you are using is infact supported by IBGD?
>> 
>> IBGD-1.8.0 (I'm using) has 2.4.21 support (formally for Rocks 
>> distribution, but I use SuSE). IBGD-1.8.2 works officially only 
>>w/2.6. 
>> 
>> 
>> Some time ago I upgraded from 1.6.1 (w/working MPI) to 1.8.0 
>> just by Mellanox staff reccomendation, and TCP/IP stack w/SDP works 
>>in 
>> 1.8.0 OK. 
>> 
>> >Pasha, any thoughts?
>> >
>> >You may also try to run some benchmarks which use VAPI only, like 
>> >`perf_main' to check if they have the same error too.
>> 
>> No, perf_main for bandwith (-trc) gives normal 791.4 Mbytes/s.
>> 
>> Yours
>> Mikhail
>> 
>> 
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at mail.cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>> 
>> 
>> ** ACCEPT: CRM114 PASS osb unique microgroom Matcher ** 
>> CLASSIFY succeeds; success probability: 1.0000  pR: 10.1903
>> Best match to file #0 (/u1/trhpc/jtang/.crm114/nonspam.css) prob: 
>>1.0000  
>> pR: 10.1903  Total features in input file: 4504
>> #0 (/u1/trhpc/jtang/.crm114/nonspam.css): features: 88296, hits: 
>>314418, 
>> prob: 1.00e-00, pR:  10.19 #1 (/u1/trhpc/jtang/.crm114/spam.css): 
>>features: 
>> 125984, hits: 199700, prob: 6.45e-11, pR: -10.19 
>---end quoted text---
>
>-- 
>Jimmy Tang
>Trinity Centre for High Performance Computing,
>Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
>http://www.tchpc.tcd.ie/