[mvapich-discuss] No execution with mvapich-gen2

[Owen Stampflee]

Sorry for the lenghty delay, here's where I'm at currently:

On Tue, 2006-02-21 at 23:14 -0500, Sayantan Sur wrote:
> Owen,
> 
> Thanks for trying out mvapich-gen2. Sorry to know about your problems.
> Hopefully, we can resolve this issue quickly.
> 
> > I get the following output running the cpi example...
> > 
> > [root at m2 examples]# mpirun_rsh -np 1 localhost ./cpi
> > mpirun: executable version 0 does not match our version 2.
> > done.
> 
> Could you tell us what happens if you do:
> 
> $ mpirun_rsh -np 1 m2 ./cpi
Same result, application doesnt run.

> > I'm using openib svn5411, and mvapich-gen2-1.0 with the 101, 104, 105,
> > 106. Oddly enough even with the recent 5411 version of openib, patch 103
> > (CQ creation) doesnt compile.
> 
> If you've been following the OpenIB mailing list, then you must be aware
> of this. Sometime last October, the ibv_create_cq (which is the Gen2
> interface) verb arguments changed. To work around this interface change,
> we introduced patch #103 which uses the new verb by DEFAULT.
> 
> So, if you are at patch level 106, then you do NOT need to specify
> -DGEN2_OLD_CQ_VERB. Just using the default mvapich.make.gcc should be
> enough.
> 
> Just for clarification, can you send us the compilation failure you get
> with patch #106? Also, if you just download the integrated tarball from
> the mvapich-gen2 download page (instead of applying all patches by
> hand), do you still get the same results?
I'll follow this up in a 2nd email about my mvapich-0.97 compile results.

> > I'm building everything in 32-bit mode, and using -D_IA32_ (those look
> > fairly sane but I could have missed something).
> > 
> > All the OpenIB pingpong tests are fine, I'm really quite stumped on
> > where to go from here.
> 
> Gen2 uses the lockable memory limits set by the system administrator. In
> order to use MVAPICH, you must set this parameter to `unlimited' or to a
> larger memory size so that MVAPICH is able to register communication
> buffers. This is common of all MPI and other higher level software on
> top of Gen2.
> 
> There are three steps to setting up the lockable memory privileges for
> users:
> 
> 1) In /etc/security/limits.conf: Add a line
> 
> *               soft    memlock         unlimited
> 
> 2) In /etc/init.d/sshd: Add a line
> 
> ulimit -l unlimited
> 
> 3) Restart sshd
> 
> /etc/init.d/sshd restart
> 
> All subsequent SSH sessions by users should have this new lockable
> memory limit set. To verify this, you can do:
> 
> $ ssh node1 ulimit -l
> 
> If this shows unlimited, then the setup was OK.
> 
> Please let us know if this was able to resolve your problems.
Tried this as well, same result... application doesnt run. I'm currently attempting to get OpenIB 1.0rc1 running as well mvapich-0.97

Cheers,
Owen