[mvapich-discuss] MVAPICH batch Integration Qs

[Michael E. Thomadakis]

Hello,

we are trying to deploy MVAPICH2 as one of our main MPI stacks on a new 
iDataPlex cluster at Texas A&M University (324 2 socket nehalem @2.8GHz, 
with 24GiB/DRAM + QDR FBB interconnect).

1) MVAPICH2 and PBS Integration

We are using currently TORQUE/Maui scheduler for the batch MPI jobs. I 
was wondering if MVAPICH2 integrates well with TORUQ/Maui so that the 
job's resources (tasks , memory, etc) can be tracked and monitored for 
batch job resource limit enforcement. Torque uses the Task Management 
(TM) API to submit jobs as opposed to say PMI that MPICH2 uses. 
Unfortunately, TORQUE cannot interoperate well with PMI interface as it 
does not know which processes correspond to a particular MPI job so it 
cannot track resource consumption (cpu time, memory / process and total 
per job) and thus it cannot kill jobs exceeding their requested limits. 
It cannot also suspend / resume an MPI job since it does not know of the 
participant processes.

Can MVAPICH2 integrate weel with TORQUE?

2) Does MVAPICH2 help the scheduler place tasks on cores ? Or can we 
request / specify explicit placement or style of placement via MVAPICH2 
job launcher to the TORQUE system?

3) Does MVAPICH2 automatically use shared memory IPC for intra-node MPI 
tasks and IB for inter-node ones? Can I specify say DAPL or OFED verbs 
libs to use?

4) Which OFED is recommended for your latest MVAPICH2

5) Any idea when PVAPICH2 1.5 will be out ?

Thank you much ......

Michael

-- 
% -------------------------------------------------------------------- \
% Michael E. Thomadakis, Ph.D.  Senior Lead Supercomputer Engineer/Res \
% E-mail: miket AT tamu DOT edu                   Texas A&M University \
% web:    http://alphamike.tamu.edu              Supercomputing Center \
% Voice:  979-862-3931                    Teague Research Center, 104B \
% FAX:    979-847-8643                  College Station, TX 77843, USA \
% -------------------------------------------------------------------- \

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20100521/f2ad9a6b/attachment.html