[mvapich-discuss] PBS Pro's qstat failed to show "Time Use" if building application with mvapich2

[Jonathan Perkins]

Thanks for providing the info.  There aren't any additional options
you can use to make this work except for possibly replacing the
rsh/ssh command used by mpirun_rsh and trying that.

Is there something like pbsdsh under /opt/pbs/bin?  If so you can try
rebuilding mvapich2 by setting RSH_CMD to that when configuring and
using mpirun_rsh in your submit scripts.

On Sat, Dec 21, 2013 at 9:15 PM, Jianyu Liu <jerry_leo at msn.com> wrote:
> HI,
>
> Here are the output of 'mpiname -a'
>
> MVAPICH2 2.0b Fri Nov  8 11:17:40 EST 2013 ch3:mrail
>
> Compilation
> CC: pgcc    -DNDEBUG -DNVALGRIND -O2
> CXX: pgCC   -DNDEBUG -DNVALGRIND
> F77: pgf77 -L/usr/lib64 -L/lib -L/lib
> FC: pgf90
>
> Configuration
> --prefix=/nuist/p/public/app/mvapich2/2.0b/pgi -with-ib-libpath=/usr/lib64
> --with-ib-include=/usr/include --enable-f77 --enable-fc CC=pgcc CXX=pgCC
> F77=pgf77 FC=pgf90
>
>
> These is no rsh in /opt/pbs/bin.
>
> I submit my job with mpirun (in /nuist/p/public/app/mvapich2/2.0b/pgi/bin),
> just like this
>
>     mpirun -np $nprocs ./wrf.exe
>
>
> Jerry
>
>> Date: Sat, 21 Dec 2013 16:32:43 -0500
>> Subject: Re: [mvapich-discuss] PBS Pro's qstat failed to show "Time Use"
>> if building application with mvapich2
>> From: perkinjo at cse.ohio-state.edu
>> To: jerry_leo at msn.com
>> CC: mvapich-discuss at cse.ohio-state.edu
>
>>
>> I'd like to know how MVAPICH2 was built and how you are running your
>> jobs. You can run `mpiname -a` and/or the output from config.log from
>> your build directory if you build MVAPICH2 yourself. Can you also
>> provide information on how you submit your jobs such as whether you're
>> using mpirun_rsh or mpiexec?
>>
>> I believe that using RSH_CMD=/path/to/pbs/rsh and --enable-rsh at
>> build time will give the desired results if you're using mpirun_rsh
>> with PBS Pro. You should also get the desired results by using hydra
>> (mpiexec).
>>
>>
>> On Sat, Dec 21, 2013 at 2:13 AM, Jianyu Liu <jerry_leo at msn.com> wrote:
>> > Hi,
>> >
>> >
>> > Here are env. info
>> >
>> > Job scheduler : PBSPro 11.3.0.
>> > OS: Red Hat Enterprise Linux Server release 6.2
>> > mvapich2 : 2.0b
>> > InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s -
>> > IB QDR / 10GigE] (rev b0)
>> > mlx4_core: 1.0-mlnx_ofed1.5.3
>> > mlx4_en: 1.5.8.3 (June 2012)
>> >
>> >
>> > If application was built with mavpich2, while checking job status with
>> > qstat, it failed to show "Time use", just like this.
>> >
>> >
>> > [jliu at log08~]# qstat
>> > Job id Name User Time Use S Queue
>> > ---------------- ---------------- ---------------- -------- - -----
>> > 94540.log05 WRF3 jliu 00:00:00 R Regular
>> >
>> >
>> > Job works fine, just the "Time Use" always shows as ZERO.
>> >
>> >
>> > But if application was built with OpenMPI, there is NO this sort of
>> > issue.
>> >
>> >
>> > My question is
>> >
>> > how to build mvapich2 to support PBS Pro or any run-time options which
>> > can make qstat show "Time use" properly ?
>> >
>> >
>> > Thanks
>> >
>> >
>> > Jerry
>> >
>> > _______________________________________________
>> > mvapich-discuss mailing list
>> > mvapich-discuss at cse.ohio-state.edu
>> > http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>> >
>> >
>>
>>
>>
>> --
>> Jonathan Perkins
>> http://www.cse.ohio-state.edu/~perkinjo
>>



-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo