[mvapich-discuss] Installing MVAPICH2 - 2.3. GDR 2 on Summit

[Chu, Ching-Hsiang]

Hi Junchao,

If GDRCopy library is not available on the system, you can simply turn it off by setting the environment variable "MV2_USE_GDRCOPY=0" when running your MPI jobs.

Currently, we do not provide MVAPICH2-GDR RPMs to use jsrun directly on Summit due to some issues of PMIx library. We are working with other developers on fixing the issues and will provide new RPMs once it gets resolved. As a workaround, we recommend using the mpirun_rsh launcher.

Example script:
=====
# use jsrun to generate the desired hostfile
jsrun -n 2 -r 1 hostname > ./hfile
# launch MPI jobs using mpirun_rsh and the hostfile
export MV2_USE_GDRCOPY=0 MV2_USE_CUDA=1
export LD_PRELOAD=$MVAPICH2_GDR_ROOT/lib64/libmpi.so
$MVAPICH2_GDR_ROOT/bin/mpirun_rsh -export-all -np 2 --hostfile ./hfile ./osu_latency D D
=====

More information about using mpirun_rsh can be found in the MVAPICH2 userguide: http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.3.2-userguide.html#x1-310005.2.1
Contents<http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.3.2-userguide.html#x1-310005.2.1>
5 Basic Usage Instructions. 5.1 Compile Applications. MVAPICH2 provides a variety of MPI compilers to support applications written in different programming languages. Please use mpicc, mpif77, mpiCC, or mpif90 to compile applications. The correct compiler should be selected depending upon the programming language of your MPI application.
mvapich.cse.ohio-state.edu
Please let us know if you have any further questions.
Thanks,

Ching-Hsiang Chu

________________________________
From: mvapich-discuss <mvapich-discuss-bounces at cse.ohio-state.edu> on behalf of Zhang, Junchao <jczhang at mcs.anl.gov>
Sent: Wednesday, September 25, 2019 5:46 PM
To: mvapich-discuss at cse.ohio-state.edu <mvapich-discuss at mailman.cse.ohio-state.edu>
Subject: Re: [mvapich-discuss] Installing MVAPICH2 - 2.3. GDR 2 on Summit

Also, on Summit, I use jsrun to submit jobs. With mvapich, how to submit jobs?  Do you have successful experience on using mvapich2 on Summit?
Thanks.

--Junchao Zhang


On Wed, Sep 25, 2019 at 10:09 AM Junchao Zhang <jczhang at mcs.anl.gov<mailto:jczhang at mcs.anl.gov>> wrote:
Hello,
 I want to install MVAPICH2-GDR 2.3.2 on OLCF Summit. I followed instructions at http://mvapich.cse.ohio-state.edu/userguide/gdr/#_installing_mvapich2_gdr_library
 But I was stuck at installing gdbcopy library. From https://github.com/NVIDIA/gdrcopy, it seems I need sudo.  Of course, I don't have. Do you have a workaround?
 Thanks.

--Junchao Zhang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20190926/dc5ab6eb/attachment.html>