[mvapich-discuss] MVAPICH 2.3a-GDR on POWER8: Run Problems

[Herten, Andreas]

Dear all,

I’m involved in operation of a POWER8+NVIDIA cluster, JURON, at Jülich Supercomputing Centre.

I downloaded the GDR built for POWER but am unable to run CUDA-aware MPI applications.

Testing with the packaged OSU microbenchmarks, the program is stuck right at the beginning. See below.

JURON runs CentOS with GCC 4.8.5, but we also tested loading GCC 5.4.0. We use an EDR InfiniBand interconnect. Our batch system is LSF – is MVAPICH built with LSF support?
Please let me know if you need further information for debugging.

> $ env | grep MV2
> MV2_ENABLE_AFFINITY=0
> MV2_USE_CUDA=1
> MV2_USE_GPUDIRECT_GDRCOPY=0

> $ bsub -n 2 -I -R "rusage[ngpus_shared=20]" mpirun /gpfs/software/opt/mvapich2/2.3a-GDR-cuda_8/libexec/osu-micro-benchmarks/get_local_rank /gpfs/software/opt/mvapich2/2.3a-GDR-cuda_8/libexec/osu-micro-benchmarks/mpi/pt2pt/osu_bibw D D
> Job <9706> is submitted to default queue <normal.i>.
> <<Waiting for dispatch ...>>
> <<Starting on juronc05>>
> # OSU MPI-CUDA Bi-Directional Bandwidth Test v5.4.0
> # Send Buffer on DEVICE (D) and Receive Buffer on DEVICE (D)
> # Size      Bandwidth (MB/s)

-Andreas
—
Dr. Andreas Herten
NVIDIA Application Lab
Jülich Supercomputing Centre
Forschungszentrum Jülich, Germany
+49 2461 61 1825

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4972 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20171123/41be48b8/attachment-0001.p7s>