[mvapich-discuss] OpenACC not supported with mvapich2-gdr 2.3 rpm - Is there another version that does?

Tue Jan 29 10:18:01 EST 2019

Hi Hari,

We have installed the PGI version and GNU version on our system and are
currently trying to test the PGI version and it is complaining about a
missing PMI library, can you please let us know what we may be doing wrong?

Because our images are readonly we use cpio to install the RPM in a
different location, but this is the same process we have used before with
your intel distributions and those have worked fine.

We install the library in /apps instead of /opt and change the wrappers to
point to the new path.

Here are the details:

[Raghu.Reddy at sg001 hello]$ module purge

[Raghu.Reddy at sg001 hello]$ module load slurm cuda/9.2 pgi/18.7
mvapich2-gdr/2.3-1-cuda9.2-pgi17.10-slurm

[Raghu.Reddy at sg001 hello]$

[Raghu.Reddy at sg001 hello]$ mpicc -o foo hello_mpi_c.c

/bin/ld: warning: libpmi.so.0, needed by
/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so, not found (try using -rpath or -rpath-link)

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Get_size'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Get_rank'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Unpublish_name'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Get_name_length_max'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Get_key_length_max'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Finalize'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Get_my_name'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Spawn_multiple'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Lookup_name'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Get_value_length_max'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Barrier'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Init'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Put'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Initialized'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Abort'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Commit'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Get_universe_size'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Get_appnum'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_KVS_Get'

/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/libmpi.
so: undefined reference to `PMI_Publish_name'

[Raghu.Reddy at sg001 hello]$

sfe01% ll
/apps/mvapich2-gdr/2.3-1/cuda9.2-pgi17.10-mofed4.2-slurm-2.3-1/lib64/

total 123200

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        13 Jan 22 20:52 libfmpich.so ->
libmpifort.so*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        12 Jan 22 20:52 libmpichcxx.so ->
libmpicxx.so*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        13 Jan 22 20:52 libmpichf90.so ->
libmpifort.so*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt         9 Jan 22 20:52 libmpich.so ->
libmpi.so*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        19 Jan 22 20:52 libmpicxx.so ->
libmpicxx.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        19 Jan 22 20:52 libmpicxx.so.12
-> libmpicxx.so.12.1.1*

-rwxr-xr-x 1 Brian.Osmond nesccmgmt    488552 Jan 22 20:52
libmpicxx.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        20 Jan 22 20:52 libmpifort.so ->
libmpifort.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        20 Jan 22 20:52 libmpifort.so.12
-> libmpifort.so.12.1.1*

-rwxr-xr-x 1 Brian.Osmond nesccmgmt   6548352 Jan 22 20:52
libmpifort.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        16 Jan 22 20:52 libmpi.so ->
libmpi.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt        16 Jan 22 20:52 libmpi.so.12 ->
libmpi.so.12.1.1*

-rwxr-xr-x 1 Brian.Osmond nesccmgmt 118675144 Jan 22 20:52 libmpi.so.12.1.1*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt         9 Jan 22 20:52 libmpl.so ->
libmpi.so*

lrwxrwxrwx 1 Brian.Osmond nesccmgmt         9 Jan 22 20:52 libopa.so ->
libmpi.so*

drwxr-xr-x 2 Brian.Osmond nesccmgmt      2048 Jan 22 20:52 pkgconfig/

sfe01%

Please let us know if you need any additional information that may help in
diagnose and the problem.

Thank you very much for your help!

Thanks,

Raghu

From: Subramoni, Hari [mailto:subramoni.1 at osu.edu] 
Sent: Monday, January 21, 2019 4:17 PM
To: Raghu Reddy <raghu.reddy at noaa.gov>; mvapich-discuss at cse.ohio-state.edu
<mvapich-discuss at mailman.cse.ohio-state.edu>
Cc: Brian Osmond <brian.osmond at noaa.gov>; Subramoni, Hari
<subramoni.1 at osu.edu>
Subject: RE: [mvapich-discuss] OpenACC not supported with mvapich2-gdr 2.3
rpm - Is there another version that does?

Hi, Raghu.

We just meant that the OMB shipped with the RPM does not have support for
OpenACC related benchmarks. The MVAPICH2-GDR codebase should still support
OpenACC.

Thx,

Hari.

From: mvapich-discuss <mvapich-discuss-bounces at cse.ohio-state.edu
<mailto:mvapich-discuss-bounces at cse.ohio-state.edu> > On Behalf Of Raghu
Reddy
Sent: Monday, January 21, 2019 3:50 PM
To: mvapich-discuss at cse.ohio-state.edu
<mailto:mvapich-discuss at cse.ohio-state.edu>
<mvapich-discuss at mailman.cse.ohio-state.edu
<mailto:mvapich-discuss at mailman.cse.ohio-state.edu> >
Cc: Brian Osmond <brian.osmond at noaa.gov <mailto:brian.osmond at noaa.gov> >
Subject: [mvapich-discuss] OpenACC not supported with mvapich2-gdr 2.3 rpm -
Is there another version that does?

Dear MVAPICH-2 GDR team,

We have downloaded the following rpm:

mvapich2-gdr-mcast.cuda9.2.mofed4.2.gnu4.8.5.slurm-2.3-1.el7.x86_64.rpm

When we extract this file, I only the following tree:

               /opt/mvapich2/gdr/2.3/mcast/no-openacc/*

Based on this it looks like OpenACC would not be supported with this RPM.
Is there a version available that support OpenACC?

Thanks,

Raghu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20190129/6351e76f/attachment-0001.html>