[Mvapich-discuss] Compiler error on Torque cluster

Tue May 9 11:03:57 EDT 2023

Hi Ron,

After looking at your hydra config logs it looks like what's happening is that your configuration is not correctly detecting the location of libpbs.so. This means there is probably an issue with your installation or your --with-pbs flag. If you have a nonstandard installation, please try to get libpbs.so into your LD_LIBRARY_PATH and LIBRARY_PATH environment variables. You can check this by doing gcc -lpbs and verifying that the error you get is related to an undefined reference to main, not /usr/bin/ld: cannot find -lpbs.

Without knowing your installation details, I cannot provide more detailed instructions.

Here are the relevant lines in the Hydra config.log:

configure:15781: checking tm.h usability
configure:15781: gcc -c -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include   -DNDEBUG -DNVALGRIND -O2  -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include  -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include conftest.c >&5
configure:15781: $? = 0
configure:15781: result: yes
configure:15781: checking tm.h presence
configure:15781: gcc -E -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include  -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include conftest.c
configure:15781: $? = 0
configure:15781: result: yes
configure:15781: checking for tm.h
configure:15781: result: yes
configure:15789: checking for tm_init in -ltorque
configure:15814: gcc -o conftest -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include   -DNDEBUG -DNVALGRIND -O2  -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include  -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include -L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib conftest.c -ltorque   >&5
configure:15814: $? = 0
configure:15823: result: yes
WRAPPER_LIBS(=' ') does not contain '-ltorque', appending
CPPFLAGS(='-I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include  -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include') does not contain '-I/opt/torque/6.1.3/include', appending
LDFLAGS(='-L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib') does not contain '-L/opt/torque/6.1.3/lib', appending
configure:15982: checking for tm.h
configure:15982: result: yes
configure:15990: checking for tm_init in -lpbs
configure:16015: gcc -o conftest -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include   -DNDEBUG -DNVALGRIND -O2  -I/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/include -I/usr/local/pace-apps/manual/packages/cuda/11.2/include  -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpl/include -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/openpa/src -D_REENTRANT -I/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/mpi/romio/include -I/opt/torque/6.1.3/include -I/opt/torque/6.1.3/include -L/usr/local/pace-apps/manual/packages/cuda/11.2/extras/CUPTI/lib64 -L/usr/local/pace-apps/manual/packages/cuda/11.2/lib64 -L/opt/torque/6.1.3/lib -L/opt/torque/6.1.3/lib conftest.c -lpbs   >&5
/bin/ld: cannot find -lpbs
collect2: error: ld returned 1 exit status
configure:16015: $? = 1

Please let me know if you have any further questions.

Thanks,
Nat

________________________________
From: Mvapich-discuss <mvapich-discuss-bounces at lists.osu.edu> on behalf of Subramoni, Hari via Mvapich-discuss <mvapich-discuss at lists.osu.edu>
Sent: Monday, May 8, 2023 13:13
To: Rahaman, Ronald O <rrahaman6 at gatech.edu>; Announcement about MVAPICH2 (MPI over InfiniBand, RoCE, Omni-Path, iWARP and EFA) Libraries developed at NBCL/OSU <mvapich-discuss at lists.osu.edu>
Subject: Re: [Mvapich-discuss] Compiler error on Torque cluster

Hi, Ron.

Sorry for the delay. We’re taking a look at it and will get back soon.

Best,

Hari.

From: Mvapich-discuss <mvapich-discuss-bounces+subramon=cse.ohio-state.edu at lists.osu.edu> On Behalf Of Rahaman, Ronald O via Mvapich-discuss
Sent: Thursday, May 4, 2023 3:48 PM
To: mvapich-discuss at lists.osu.edu
Subject: [Mvapich-discuss] Compiler error on Torque cluster

’m building MVAPICH 2. 3. 7 for a Torque cluster with RHEL 7. 9 and GCC 10. I’m getting a compiler error about 'HYDT_bsci_launcher_pbs_init' being undeclared. Here is my config. log and src/pm/hydra/config. log: https: //www. dropbox. com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

    Report Suspicious  <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vwQd0oZND6YAhRdwHo7Fd0rJ4aAOeq2b5G9bRFjIg3wrDHdqgr6-kEeKQABbt1IaZGX4GQmxf5ojfeeEaTpLezdajC4qkmq7M_I6coxrq0m5dScFE2DrnUktzh7ezGS_H8w7140GIslYk6JWHmSG_g$>   ‌

ZjQcmQRYFpfptBannerEnd

’m building MVAPICH 2.3.7 for a Torque cluster with RHEL 7.9 and GCC 10.  I’m getting a compiler error about 'HYDT_bsci_launcher_pbs_init' being undeclared.  Here is my config.log and src/pm/hydra/config.log:

https://www.dropbox.com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0<https://urldefense.com/v3/__https:/www.dropbox.com/sh/vqg35ylbjgpalkx/AABP5ml6cCai0ZL496UiL6wPa?dl=0__;!!KGKeukY!2wdfpNssoTC7KRMsB2LkULjCWHcN3WDucL1rgKAU7b0Eo22L0HWpr-wpKlQAFxo7JjX20Xo6b3ntpcNL0QXa2o5B7klDHqRt$>

And the compiler error is below.

Thanks!

Ron

make[3]: Entering directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'

  CC       tools/bootstrap/src/bsci_init.lo

tools/bootstrap/src/bsci_init.c:17:266: error: 'HYDT_bsci_launcher_pbs_init' undeclared here (not in a function); did you mean 'HYDT_bsci_launcher_sge_init'?

   17 | t_array[])(void) = {  HYDT_bsci_launcher_ssh_init,  HYDT_bsci_launcher_rsh_init,  HYDT_bsci_launcher_fork_init,  HYDT_bsci_launcher_slurm_init,  HYDT_bsci_launcher_ll_init,  HYDT_bsci_launcher_lsf_init,  HYDT_bsci_launcher_sge_init,  HYDT_bsci_launcher_pbs_init,  HYDT_bsci_launcher_manual_init,  HYDT_bsci_launcher_persist_init,  NULL };

      |                                                                                                                                                                                                                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~

      |                                                                                                                                                                                                                                           HYDT_bsci_launcher_sge_init

make[3]: *** [tools/bootstrap/src/bsci_init.lo] Error 1

make[3]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'

make[2]: *** [all-recursive] Error 1

make[2]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1/src/pm/hydra'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/storage/c-asdl1/pace-rrahaman6/build/mvapich2-2.3.7-1'

make: *** [all] Error 2

--------

Ron Rahaman

Research Scientist II, Research Software Engineer

Partnership for an Advanced Computing Environment (PACE)

Georgia Institute of Technology

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/mvapich-discuss/attachments/20230509/1a52299e/attachment-0006.html>