[Mvapich-discuss] Unable to build older versions with newest NVIDIA compilers (24.5 and 24.7)

Tue Aug 27 18:46:11 EDT 2024

Hi Sylvain,

Thanks for your note. There is no active development for the MVAPICH2 2.3.* series anymore. Two weeks back, we have also released MVAPICH-PLus 4.0b. Please test MVAPICH-3.0 or MVAPICH-Plus 4.0b and let us know if you still encounter any issues. If there are issues, we will be happy to take a look at them.

Thanks,

DK

________________________________________
From: Mvapich-discuss <mvapich-discuss-bounces+panda.2=osu.edu at lists.osu.edu> on behalf of Korzennik, Sylvain via Mvapich-discuss <mvapich-discuss at lists.osu.edu>
Sent: Tuesday, August 27, 2024 4:55 PM
To: Announcement about MVAPICH2 (MPI over InfiniBand, RoCE, Omni-Path,  iWARP and EFA) Libraries developed at NBCL/OSU
Subject: [Mvapich-discuss] Unable to build older versions with newest NVIDIA    compilers (24.5 and 24.7)

While I can build mvapich-3. 0 (I have yet to test) I am unable to build mvapich2-2. 3. 6 or mvapich2-2. 3. 7-1p1 with the newest NVIDIA compilers i. e. 24. 5 and 24. 7.   I was able to build them with the 24. 3 version fine using the same steps. The
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/KGKeukY!vYQd06kpI8bsxpiRd4Zo15tUlHclWU7QPfUNNu0rIn9PmzkcSwf_Vr0L4subII3uO8WukbKdN-spOaUN7m65Pc6XmNVxNLQRWeQC5WQoGWr56vwvbV_0ADKOGOppbs8RQldbjQ$>
Report Suspicious

ZjQcmQRYFpfptBannerEnd
While I can build mvapich-3.0 (I have yet to test) I am unable to build mvapich2-2.3.6 or mvapich2-2.3.7-1p1 with the newest NVIDIA compilers i.e. 24.5 and 24.7.
I was able to build them with the 24.3 version fine using the same steps. The errors are:
egrep 'Error|error:' 24.[57]/mvapich2*/make.log

24.5/mvapich2-2.3.6/make.log:/share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: error: /share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: /tmp/nvcklSlbE2EM_vOA.ll:5881:90936: error: element 1 of struct initializer doesn't match struct element type
24.5/mvapich2-2.3.6/make.log:make[2]: *** [Makefile:33956: src/mpid/ch3/channels/mrail/src/gen2/lib_libmpi_la-ibv_channel_manager.lo] Error 1
24.5/mvapich2-2.3.6/make.log:make[1]: *** [Makefile:44389: all-recursive] Error 1
24.5/mvapich2-2.3.6/make.log:make: *** [Makefile:12025: all] Error 2

24.5/mvapich2-2.3.7-1p1/make.log:/share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: error: /share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: /tmp/nvcaw9LbaJlkSF2o.ll:5893:90936: error: element 1 of struct initializer doesn't match struct element type
24.5/mvapich2-2.3.7-1p1/make.log:make[2]: *** [Makefile:33960: src/mpid/ch3/channels/mrail/src/gen2/lib_libmpi_la-ibv_channel_manager.lo] Error 1
24.5/mvapich2-2.3.7-1p1/make.log:make[1]: *** [Makefile:44393: all-recursive] Error 1
24.5/mvapich2-2.3.7-1p1/make.log:make: *** [Makefile:12029: all] Error 2

24.7/mvapich2-2.3.6/make.log:/share/apps/nvidia/Linux_x86_64/24.7/compilers/share/llvm/bin/llc: error: /share/apps/nvidia/Linux_x86_64/24.7/compilers/share/llvm/bin/llc: /tmp/nvcq8PKbWXpeJf6O.ll:5881:90936: error: element 1 of struct initializer doesn't match struct element type
24.7/mvapich2-2.3.6/make.log:make[2]: *** [Makefile:33956: src/mpid/ch3/channels/mrail/src/gen2/lib_libmpi_la-ibv_channel_manager.lo] Error 1
24.7/mvapich2-2.3.6/make.log:make[1]: *** [Makefile:44389: all-recursive] Error 1
24.7/mvapich2-2.3.6/make.log:make: *** [Makefile:12025: all] Error 2

24.7/mvapich2-2.3.7-1p1/make.log:/share/apps/nvidia/Linux_x86_64/24.7/compilers/share/llvm/bin/llc: error: /share/apps/nvidia/Linux_x86_64/24.7/compilers/share/llvm/bin/llc: /tmp/nvctNDHc56lxqGMZ.ll:5893:90936: error: element 1 of struct initializer doesn't match struct element type
24.7/mvapich2-2.3.7-1p1/make.log:make[2]: *** [Makefile:33960: src/mpid/ch3/channels/mrail/src/gen2/lib_libmpi_la-ibv_channel_manager.lo] Error 1
24.7/mvapich2-2.3.7-1p1/make.log:make[1]: *** [Makefile:44393: all-recursive] Error 1
24.7/mvapich2-2.3.7-1p1/make.log:make: *** [Makefile:12029: all] Error 2

Any suggestions?

I could report this to NVIDIA (formerly PGI) but we no longer pay for support. The make.log has a lot of warnings, tho, and ends up with a weird very very long line just after the error:
/share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: error: /share/apps/nvidia/Linux_x86_64/24.5/compilers/share/llvm/bin/llc: /tmp/nvcaw9LbaJlkSF2o.ll:5893:90936: error: element 1 of struct initializer doesn't match struct element type
that starts with
@wc = internal global  <{  <{ <{ i64, i32, i32, i32, i32, <{ i32 }> , i32, i32, i32,
and ends with
}> }>  zeroinitializer }>, align 16

    Cheers,
      Sylvain

---

Sylvain G. Korzennik, Senior Physicist and HPC analyst

(Solar, Stellar and Planetary Sciences Division, Smithsonian Astrophysical Observatory)

Center for Astrophysics | Harvard & Smithsonian

60 Garden Street | MS 15 | Cambridge, MA 02138

Email: skorzennik at cfa.harvard.edu<mailto:skorzennik at cfa.harvard.edu> | Ph. 617-496-7916

[CfA_Logo_Horizontal_RGB.jpg]

cfa.harvard.edu<https://urldefense.com/v3/__http://cfa.harvard.edu/__;!!KGKeukY!1lInWJ1zHWAdIwS80_dY_DpBHPm_4e0VxcIev1R6PSZs-EqkjQ2IWefBC1VYnLl3pleGQ25XDZEbUryZ8hjVDGtsTcu1AyWU8W2DHQ$> | Facebook<https://urldefense.com/v3/__http://cfa.harvard.edu/facebook__;!!KGKeukY!1lInWJ1zHWAdIwS80_dY_DpBHPm_4e0VxcIev1R6PSZs-EqkjQ2IWefBC1VYnLl3pleGQ25XDZEbUryZ8hjVDGtsTcu1AyUfVGW43Q$> | Twitter<https://urldefense.com/v3/__http://cfa.harvard.edu/twitter__;!!KGKeukY!1lInWJ1zHWAdIwS80_dY_DpBHPm_4e0VxcIev1R6PSZs-EqkjQ2IWefBC1VYnLl3pleGQ25XDZEbUryZ8hjVDGtsTcu1AyXYITiPRA$> | YouTube<https://urldefense.com/v3/__http://cfa.harvard.edu/youtube__;!!KGKeukY!1lInWJ1zHWAdIwS80_dY_DpBHPm_4e0VxcIev1R6PSZs-EqkjQ2IWefBC1VYnLl3pleGQ25XDZEbUryZ8hjVDGtsTcu1AyUrp01lBg$> | Newsletter<https://urldefense.com/v3/__http://cfa.harvard.edu/newsletter__;!!KGKeukY!1lInWJ1zHWAdIwS80_dY_DpBHPm_4e0VxcIev1R6PSZs-EqkjQ2IWefBC1VYnLl3pleGQ25XDZEbUryZ8hjVDGtsTcu1AyWFc3UEIA$>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CfA_Logo_Horizontal_RGB.jpg
Type: image/jpeg
Size: 275645 bytes
Desc: CfA_Logo_Horizontal_RGB.jpg
URL: <http://lists.osu.edu/pipermail/mvapich-discuss/attachments/20240827/996d6569/attachment-0002.jpg>