From bourdin at mcmaster.ca Tue Apr 8 10:34:19 2025 From: bourdin at mcmaster.ca (Blaise Bourdin) Date: Tue, 8 Apr 2025 14:34:19 +0000 Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) Message-ID: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hi, I am running into the same problem when building the tarball for MVAPICH 4.0 at https://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich-4.0.tar.gz with the AMD AOCC compilers v5.0 and trying to use the RPM distribution of MVAPICH-Plus 4.0: When building MVAPICH 4.0, I get the following error message: ./configure --prefix=/2/sw/HPC/openmpi/5.0.7 CC=clang FC=flang CXX=clang++ --with-slurm libtool: link: clang -DOPAL_CONFIGURE_USER=\"bourdinb\" -DOPAL_CONFIGURE_HOST=\"bbserv\" "-DOPAL_CONFIGURE_DATE=\"Mon Apr 7 14:42:59 UTC 2025\"" -DOMPI_BUILD_USER=\"bourdinb\" -DOMPI_BUILD_HOST=\"bbserv\" "-DOMPI_BUILD_DATE=\"Mon Apr 7 02:58:17 PM UTC 2025\"" "-DOMPI_BUILD_CFLAGS=\"-O3 -DNDEBUG -finline-functions -mcx16\"" "-DOMPI_BUILD_CPPFLAGS=\"-iquote../../.. -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/ -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/\"" "-DOMPI_BUILD_CXXFLAGS=\"-DNDEBUG \"" -DOMPI_BUILD_CXXCPPFLAGS=\"@CXXCPPFLAGS@\" -DOMPI_BUILD_FFLAGS=\"\" -DOMPI_BUILD_FCFLAGS=\"\" -DOMPI_BUILD_LDFLAGS=\"\" "-DOMPI_BUILD_LIBS=\"-lm /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_core.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_pthreads.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/libpmix.la\"" -DOPAL_CC_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang\" -DOMPI_CXX_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang++\" -O3 -DNDEBUG -finline-functions -mcx16 -o .libs/ompi_info ompi_info.o param.o ../../../ompi/.libs/libmpi.so -L/opt/mellanox/hcoll/lib -L../src/hcoll/commpatterns -L../src/hcoll/netpatterns -L/usr/lib64 -L/usr/lib /opt/mellanox/hcoll/lib/libhcoll.so /opt/mellanox/hcoll/lib/libocoms.so -lrt -lrdmacm -libverbs -lpthread /2/sw/HPC/src/openmpi-5.0.7/opal/.libs/libopen-pal.so ../../../opal/.libs/libopen-pal.so -lucp -lucs -lucm -luct /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/.libs/libpmix.so -lmunge -ldl /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_core.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_pthreads.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -Wl,-rpath -Wl,/2/sw/HPC/openmpi/5.0.7/lib ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __extendhfsf2 >>> referenced by ../../../ompi/.libs/libmpi.so ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __truncsfhf2 >>> referenced by ../../../ompi/.libs/libmpi.so clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [Makefile:1478: ompi_info] Error 1 make[2]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi/tools/ompi_info' make[1]: *** [Makefile:2710: all-recursive] Error 1 make[1]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi' make: *** [Makefile:1539: all-recursive] Error 1 The full report is here: https://urldefense.com/v3/__https://macdrive.mcmaster.ca/f/5879ebda7cfe49c283fe/__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSUabVlcBg$ Trying the MVAPICH-4.0-plus rpm, I Geta. Similar error: bbserv:~ $ sudo rpm --prefix=/2/sw/HPC/mvapich2/4.0-plus -Umvapich-plus-4.0-nogpu.rhel9.ofed24.10.ucx.gcc13.2.0.slurm-4.0-1.x86_64.rpmrpm Updating / installing... 1:mvapich-plus-4.0-nogpu.rhel9.ofed################################# [100%] bbserv:~ $ export PATH=/2/sw/HPC/mvapich2/4.0-plus/bin:$PATH bbserv:~ $ export LD_LIBRARY_PATH=/2/sw/HPC/mvapich2/4.0-plus/lib:$LD_LIBRARY_PATH bbserv:~ $ cd Development/SandBox/MPI bbserv:MPI $ which mpicc /2/sw/HPC/mvapich2/4.0-plus/bin/mpicc bbserv:MPI $ mpicc -o cpi cpi.c /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__extendhfsf2' /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__truncsfhf2' collect2: error: ld returned 1 exit status I have been trying to figure out which library is supposed to provide these two symbols, without luck so far. Has anybody encountered the same problem? Regards, Blaise ? Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics (Tier 1) Professor, Department of Mathematics & Statistics Hamilton Hall room 409A, McMaster University 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada https://urldefense.com/v3/__https://www.math.mcmaster.ca/bourdin__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSXhA-wKxg$ | +1 (905) 525 9140 ext. 27243 From shineman.5 at osu.edu Tue Apr 8 13:52:23 2025 From: shineman.5 at osu.edu (Shineman, Nat) Date: Tue, 8 Apr 2025 17:52:23 +0000 Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) In-Reply-To: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> References: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> Message-ID: Hi Blaise, Not sure how exactly you are configuring your environment, but it looks like you are actually building OpenMPI, not MVAPICH in your source build. In that case, you are having the same error with both MVAPICH and OMPI, which suggests that this is a system error or an issue with your AOCC installation. If these libraries are in your runtime paths, they may also be impacting the RPM installation. In addition, the RPM you have is built with GCC 13 compilers, not AOCC so you will want to make sure you have the same compiler installed. Thanks, Nat ________________________________ From: Mvapich-discuss on behalf of Blaise Bourdin via Mvapich-discuss Sent: Tuesday, April 8, 2025 10:34 To: mvapich-discuss at lists.osu.edu Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hi, I am running into the same problem when building the tarball for MVAPICH 4.0 at https://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich-4.0.tar.gz with the AMD AOCC compilers v5.0 and trying to use the RPM distribution of MVAPICH-Plus 4.0: When building MVAPICH 4.0, I get the following error message: ./configure --prefix=/2/sw/HPC/openmpi/5.0.7 CC=clang FC=flang CXX=clang++ --with-slurm libtool: link: clang -DOPAL_CONFIGURE_USER=\"bourdinb\" -DOPAL_CONFIGURE_HOST=\"bbserv\" "-DOPAL_CONFIGURE_DATE=\"Mon Apr 7 14:42:59 UTC 2025\"" -DOMPI_BUILD_USER=\"bourdinb\" -DOMPI_BUILD_HOST=\"bbserv\" "-DOMPI_BUILD_DATE=\"Mon Apr 7 02:58:17 PM UTC 2025\"" "-DOMPI_BUILD_CFLAGS=\"-O3 -DNDEBUG -finline-functions -mcx16\"" "-DOMPI_BUILD_CPPFLAGS=\"-iquote../../.. -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/ -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/\"" "-DOMPI_BUILD_CXXFLAGS=\"-DNDEBUG \"" -DOMPI_BUILD_CXXCPPFLAGS=\"@CXXCPPFLAGS@\" -DOMPI_BUILD_FFLAGS=\"\" -DOMPI_BUILD_FCFLAGS=\"\" -DOMPI_BUILD_LDFLAGS=\"\" "-DOMPI_BUILD_LIBS=\"-lm /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_core.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_pthreads.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/libpmix.la\"" -DOPAL_CC_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang\" -DOMPI_CXX_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang++\" -O3 -DNDEBUG -finline-functions -mcx16 -o .libs/ompi_info ompi_info.o param.o ../../../ompi/.libs/libmpi.so -L/opt/mellanox/hcoll/lib -L../src/hcoll/commpatterns -L../src/hcoll/netpatterns -L/usr/lib64 -L/usr/lib /opt/mellanox/hcoll/lib/libhcoll.so /opt/mellanox/hcoll/lib/libocoms.so -lrt -lrdmacm -libverbs -lpthread /2/sw/HPC/src/openmpi-5.0.7/opal/.libs/libopen-pal.so ../../../opal/.libs/libopen-pal.so -lucp -lucs -lucm -luct /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/.libs/libpmix.so -lmunge -ldl /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_core.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_pthreads.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -Wl,-rpath -Wl,/2/sw/HPC/openmpi/5.0.7/lib ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __extendhfsf2 >>> referenced by ../../../ompi/.libs/libmpi.so ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __truncsfhf2 >>> referenced by ../../../ompi/.libs/libmpi.so clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [Makefile:1478: ompi_info] Error 1 make[2]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi/tools/ompi_info' make[1]: *** [Makefile:2710: all-recursive] Error 1 make[1]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi' make: *** [Makefile:1539: all-recursive] Error 1 The full report is here: https://urldefense.com/v3/__https://macdrive.mcmaster.ca/f/5879ebda7cfe49c283fe/__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSUabVlcBg$ Trying the MVAPICH-4.0-plus rpm, I Geta. Similar error: bbserv:~ $ sudo rpm --prefix=/2/sw/HPC/mvapich2/4.0-plus -Umvapich-plus-4.0-nogpu.rhel9.ofed24.10.ucx.gcc13.2.0.slurm-4.0-1.x86_64.rpmrpm Updating / installing... 1:mvapich-plus-4.0-nogpu.rhel9.ofed################################# [100%] bbserv:~ $ export PATH=/2/sw/HPC/mvapich2/4.0-plus/bin:$PATH bbserv:~ $ export LD_LIBRARY_PATH=/2/sw/HPC/mvapich2/4.0-plus/lib:$LD_LIBRARY_PATH bbserv:~ $ cd Development/SandBox/MPI bbserv:MPI $ which mpicc /2/sw/HPC/mvapich2/4.0-plus/bin/mpicc bbserv:MPI $ mpicc -o cpi cpi.c /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__extendhfsf2' /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__truncsfhf2' collect2: error: ld returned 1 exit status I have been trying to figure out which library is supposed to provide these two symbols, without luck so far. Has anybody encountered the same problem? Regards, Blaise ? Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics (Tier 1) Professor, Department of Mathematics & Statistics Hamilton Hall room 409A, McMaster University 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada https://urldefense.com/v3/__https://www.math.mcmaster.ca/bourdin__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSXhA-wKxg$ | +1 (905) 525 9140 ext. 27243 _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.goldman at intel.com Wed Apr 16 13:42:03 2025 From: adam.goldman at intel.com (Goldman, Adam) Date: Wed, 16 Apr 2025 17:42:03 +0000 Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 In-Reply-To: References: Message-ID: !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hello, I noticed that the CHANGES list "Intel GPU support." Is this correct for v7.5? I am not able to find a way to compile with the level-zero library and the README does not mention "Intel GPU" anywhere. Thank you, Adam Goldman -----Original Message----- From: Mvapich-discuss On Behalf Of Panda, Dhabaleswar via Mvapich-discuss Sent: Friday, November 1, 2024 7:07 PM To: Mvapich-discuss at lists.osu.edu Cc: Panda, Dhabaleswar Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 The MVAPICH team is pleased to announce the release of OSU Micro-Benchmarks (OMB) 7.5. Please note that OMB is also available through the Spack package manager. Now the system administrators and users of OSU Micro-Benchmarks (OMB) will be able to install these libraries on their systems using Spack. The new features, enhancements, and bug fixes for OSU Micro-Benchmarks (OMB) 7.5 are listed here: * New Features & Enhancements (since 7.4) - New OpenSHMEM benchmarks * osu_oshm_get_bw, osu_oshm_get_nb_bw * osu_oshm_put_bw, osu_oshm_put_nb_bw * osu_oshm_get_overlap - New partitioned point-to-point benchmkark * osu_partitioned_latency - Add NVIDIA, AMD, and Intel GPU support for neighborhood collective benchmarks * osu_neighbor_allgather, osu_neighbor_allgatherv, * osu_neighbor_alltoall, osu_neighbor_alltoallv, * osu_neighbor_alltoallw, osu_ineighbor_allgatherv, * osu_ineighbor_allgatherv, osu_ineighbor_alltoall, * osu_ineighbor_alltoallv, osu_ineighbor_alltoallw - Add Intel GPU support for the following benchmarks * Point-to-point benchmarks * osu_bibw, osu_bw, osu_mbw_mr, osu_latency, osu_multi_lat, * osu_latency_mp, osu_latency_mt, osu_bw_persistent, * osu_bibw_persistent, osu_latency_persistent, * osu_partitioned_latency * Network congestion benchmarks * osu_bw_fan_in, osu_bw_fan_out * Blocking collective benchmarks * osu_allgather, osu_allgatherv, osu_alltoall, osu_allreduce, * osu_alltoallv, osu_alltoallw, osu_bcast, osu_barrier, osu_gather, * osu_gatherv, osu_reduce, osu_reduce_scatter, osu_scatter, * osu_scatterv, osu_reduce_scatter_block * Non-Blocking collective benchmarks * osu_iallgather, osu_iallgatherv, osu_iallreduce, osu_ialltoall, * osu_ialltoallv, osu_ialltoallw, osu_ibcast, osu_ibarrier, * osu_igather, osu_igatherv, osu_ireduce, osu_iscatter, * osu_iscatterv, osu_ireduce_scatter, osu_ireduce_scatter_block * Persistent collective benchmarks * osu_allgather_persistent, osu_allgatherv_persistent, * osu_allreduce_persistent, osu_alltoall_persistent, * osu_alltoallv_persistent, osu_alltoallw_persistent, * osu_barrier_persistent, osu_bcast_persistent, * osu_gather_persistent, osu_gatherv_persistent, * osu_reduce_persistent, osu_reduce_scatter_persistent, * osu_scatter_persistent, osu_scatterv_persistent * One-sided benchmarks * osu_acc_latency, osu_cas_latency, osu_fop_latency, osu_get_bw, * osu_get_latency, osu_put_bibw, osu_put_bw, osu_put_latency, * osu_get_acc_latency - Changed default MPI_TYPE of reduce benchmarks to MPI_INT. * osu_reduce, osu_allreduce, osu_reduce_scatter, osu_reduce_scatter_block * Bug Fixes - Fixed RCCL build in ROCm 6 due to installation path changes. - Fixed a bug leading to nvcc command not found error when builing with NCCL support. * Thanks to Raghu Raja @Amazon for report. - Resolved a bug in the maximum memory limit check, which previously considered only the memory allocated by a single buffer. This check and command line option 'M' no longer exists. * Thanks to Van Man Nguyen @Eviden for report and patch. - Fixed bug while printing validation results in some collective benchmarks. - Updated reduction benchmarks to use MPI_SIGNED_CHAR instead of MPI_CHAR. * Thanks to R??mi Dehenne @CEA for report and patch. - Fixed a bug in Reduce, Reduce-Scatter benchmarks that caused validation to fail when running with ROCM support. * Thanks to Anton Daumen @Eviden for report and patch. * Thanks to Edgar Gabriel @AMD and Mustafa Abduljabbar @AMD for report. - Fixed RCCL check in configure script. * Thanks to Anton Daumen @Eviden for report. For downloading OMB 7.5 and associated README instructions, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH libraries (and registered at the MVAPICH site) has crossed 3,400 worldwide (in 92 countries). The number of downloads from the MVAPICH site has crossed 1,835,000 (1.835 million). The MVAPICH team would like to thank all its users and organizations!! _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss From chen.10252 at buckeyemail.osu.edu Wed Apr 16 21:33:42 2025 From: chen.10252 at buckeyemail.osu.edu (Chen, Chen Chun) Date: Thu, 17 Apr 2025 01:33:42 +0000 Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 In-Reply-To: References: Message-ID: Hi Adam, OMB supports Intel GPUs through SYCL. Users can enable this support by adding the following configuration flags: --enable-sycl --with-sycl=/PATH/TO/ONEAPI For example: ./configure --enable-sycl --with-sycl=/opt/intel/oneapi CC=mpicc CXX=mpicxx You?ll also find more detailed setup instructions in the README (https://mvapich.cse.ohio-state.edu/static/media/mvapich/README-OMB.txt). Some SYCL-related content can be found in the README for reference: SYCL extensions can be enabled by configuring OMB with --enable-sycl option as shown below. ./configure CC=/path/to/mpicc CXX=/path/to/mpicxx --enable-sycl --with-sycl=/path/to/sycl/install make make install ? If both CUDA and OpenACC support is enabled you can switch between the modes using the -d [cuda|openacc] option to the benchmarks. If ROCm support is enabled, you need to use -d rocm option to make the benchmarks use this feature. If SYCL support is enabled, you need to use -d sycl option to make the benchmarks use this feature. Whether a process allocates its communication buffers on the GPU device or on the host can be controlled at run-time. Use the -h option for more help. If you have any questions, feel free to reach out?we?re always happy to help! Best, Chen-Chun From: Mvapich-discuss on behalf of Goldman, Adam via Mvapich-discuss Date: Wednesday, April 16, 2025 at 1:42?PM To: mvapich-discuss at lists.osu.edu Subject: Re: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hello, I noticed that the CHANGES list "Intel GPU support." Is this correct for v7.5? I am not able to find a way to compile with the level-zero library and the README does not mention "Intel GPU" anywhere. Thank you, Adam Goldman -----Original Message----- From: Mvapich-discuss On Behalf Of Panda, Dhabaleswar via Mvapich-discuss Sent: Friday, November 1, 2024 7:07 PM To: Mvapich-discuss at lists.osu.edu Cc: Panda, Dhabaleswar Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 The MVAPICH team is pleased to announce the release of OSU Micro-Benchmarks (OMB) 7.5. Please note that OMB is also available through the Spack package manager. Now the system administrators and users of OSU Micro-Benchmarks (OMB) will be able to install these libraries on their systems using Spack. The new features, enhancements, and bug fixes for OSU Micro-Benchmarks (OMB) 7.5 are listed here: * New Features & Enhancements (since 7.4) - New OpenSHMEM benchmarks * osu_oshm_get_bw, osu_oshm_get_nb_bw * osu_oshm_put_bw, osu_oshm_put_nb_bw * osu_oshm_get_overlap - New partitioned point-to-point benchmkark * osu_partitioned_latency - Add NVIDIA, AMD, and Intel GPU support for neighborhood collective benchmarks * osu_neighbor_allgather, osu_neighbor_allgatherv, * osu_neighbor_alltoall, osu_neighbor_alltoallv, * osu_neighbor_alltoallw, osu_ineighbor_allgatherv, * osu_ineighbor_allgatherv, osu_ineighbor_alltoall, * osu_ineighbor_alltoallv, osu_ineighbor_alltoallw - Add Intel GPU support for the following benchmarks * Point-to-point benchmarks * osu_bibw, osu_bw, osu_mbw_mr, osu_latency, osu_multi_lat, * osu_latency_mp, osu_latency_mt, osu_bw_persistent, * osu_bibw_persistent, osu_latency_persistent, * osu_partitioned_latency * Network congestion benchmarks * osu_bw_fan_in, osu_bw_fan_out * Blocking collective benchmarks * osu_allgather, osu_allgatherv, osu_alltoall, osu_allreduce, * osu_alltoallv, osu_alltoallw, osu_bcast, osu_barrier, osu_gather, * osu_gatherv, osu_reduce, osu_reduce_scatter, osu_scatter, * osu_scatterv, osu_reduce_scatter_block * Non-Blocking collective benchmarks * osu_iallgather, osu_iallgatherv, osu_iallreduce, osu_ialltoall, * osu_ialltoallv, osu_ialltoallw, osu_ibcast, osu_ibarrier, * osu_igather, osu_igatherv, osu_ireduce, osu_iscatter, * osu_iscatterv, osu_ireduce_scatter, osu_ireduce_scatter_block * Persistent collective benchmarks * osu_allgather_persistent, osu_allgatherv_persistent, * osu_allreduce_persistent, osu_alltoall_persistent, * osu_alltoallv_persistent, osu_alltoallw_persistent, * osu_barrier_persistent, osu_bcast_persistent, * osu_gather_persistent, osu_gatherv_persistent, * osu_reduce_persistent, osu_reduce_scatter_persistent, * osu_scatter_persistent, osu_scatterv_persistent * One-sided benchmarks * osu_acc_latency, osu_cas_latency, osu_fop_latency, osu_get_bw, * osu_get_latency, osu_put_bibw, osu_put_bw, osu_put_latency, * osu_get_acc_latency - Changed default MPI_TYPE of reduce benchmarks to MPI_INT. * osu_reduce, osu_allreduce, osu_reduce_scatter, osu_reduce_scatter_block * Bug Fixes - Fixed RCCL build in ROCm 6 due to installation path changes. - Fixed a bug leading to nvcc command not found error when builing with NCCL support. * Thanks to Raghu Raja @Amazon for report. - Resolved a bug in the maximum memory limit check, which previously considered only the memory allocated by a single buffer. This check and command line option 'M' no longer exists. * Thanks to Van Man Nguyen @Eviden for report and patch. - Fixed bug while printing validation results in some collective benchmarks. - Updated reduction benchmarks to use MPI_SIGNED_CHAR instead of MPI_CHAR. * Thanks to R??mi Dehenne @CEA for report and patch. - Fixed a bug in Reduce, Reduce-Scatter benchmarks that caused validation to fail when running with ROCM support. * Thanks to Anton Daumen @Eviden for report and patch. * Thanks to Edgar Gabriel @AMD and Mustafa Abduljabbar @AMD for report. - Fixed RCCL check in configure script. * Thanks to Anton Daumen @Eviden for report. For downloading OMB 7.5 and associated README instructions, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH libraries (and registered at the MVAPICH site) has crossed 3,400 worldwide (in 92 countries). The number of downloads from the MVAPICH site has crossed 1,835,000 (1.835 million). The MVAPICH team would like to thank all its users and organizations!! _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.thompson at nasa.gov Wed Apr 30 13:42:56 2025 From: matthew.thompson at nasa.gov (Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]) Date: Wed, 30 Apr 2025 17:42:56 +0000 Subject: [Mvapich-discuss] SIGFPE on MPI_Init_thread with GCC 14? Message-ID: All, I'm hoping this oddity is something people here have seen before. I built MVAPICH 4.0 with GCC 14.2.0 and it is dying with a SIGFPE on this line: call MPI_Init_thread(MPI_THREAD_MULTIPLE, provided, ierror) Very boring. Thing is, Intel ifort 2021.13 + MVAPICH 4.0 is just fine with the same code. And I can make a simple MPI program to see if, for some reason, I can't call MPI_Init_thread with MPI_THREAD_MULTIPLE, but no, I can. Does this ring a bell for anyone? It's just...weird. I mean...SIGFPE?! Matt [signature_3763662456] Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson at nasa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 63716 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 358 bytes Desc: image002.png URL: From panda at cse.ohio-state.edu Wed Apr 30 23:24:47 2025 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Thu, 1 May 2025 03:24:47 +0000 Subject: [Mvapich-discuss] MUG '25 Call for Presentations Message-ID: The MVAPICH team is excited to host the 13th annual MVAPICH User Group (MUG) conference. It will take place from August 18-20, 2025 in Columbus, Ohio, USA. It will be held in a hybrid manner. The MUG conference aims to bring together the MVAPICH users, researchers, developers, and system administrators to share their experiences, knowledge and to learn from each other. The event includes keynote talks, invited tutorials, invited talks, contributed presentations, Open Mic session, hands-on sessions with the MVAPICH developers, etc. This year, we will be holding special tutorials and demo/hands-on sessions during the first day of the event (August 18th). Other talks and sessions will be held on August 19-20. A set of short contributed presentations from the MVAPICH users will be included in the event. Topics for presentations include, but are not limited to: - Case studies and best practices of novel applications from different application domains, such as: astronomy, bioinformatics, biology, earth and atmospheric sciences, fluid dynamics, materials science and engineering, medicine, physics, and AI (machine learning, deep learning (training and inference)) - Performance and scalability studies of applications on large-scale systems - Special tuning and optimization strategies to exploit maximum performance and scalability - Tools and code instrumentation for measuring and monitoring performance and/or resilience - Tools for parallel program development (e.g. debuggers and integrated development environments) - Unique usage scenarios with GPUs, DPUs, APUs, FPGAs, Energy-Awareness, Virtualization, Quantum simulation, etc. The submission should include the title of the presentation, speaker(s), short bio of the speaker(s), and a draft version of the presentation (around 10-15 slides in PDF or PowerPoint format). Please send your submission in a single file to mug at cse.ohio-state.edu. Presentation Submission Deadline: July 11, 2025 Notification of Acceptance: July 18, 2025 More details on the conference and Call for Presentations are available from http://mug.mvapich.cse.ohio-state.edu/ Thanks, The MVAPICH Team From bourdin at mcmaster.ca Tue Apr 8 10:34:19 2025 From: bourdin at mcmaster.ca (Blaise Bourdin) Date: Tue, 8 Apr 2025 14:34:19 +0000 Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) Message-ID: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hi, I am running into the same problem when building the tarball for MVAPICH 4.0 at https://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich-4.0.tar.gz with the AMD AOCC compilers v5.0 and trying to use the RPM distribution of MVAPICH-Plus 4.0: When building MVAPICH 4.0, I get the following error message: ./configure --prefix=/2/sw/HPC/openmpi/5.0.7 CC=clang FC=flang CXX=clang++ --with-slurm libtool: link: clang -DOPAL_CONFIGURE_USER=\"bourdinb\" -DOPAL_CONFIGURE_HOST=\"bbserv\" "-DOPAL_CONFIGURE_DATE=\"Mon Apr 7 14:42:59 UTC 2025\"" -DOMPI_BUILD_USER=\"bourdinb\" -DOMPI_BUILD_HOST=\"bbserv\" "-DOMPI_BUILD_DATE=\"Mon Apr 7 02:58:17 PM UTC 2025\"" "-DOMPI_BUILD_CFLAGS=\"-O3 -DNDEBUG -finline-functions -mcx16\"" "-DOMPI_BUILD_CPPFLAGS=\"-iquote../../.. -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/ -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/\"" "-DOMPI_BUILD_CXXFLAGS=\"-DNDEBUG \"" -DOMPI_BUILD_CXXCPPFLAGS=\"@CXXCPPFLAGS@\" -DOMPI_BUILD_FFLAGS=\"\" -DOMPI_BUILD_FCFLAGS=\"\" -DOMPI_BUILD_LDFLAGS=\"\" "-DOMPI_BUILD_LIBS=\"-lm /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_core.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_pthreads.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/libpmix.la\"" -DOPAL_CC_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang\" -DOMPI_CXX_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang++\" -O3 -DNDEBUG -finline-functions -mcx16 -o .libs/ompi_info ompi_info.o param.o ../../../ompi/.libs/libmpi.so -L/opt/mellanox/hcoll/lib -L../src/hcoll/commpatterns -L../src/hcoll/netpatterns -L/usr/lib64 -L/usr/lib /opt/mellanox/hcoll/lib/libhcoll.so /opt/mellanox/hcoll/lib/libocoms.so -lrt -lrdmacm -libverbs -lpthread /2/sw/HPC/src/openmpi-5.0.7/opal/.libs/libopen-pal.so ../../../opal/.libs/libopen-pal.so -lucp -lucs -lucm -luct /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/.libs/libpmix.so -lmunge -ldl /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_core.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_pthreads.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -Wl,-rpath -Wl,/2/sw/HPC/openmpi/5.0.7/lib ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __extendhfsf2 >>> referenced by ../../../ompi/.libs/libmpi.so ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __truncsfhf2 >>> referenced by ../../../ompi/.libs/libmpi.so clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [Makefile:1478: ompi_info] Error 1 make[2]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi/tools/ompi_info' make[1]: *** [Makefile:2710: all-recursive] Error 1 make[1]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi' make: *** [Makefile:1539: all-recursive] Error 1 The full report is here: https://urldefense.com/v3/__https://macdrive.mcmaster.ca/f/5879ebda7cfe49c283fe/__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSUabVlcBg$ Trying the MVAPICH-4.0-plus rpm, I Geta. Similar error: bbserv:~ $ sudo rpm --prefix=/2/sw/HPC/mvapich2/4.0-plus -Umvapich-plus-4.0-nogpu.rhel9.ofed24.10.ucx.gcc13.2.0.slurm-4.0-1.x86_64.rpmrpm Updating / installing... 1:mvapich-plus-4.0-nogpu.rhel9.ofed################################# [100%] bbserv:~ $ export PATH=/2/sw/HPC/mvapich2/4.0-plus/bin:$PATH bbserv:~ $ export LD_LIBRARY_PATH=/2/sw/HPC/mvapich2/4.0-plus/lib:$LD_LIBRARY_PATH bbserv:~ $ cd Development/SandBox/MPI bbserv:MPI $ which mpicc /2/sw/HPC/mvapich2/4.0-plus/bin/mpicc bbserv:MPI $ mpicc -o cpi cpi.c /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__extendhfsf2' /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__truncsfhf2' collect2: error: ld returned 1 exit status I have been trying to figure out which library is supposed to provide these two symbols, without luck so far. Has anybody encountered the same problem? Regards, Blaise ? Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics (Tier 1) Professor, Department of Mathematics & Statistics Hamilton Hall room 409A, McMaster University 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada https://urldefense.com/v3/__https://www.math.mcmaster.ca/bourdin__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSXhA-wKxg$ | +1 (905) 525 9140 ext. 27243 From shineman.5 at osu.edu Tue Apr 8 13:52:23 2025 From: shineman.5 at osu.edu (Shineman, Nat) Date: Tue, 8 Apr 2025 17:52:23 +0000 Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) In-Reply-To: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> References: <3611E4C6-60CD-4A29-8D71-C8513866F2A1@mcmaster.ca> Message-ID: Hi Blaise, Not sure how exactly you are configuring your environment, but it looks like you are actually building OpenMPI, not MVAPICH in your source build. In that case, you are having the same error with both MVAPICH and OMPI, which suggests that this is a system error or an issue with your AOCC installation. If these libraries are in your runtime paths, they may also be impacting the RPM installation. In addition, the RPM you have is built with GCC 13 compilers, not AOCC so you will want to make sure you have the same compiler installed. Thanks, Nat ________________________________ From: Mvapich-discuss on behalf of Blaise Bourdin via Mvapich-discuss Sent: Tuesday, April 8, 2025 10:34 To: mvapich-discuss at lists.osu.edu Subject: [Mvapich-discuss] Mvapich4 and rocky linux 9.5 (RHEL9.5) !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hi, I am running into the same problem when building the tarball for MVAPICH 4.0 at https://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich-4.0.tar.gz with the AMD AOCC compilers v5.0 and trying to use the RPM distribution of MVAPICH-Plus 4.0: When building MVAPICH 4.0, I get the following error message: ./configure --prefix=/2/sw/HPC/openmpi/5.0.7 CC=clang FC=flang CXX=clang++ --with-slurm libtool: link: clang -DOPAL_CONFIGURE_USER=\"bourdinb\" -DOPAL_CONFIGURE_HOST=\"bbserv\" "-DOPAL_CONFIGURE_DATE=\"Mon Apr 7 14:42:59 UTC 2025\"" -DOMPI_BUILD_USER=\"bourdinb\" -DOMPI_BUILD_HOST=\"bbserv\" "-DOMPI_BUILD_DATE=\"Mon Apr 7 02:58:17 PM UTC 2025\"" "-DOMPI_BUILD_CFLAGS=\"-O3 -DNDEBUG -finline-functions -mcx16\"" "-DOMPI_BUILD_CPPFLAGS=\"-iquote../../.. -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/include -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/ -I/2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/\"" "-DOMPI_BUILD_CXXFLAGS=\"-DNDEBUG \"" -DOMPI_BUILD_CXXCPPFLAGS=\"@CXXCPPFLAGS@\" -DOMPI_BUILD_FFLAGS=\"\" -DOMPI_BUILD_FCFLAGS=\"\" -DOMPI_BUILD_LDFLAGS=\"\" "-DOMPI_BUILD_LIBS=\"-lm /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_core.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/libevent_pthreads.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/libpmix.la\"" -DOPAL_CC_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang\" -DOMPI_CXX_ABSOLUTE=\"/2/sw/spack/opt/spack/linux-rocky9-x86_64_v3/gcc-11.4.1/aocc-5.0.0-7bkalkd6lei7daktomzuoomjtqf4l7wp/bin/clang++\" -O3 -DNDEBUG -finline-functions -mcx16 -o .libs/ompi_info ompi_info.o param.o ../../../ompi/.libs/libmpi.so -L/opt/mellanox/hcoll/lib -L../src/hcoll/commpatterns -L../src/hcoll/netpatterns -L/usr/lib64 -L/usr/lib /opt/mellanox/hcoll/lib/libhcoll.so /opt/mellanox/hcoll/lib/libocoms.so -lrt -lrdmacm -libverbs -lpthread /2/sw/HPC/src/openmpi-5.0.7/opal/.libs/libopen-pal.so ../../../opal/.libs/libopen-pal.so -lucp -lucs -lucm -luct /2/sw/HPC/src/openmpi-5.0.7/3rd-party/openpmix/src/.libs/libpmix.so -lmunge -ldl /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_core.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/libevent-2.1.12-stable-ompi/.libs/libevent_pthreads.so /2/sw/HPC/src/openmpi-5.0.7/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -Wl,-rpath -Wl,/2/sw/HPC/openmpi/5.0.7/lib ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __extendhfsf2 >>> referenced by ../../../ompi/.libs/libmpi.so ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __truncsfhf2 >>> referenced by ../../../ompi/.libs/libmpi.so clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [Makefile:1478: ompi_info] Error 1 make[2]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi/tools/ompi_info' make[1]: *** [Makefile:2710: all-recursive] Error 1 make[1]: Leaving directory '/2/sw/HPC/src/openmpi-5.0.7/ompi' make: *** [Makefile:1539: all-recursive] Error 1 The full report is here: https://urldefense.com/v3/__https://macdrive.mcmaster.ca/f/5879ebda7cfe49c283fe/__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSUabVlcBg$ Trying the MVAPICH-4.0-plus rpm, I Geta. Similar error: bbserv:~ $ sudo rpm --prefix=/2/sw/HPC/mvapich2/4.0-plus -Umvapich-plus-4.0-nogpu.rhel9.ofed24.10.ucx.gcc13.2.0.slurm-4.0-1.x86_64.rpmrpm Updating / installing... 1:mvapich-plus-4.0-nogpu.rhel9.ofed################################# [100%] bbserv:~ $ export PATH=/2/sw/HPC/mvapich2/4.0-plus/bin:$PATH bbserv:~ $ export LD_LIBRARY_PATH=/2/sw/HPC/mvapich2/4.0-plus/lib:$LD_LIBRARY_PATH bbserv:~ $ cd Development/SandBox/MPI bbserv:MPI $ which mpicc /2/sw/HPC/mvapich2/4.0-plus/bin/mpicc bbserv:MPI $ mpicc -o cpi cpi.c /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__extendhfsf2' /usr/bin/ld: /2/sw/HPC/mvapich2/4.0-plus/lib/libmpi.so: undefined reference to `__truncsfhf2' collect2: error: ld returned 1 exit status I have been trying to figure out which library is supposed to provide these two symbols, without luck so far. Has anybody encountered the same problem? Regards, Blaise ? Canada Research Chair in Mathematical and Computational Aspects of Solid Mechanics (Tier 1) Professor, Department of Mathematics & Statistics Hamilton Hall room 409A, McMaster University 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada https://urldefense.com/v3/__https://www.math.mcmaster.ca/bourdin__;!!KGKeukY!xyX04qOWTVd8yh6Z9MzPY_jJZIq060U7v7zcjNZL86tTYyzaiUoPssYvQpFKFn8IUywVFarDbNuAZskXkQttKSXhA-wKxg$ | +1 (905) 525 9140 ext. 27243 _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.goldman at intel.com Wed Apr 16 13:42:03 2025 From: adam.goldman at intel.com (Goldman, Adam) Date: Wed, 16 Apr 2025 17:42:03 +0000 Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 In-Reply-To: References: Message-ID: !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hello, I noticed that the CHANGES list "Intel GPU support." Is this correct for v7.5? I am not able to find a way to compile with the level-zero library and the README does not mention "Intel GPU" anywhere. Thank you, Adam Goldman -----Original Message----- From: Mvapich-discuss On Behalf Of Panda, Dhabaleswar via Mvapich-discuss Sent: Friday, November 1, 2024 7:07 PM To: Mvapich-discuss at lists.osu.edu Cc: Panda, Dhabaleswar Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 The MVAPICH team is pleased to announce the release of OSU Micro-Benchmarks (OMB) 7.5. Please note that OMB is also available through the Spack package manager. Now the system administrators and users of OSU Micro-Benchmarks (OMB) will be able to install these libraries on their systems using Spack. The new features, enhancements, and bug fixes for OSU Micro-Benchmarks (OMB) 7.5 are listed here: * New Features & Enhancements (since 7.4) - New OpenSHMEM benchmarks * osu_oshm_get_bw, osu_oshm_get_nb_bw * osu_oshm_put_bw, osu_oshm_put_nb_bw * osu_oshm_get_overlap - New partitioned point-to-point benchmkark * osu_partitioned_latency - Add NVIDIA, AMD, and Intel GPU support for neighborhood collective benchmarks * osu_neighbor_allgather, osu_neighbor_allgatherv, * osu_neighbor_alltoall, osu_neighbor_alltoallv, * osu_neighbor_alltoallw, osu_ineighbor_allgatherv, * osu_ineighbor_allgatherv, osu_ineighbor_alltoall, * osu_ineighbor_alltoallv, osu_ineighbor_alltoallw - Add Intel GPU support for the following benchmarks * Point-to-point benchmarks * osu_bibw, osu_bw, osu_mbw_mr, osu_latency, osu_multi_lat, * osu_latency_mp, osu_latency_mt, osu_bw_persistent, * osu_bibw_persistent, osu_latency_persistent, * osu_partitioned_latency * Network congestion benchmarks * osu_bw_fan_in, osu_bw_fan_out * Blocking collective benchmarks * osu_allgather, osu_allgatherv, osu_alltoall, osu_allreduce, * osu_alltoallv, osu_alltoallw, osu_bcast, osu_barrier, osu_gather, * osu_gatherv, osu_reduce, osu_reduce_scatter, osu_scatter, * osu_scatterv, osu_reduce_scatter_block * Non-Blocking collective benchmarks * osu_iallgather, osu_iallgatherv, osu_iallreduce, osu_ialltoall, * osu_ialltoallv, osu_ialltoallw, osu_ibcast, osu_ibarrier, * osu_igather, osu_igatherv, osu_ireduce, osu_iscatter, * osu_iscatterv, osu_ireduce_scatter, osu_ireduce_scatter_block * Persistent collective benchmarks * osu_allgather_persistent, osu_allgatherv_persistent, * osu_allreduce_persistent, osu_alltoall_persistent, * osu_alltoallv_persistent, osu_alltoallw_persistent, * osu_barrier_persistent, osu_bcast_persistent, * osu_gather_persistent, osu_gatherv_persistent, * osu_reduce_persistent, osu_reduce_scatter_persistent, * osu_scatter_persistent, osu_scatterv_persistent * One-sided benchmarks * osu_acc_latency, osu_cas_latency, osu_fop_latency, osu_get_bw, * osu_get_latency, osu_put_bibw, osu_put_bw, osu_put_latency, * osu_get_acc_latency - Changed default MPI_TYPE of reduce benchmarks to MPI_INT. * osu_reduce, osu_allreduce, osu_reduce_scatter, osu_reduce_scatter_block * Bug Fixes - Fixed RCCL build in ROCm 6 due to installation path changes. - Fixed a bug leading to nvcc command not found error when builing with NCCL support. * Thanks to Raghu Raja @Amazon for report. - Resolved a bug in the maximum memory limit check, which previously considered only the memory allocated by a single buffer. This check and command line option 'M' no longer exists. * Thanks to Van Man Nguyen @Eviden for report and patch. - Fixed bug while printing validation results in some collective benchmarks. - Updated reduction benchmarks to use MPI_SIGNED_CHAR instead of MPI_CHAR. * Thanks to R??mi Dehenne @CEA for report and patch. - Fixed a bug in Reduce, Reduce-Scatter benchmarks that caused validation to fail when running with ROCM support. * Thanks to Anton Daumen @Eviden for report and patch. * Thanks to Edgar Gabriel @AMD and Mustafa Abduljabbar @AMD for report. - Fixed RCCL check in configure script. * Thanks to Anton Daumen @Eviden for report. For downloading OMB 7.5 and associated README instructions, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH libraries (and registered at the MVAPICH site) has crossed 3,400 worldwide (in 92 countries). The number of downloads from the MVAPICH site has crossed 1,835,000 (1.835 million). The MVAPICH team would like to thank all its users and organizations!! _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss From chen.10252 at buckeyemail.osu.edu Wed Apr 16 21:33:42 2025 From: chen.10252 at buckeyemail.osu.edu (Chen, Chen Chun) Date: Thu, 17 Apr 2025 01:33:42 +0000 Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 In-Reply-To: References: Message-ID: Hi Adam, OMB supports Intel GPUs through SYCL. Users can enable this support by adding the following configuration flags: --enable-sycl --with-sycl=/PATH/TO/ONEAPI For example: ./configure --enable-sycl --with-sycl=/opt/intel/oneapi CC=mpicc CXX=mpicxx You?ll also find more detailed setup instructions in the README (https://mvapich.cse.ohio-state.edu/static/media/mvapich/README-OMB.txt). Some SYCL-related content can be found in the README for reference: SYCL extensions can be enabled by configuring OMB with --enable-sycl option as shown below. ./configure CC=/path/to/mpicc CXX=/path/to/mpicxx --enable-sycl --with-sycl=/path/to/sycl/install make make install ? If both CUDA and OpenACC support is enabled you can switch between the modes using the -d [cuda|openacc] option to the benchmarks. If ROCm support is enabled, you need to use -d rocm option to make the benchmarks use this feature. If SYCL support is enabled, you need to use -d sycl option to make the benchmarks use this feature. Whether a process allocates its communication buffers on the GPU device or on the host can be controlled at run-time. Use the -h option for more help. If you have any questions, feel free to reach out?we?re always happy to help! Best, Chen-Chun From: Mvapich-discuss on behalf of Goldman, Adam via Mvapich-discuss Date: Wednesday, April 16, 2025 at 1:42?PM To: mvapich-discuss at lists.osu.edu Subject: Re: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 !-------------------------------------------------------------------| This Message Is From an External Sender This message came from outside your organization. |-------------------------------------------------------------------! Hello, I noticed that the CHANGES list "Intel GPU support." Is this correct for v7.5? I am not able to find a way to compile with the level-zero library and the README does not mention "Intel GPU" anywhere. Thank you, Adam Goldman -----Original Message----- From: Mvapich-discuss On Behalf Of Panda, Dhabaleswar via Mvapich-discuss Sent: Friday, November 1, 2024 7:07 PM To: Mvapich-discuss at lists.osu.edu Cc: Panda, Dhabaleswar Subject: [Mvapich-discuss] Announcing the release of OSU Micro-Benchmarks (OMB) 7.5 The MVAPICH team is pleased to announce the release of OSU Micro-Benchmarks (OMB) 7.5. Please note that OMB is also available through the Spack package manager. Now the system administrators and users of OSU Micro-Benchmarks (OMB) will be able to install these libraries on their systems using Spack. The new features, enhancements, and bug fixes for OSU Micro-Benchmarks (OMB) 7.5 are listed here: * New Features & Enhancements (since 7.4) - New OpenSHMEM benchmarks * osu_oshm_get_bw, osu_oshm_get_nb_bw * osu_oshm_put_bw, osu_oshm_put_nb_bw * osu_oshm_get_overlap - New partitioned point-to-point benchmkark * osu_partitioned_latency - Add NVIDIA, AMD, and Intel GPU support for neighborhood collective benchmarks * osu_neighbor_allgather, osu_neighbor_allgatherv, * osu_neighbor_alltoall, osu_neighbor_alltoallv, * osu_neighbor_alltoallw, osu_ineighbor_allgatherv, * osu_ineighbor_allgatherv, osu_ineighbor_alltoall, * osu_ineighbor_alltoallv, osu_ineighbor_alltoallw - Add Intel GPU support for the following benchmarks * Point-to-point benchmarks * osu_bibw, osu_bw, osu_mbw_mr, osu_latency, osu_multi_lat, * osu_latency_mp, osu_latency_mt, osu_bw_persistent, * osu_bibw_persistent, osu_latency_persistent, * osu_partitioned_latency * Network congestion benchmarks * osu_bw_fan_in, osu_bw_fan_out * Blocking collective benchmarks * osu_allgather, osu_allgatherv, osu_alltoall, osu_allreduce, * osu_alltoallv, osu_alltoallw, osu_bcast, osu_barrier, osu_gather, * osu_gatherv, osu_reduce, osu_reduce_scatter, osu_scatter, * osu_scatterv, osu_reduce_scatter_block * Non-Blocking collective benchmarks * osu_iallgather, osu_iallgatherv, osu_iallreduce, osu_ialltoall, * osu_ialltoallv, osu_ialltoallw, osu_ibcast, osu_ibarrier, * osu_igather, osu_igatherv, osu_ireduce, osu_iscatter, * osu_iscatterv, osu_ireduce_scatter, osu_ireduce_scatter_block * Persistent collective benchmarks * osu_allgather_persistent, osu_allgatherv_persistent, * osu_allreduce_persistent, osu_alltoall_persistent, * osu_alltoallv_persistent, osu_alltoallw_persistent, * osu_barrier_persistent, osu_bcast_persistent, * osu_gather_persistent, osu_gatherv_persistent, * osu_reduce_persistent, osu_reduce_scatter_persistent, * osu_scatter_persistent, osu_scatterv_persistent * One-sided benchmarks * osu_acc_latency, osu_cas_latency, osu_fop_latency, osu_get_bw, * osu_get_latency, osu_put_bibw, osu_put_bw, osu_put_latency, * osu_get_acc_latency - Changed default MPI_TYPE of reduce benchmarks to MPI_INT. * osu_reduce, osu_allreduce, osu_reduce_scatter, osu_reduce_scatter_block * Bug Fixes - Fixed RCCL build in ROCm 6 due to installation path changes. - Fixed a bug leading to nvcc command not found error when builing with NCCL support. * Thanks to Raghu Raja @Amazon for report. - Resolved a bug in the maximum memory limit check, which previously considered only the memory allocated by a single buffer. This check and command line option 'M' no longer exists. * Thanks to Van Man Nguyen @Eviden for report and patch. - Fixed bug while printing validation results in some collective benchmarks. - Updated reduction benchmarks to use MPI_SIGNED_CHAR instead of MPI_CHAR. * Thanks to R??mi Dehenne @CEA for report and patch. - Fixed a bug in Reduce, Reduce-Scatter benchmarks that caused validation to fail when running with ROCM support. * Thanks to Anton Daumen @Eviden for report and patch. * Thanks to Edgar Gabriel @AMD and Mustafa Abduljabbar @AMD for report. - Fixed RCCL check in configure script. * Thanks to Anton Daumen @Eviden for report. For downloading OMB 7.5 and associated README instructions, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH libraries (and registered at the MVAPICH site) has crossed 3,400 worldwide (in 92 countries). The number of downloads from the MVAPICH site has crossed 1,835,000 (1.835 million). The MVAPICH team would like to thank all its users and organizations!! _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss _______________________________________________ Mvapich-discuss mailing list Mvapich-discuss at lists.osu.edu https://lists.osu.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.thompson at nasa.gov Wed Apr 30 13:42:56 2025 From: matthew.thompson at nasa.gov (Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]) Date: Wed, 30 Apr 2025 17:42:56 +0000 Subject: [Mvapich-discuss] SIGFPE on MPI_Init_thread with GCC 14? Message-ID: All, I'm hoping this oddity is something people here have seen before. I built MVAPICH 4.0 with GCC 14.2.0 and it is dying with a SIGFPE on this line: call MPI_Init_thread(MPI_THREAD_MULTIPLE, provided, ierror) Very boring. Thing is, Intel ifort 2021.13 + MVAPICH 4.0 is just fine with the same code. And I can make a simple MPI program to see if, for some reason, I can't call MPI_Init_thread with MPI_THREAD_MULTIPLE, but no, I can. Does this ring a bell for anyone? It's just...weird. I mean...SIGFPE?! Matt [signature_3763662456] Matt Thompson Lead Scientific Software Engineer/Supervisor Global Modeling and Assimilation Office Science Systems and Applications, Inc. Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 o: 301-614-6712 matthew.thompson at nasa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 63716 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 358 bytes Desc: image002.png URL: From panda at cse.ohio-state.edu Wed Apr 30 23:24:47 2025 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Thu, 1 May 2025 03:24:47 +0000 Subject: [Mvapich-discuss] MUG '25 Call for Presentations Message-ID: The MVAPICH team is excited to host the 13th annual MVAPICH User Group (MUG) conference. It will take place from August 18-20, 2025 in Columbus, Ohio, USA. It will be held in a hybrid manner. The MUG conference aims to bring together the MVAPICH users, researchers, developers, and system administrators to share their experiences, knowledge and to learn from each other. The event includes keynote talks, invited tutorials, invited talks, contributed presentations, Open Mic session, hands-on sessions with the MVAPICH developers, etc. This year, we will be holding special tutorials and demo/hands-on sessions during the first day of the event (August 18th). Other talks and sessions will be held on August 19-20. A set of short contributed presentations from the MVAPICH users will be included in the event. Topics for presentations include, but are not limited to: - Case studies and best practices of novel applications from different application domains, such as: astronomy, bioinformatics, biology, earth and atmospheric sciences, fluid dynamics, materials science and engineering, medicine, physics, and AI (machine learning, deep learning (training and inference)) - Performance and scalability studies of applications on large-scale systems - Special tuning and optimization strategies to exploit maximum performance and scalability - Tools and code instrumentation for measuring and monitoring performance and/or resilience - Tools for parallel program development (e.g. debuggers and integrated development environments) - Unique usage scenarios with GPUs, DPUs, APUs, FPGAs, Energy-Awareness, Virtualization, Quantum simulation, etc. The submission should include the title of the presentation, speaker(s), short bio of the speaker(s), and a draft version of the presentation (around 10-15 slides in PDF or PowerPoint format). Please send your submission in a single file to mug at cse.ohio-state.edu. Presentation Submission Deadline: July 11, 2025 Notification of Acceptance: July 18, 2025 More details on the conference and Call for Presentations are available from http://mug.mvapich.cse.ohio-state.edu/ Thanks, The MVAPICH Team