From panda at cse.ohio-state.edu Wed Aug 3 01:04:47 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Wed, 3 Aug 2022 05:04:47 +0000 Subject: [Mvapich] The Preliminary Program for MUG '22 Conference is now available In-Reply-To: References: Message-ID: The preliminary program for the 10th annual MVAPICH User Group (MUG) conference is now available from http://mug.mvapich.cse.ohio-state.edu/program/ The highlights of the conference include: - Two keynote talks (Prof. Taisuke Boku from Univ. of Tsukuba (Japan) and Dr. Dan Stanzione, TACC) - Eight tutorials and demos (Broadcom, Cornelis Networks, NVIDIA, OSU, and X-ScaleSolutions) - 17 invited talks (Broadcom, Cornelis Networks, Dell, INL, LLNL, Microsoft Azure, NERSC, NSF, NVIDIA, ParaTools, SDSC, The Ohio State University, Univ. of Cambridge (UK), Univ. of Oregon, Univ. of Tennessee, Knoxville, and X-ScaleSolutions) - 12 student poster presentations (Northeastern University; The Ohio State University; The University of Texas, San Antonio; Univ. of Illinois at Chicago; Univ. of Oregon; and Virginia Polytechnic Institute and State University) - 7 short talks (The MVAPICH group) The event is being held in a hybrid manner during August 22-24, 2022 in Columbus, Ohio, USA. Interested in attending the conference? More information on registration (in-person and online), hotel accommodation, and travel is available from http://mug.mvapich.cse.ohio-state.edu/ Thanks, The MVAPICH Team The MUG conference is proud to be sponsored by Broadcom, Cornelis Networks, NSF, NVIDIA, Ohio Supercomputer Center, The Ohio State University, ParaTools, and X-ScaleSolutions. -------------- next part -------------- An HTML attachment was scrubbed... URL: From panda at cse.ohio-state.edu Thu Aug 11 23:50:53 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Fri, 12 Aug 2022 03:50:53 +0000 Subject: [Mvapich] Announcing the Release of OSU InfiniBand Analysis and Monitoring (INAM) Tool v0.9.8 Message-ID: The MVAPICH team is pleased to announce the release of OSU InfiniBand Network Analysis and Monitoring (INAM) Tool v0.9.8. OSU INAM monitors InfiniBand clusters in real time by querying various subnet management entities in the network. It is also capable of interacting with the MVAPICH2-X software stack to gain insights into the communication pattern of the application and classify the data transferred into Point-to-Point, Collective and Remote Memory Access (RMA). OSU INAM can also remotely monitor several parameters of MPI processes in conjunction with MVAPICH2-X. OSU INAM v0.9.8 (08/11/2022) * Major Features & Enhancements (since 0.9.7): - Support for MySQL and InfluxDB as database backends - Enhanced database insertion using InfluxDB - MPI_T Performance Variables (PVARs) - Fabric topology - InfiniBand port data counters and errors - Support for continuous queries to improve visualization performance - Support for SLURM multi-cluster configuration - Significantly improved database query performance when using InfluxDB resulting in improvements for the following: - Live switch page - Live jobs page - Live node page - Historic switch page - Historic node pages - Historic jobs pages - Support for automatic data retention policy when using InfluxDB For downloading OSU INAM v0.9.8 and associated user guide, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform that the number of organizations using MVAPICH2 libraries (and registered at the MVAPICH site) has crossed 3,275 worldwide (in 90 countries). The number of downloads from the MVAPICH site has crossed 1,616,000 (1.61 million). The MVAPICH team would like to thank all its users and organizations!! From panda at cse.ohio-state.edu Mon Aug 15 19:24:30 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Mon, 15 Aug 2022 23:24:30 +0000 Subject: [Mvapich] Announcing the release of MVAPICH2 3.0a Message-ID: The MVAPICH team is pleased to announce the release of MVAPICH2-3.0a. The MVAPICH2-3.0a is the first release based on MPICH 3.4.3 series with support for the new and enhanced CH4 channel. MVAPICH2-3.0a brings in support for various modern interconnects such as Slingshot-11 from HPE/Cray, Omni-Path Express (OPX) from Cornelis networks, and the Columbiaville series of network adapters from Intel. MVAPICH2-3.0a also supports the OFI and UCX communication devices natively in combination with MVAPICH2's advanced collective communication capabilities. The major features and enhancements available in MVAPICH2-3.0a are as follows: - Based on MPICH 3.4.3 - Added support for the ch4:ucx and ch4:ofi devices - Support for MVAPICH2 enhanced collectives over OFI and UCX - Added support for the Cray Slingshot 11 interconnect over OFI - Supports Cray Slingshot 11 network adapters - Added support for the Cornelis OPX library over OFI - Supports Intel Omni-Path adapters - Added support for the Intel PSM3 library over OFI - Supports Intel Columbiaville network adapters - Added support for IB verbs over UCX - Supports IB and RoCE network adapters - Disabled ch3:mrail device - Removed ch3:psm device For downloading MVAPICH2-3.0a library and associated user guides, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform that the number of organizations using MVAPICH2 libraries (and registered at the MVAPICH site) has crossed 3,275 worldwide (in 90 countries). The number of downloads from the MVAPICH site has crossed 1,616,000 (1.61 million). The MVAPICH team would like to thank all its users and organizations!! From panda at cse.ohio-state.edu Fri Aug 19 00:44:58 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Fri, 19 Aug 2022 04:44:58 +0000 Subject: [Mvapich] Final Program for MUG '22 Conference is now available Message-ID: The final program for the 10th annual MVAPICH User Group (MUG) conference is now available from http://mug.mvapich.cse.ohio-state.edu/program/ Details of the conference include: - Two Keynote Talks 1. Cygnus-D: The Big Memory Supercomputer for HPC, Big Data and AI, Taisuke Boku from Univ. of Tsukuba (Japan) 2. On the Horizon ? Interconnects in Frontera and it?s coming replacement system, Dan Stanzione, TACC - Eight Tutorials and Demos 1. OPX- A High-Performance libfabrics provider for Omni-Path Networks, Dennis Dalessandro, Cornelis Networks 2. Offloading Collective Operations to the BlueField DPU, Richard Graham, NVIDIA/Mella 3. A Tutorial on HPC and ML Communication Benchmarking, Moshe Voloshin, Broadcom 4. Accelerating HPC Applications with MVAPICH2-DPU and Live Demos, Donglai Dai and Kyle Schaefer, X-ScaleSolutions 5. Boosting Performance of HPC Applications with MVAPICH2, Hari Subramoni and Nat Shineman, The Ohio State University 6. Visualize, Analyze, and Correlate Networking Activities for Parallel Programs on InfiniBand and HPC Clusters using the OSU INAM Tool, Hari Subramoni and Pouya Kousha, The Ohio State University 7. High Performance Machine Learning and Deep Learning with MVAPICH2, Aamir Shafi and Arpan Jain, The Ohio State University 8. Benchmarking Parallel Python and Java Applications using OMB and MVAPICH2, Aamir Shafi and Nawras Alnaasan, The Ohio State University - 18 Invited Talks 1. Overview of the MVAPICH Project and Future Roadmap, Dhabaleswar K (DK) Panda, The Ohio State University 2. Applying MPI to Manage HPC-scale Datasets, Adam Moody, Lawrence Livermore National Laboratory (LLNL) 3. Aggressive Asynchronous Communication in the MOOSE framework using MVAPICH2, Idaho National Laboratory (INL) 4. A Deep Dive into DPU Computing - Addressing HPC/AI Performance Bottlenecks, Gilad Shainer, NVIDIA 5. DMA Software Support for Broadcom Ethernet NICs, Hemal Shah, Broadcom 6. MVAPICH2 at Azure: Enabling High Performance on Cloud, Jithin Jose, Microsoft Azure 7. Cyberinfrastructure Research, Learning and Workforce Development (LWD) Programs at NSF, Ashok Srinivasan, NSF 8. Performance Engineering using MVAPICH and TAU, Sameer Shende, ParaTools and University of Oregon 9. Performance of Applications using MVAPICH2 and MVAPICH2-GDR on SDSC's Expanse Supercomputer, Mahidhar Tatineni, San Diego Supercomputer Center (SDSC) 10. Introduction to Cornelis Networks and the Omni-Path Architecture, Douglas Fuller, Cornelis Networks 11. Offloading MPI collectives to DPU in a real HPC application: the Xcompact3D proof-of-concept, Filippo Spiga, NVIDIA 12. HPC platform efficiency for large-scale workloads, Martin Hilgerman, Dell 13. MVAPICH at the Cambridge Open Zettascale Lab?, Christopher Edsall, University of Cambridge, UK 14. FFT Computation towards Exascale, Alan Ayala and Stan Tomov, The University of Tennessee, Knoxville 15. Solving MPI Integration problems with Spack, Greg Becker, Lawrence Livermore National Laboratory (LLNL) 16. MVAPICH2 at NERSC, Shazeb Siddiqui (NERSC), Sameer Shende (ParaTools), and Prathmesh Sambrekar, NERSC 17. Accelerating HPC and DL applications using MVAPICH2-DPU library and X-ScaleAI package, Donglai Dai, X-ScaleSolutions 18. MPI4Spark: A High-Performance Communication Framework for Spark using MPI, Aamir Shafi, The Ohio State University - 12 Student Poster Presentations 1. Jurdana Masuma Iqrah, University of Texas at San Antonio, Auto-labeling Sea Ice and Open Water Segmentation and Classification for Sentinel-2 Satellite Imagery in Polar Regions 2. Ahmad Hossein Yazdani, Virginia Polytechnic Institute and State University, Profiling User I/O Behavior for Leadership Scale HPC Systems 3. Jordi Alcaraz Rodriguez, University of Oregon, Performance Engineering using MVAPICH and TAU via the MPI Tools Interface 4. Buddhi Ashan, Mallika Kankanamalage, The University of Texas at San Antonio, Heterogeneous Parallel and Distributed Computing for Efficient Polygon Overlay Computation over Large Polygonal Datasets 5. Hasanul Mahmud, The University of Texas at San Antonio, Toward an Energy-efficient framework for DNN inference at the Edge 6. Sunyu Yao, Virginia Polytechnic Institute and State University, GenFaaS: Automated FaaSification of Monolithic Workflows 7. Yao Xu, Northeastern University, A Hybrid Two-Phase-Commit Algorithm in Checkpointing Collective Communications 8. Christopher Holder, Florida State University, Layer 2 Scaling 9. Nawras Alnaasan, The Ohio State University, OMB-Py: Python Micro-Benchmarks for Evaluating Performance of MPI Libraries and Machine Learning Applications on HPC System 10. Pouya Kousha, The Ohio State University, Cross-layer Visualization of Network Communication for HPC Clusters 11. Shulei Xu, The Ohio State University, HPC Meets Clouds: MPI Performance Characterization & Optimization on Emerging HPC Cloud System 12. Tu Tran, The Ohio State University, Designing Hierarchical Multi-HCA Aware Allgather in MPI - 7 Short Talks (The MVAPICH group, The Ohio State University) 1. High Performance MPI over Slingshot, Kawthar Shafie Khorassani 2. Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters, Qinghua Zhou 3. ?Hey CAI? - Conversational AI Enabled User Interface for HPC Tools, Pouya Kousha 4. Hybrid Five-Dimensional Parallel DNN Training for Out-of-core Models, Arpan Jain 5. Highly Efficient Alltoall and Alltoallv Communication Algorithms for GPU Systems , Chen-Chun Chen 6. Network Assisted Non-Contiguous Transfers for GPU-Aware MPI Libraries, Kaushik Kandadi Suresh 7. Towards Architecture-aware Hierarchical Communication Trees on Modern HPC Systems The event is being held in a hybrid manner during August 22-24, 2022, in Columbus, Ohio, USA. Interested in attending the conference? More information on registration (in-person and online), hotel accommodation, and travel is available from http://mug.mvapich.cse.ohio-state.edu/ Thanks, The MVAPICH Team The MUG conference is proud to be sponsored by Broadcom, Cornelis Networks, NSF, NVIDIA, Ohio Supercomputer Center, The Ohio State University, ParaTools, and X-ScaleSolutions. -------------- next part -------------- An HTML attachment was scrubbed... URL: From panda at cse.ohio-state.edu Fri Aug 19 23:24:19 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Sat, 20 Aug 2022 03:24:19 +0000 Subject: [Mvapich] Announcing the release of OSU Micro-Benchmarks (OMB) 6.0 Message-ID: The MVAPICH team is pleased to announce the release of OSU Micro-Benchmarks (OMB) 6.0. OSU Micro-Benchmarks 6.0 introduces support for Java and Python pt2pt and collective benchmarks. The new features, enhancements, and bug fixes for OSU Micro-Benchmarks (OMB) 6.0 are listed here: * New Features & Enhancements (since v5.9) - Add support for Java pt2pt benchmarks * OSULatency - Latency Test * OSUBandwidth - Bandwidth Test - OSUBandwidthOMPI - Bandwidth Test for Open MPI Java Bindings * OSUBiBandwidth - Bidirectional Bandwidth Test - OSUBiBandwidthOMPI - BiBandwidth Test for Open MPI Java Bindings - Add support for Java collective benchmarks * OSUAllgather - MPI_Allgather Latency Test * OSUAllgatherv - MPI_Allgatherv Latency Test * OSUAllReduce - MPI_Allreduce Latency Test * OSUAlltoall - MPI_Alltoall Latency Test * OSUAlltoallv - MPI_Alltoallv Latency Test * OSUBarrier - MPI_Barrier Latency Test * OSUBcast - MPI_Bcast Latency Test * OSUGather - MPI_Gather Latency Test * OSUGatherv - MPI_Gatherv Latency Test * OSUReduce - MPI_Reduce Latency Test * OSUReduceScatter - MPI_Reduce_scatter Latency Test * OSUScatter - MPI_Scatter Latency Test * OSUScatterv - MPI_Scatterv Latency Test - Add support for Python pt2pt benchmarks * osu_latency - Latency Test * osu_bw - Bandwidth Test * osu_bibw - Bidirectional Bandwidth Test * osu_multi_lat - Multi-pair Latency Test - Add support for Python collective benchmarks * osu_allgather - MPI_Allgather Latency Test * osu_allgatherv - MPI_Allgatherv Latency Test * osu_allreduce - MPI_Allreduce Latency Test * osu_alltoall - MPI_Alltoall Latency Test * osu_alltoallv - MPI_Alltoallv Latency Test * osu_barrier - MPI_Barrier Latency Test * osu_bcast - MPI_Bcast Latency Test * osu_gather - MPI_Gather Latency Test * osu_gatherv - MPI_Gatherv Latency Test * osu_reduce - MPI_Reduce Latency Test * osu_reduce_scatter - MPI_Reduce_scatter Latency Test * osu_scatter - MPI_Scatter Latency Test * osu_scatterv - MPI_Scatterv Latency Test * Bug Fixes (since v5.9) - Fix bug in data validation support for CUDA managed memory benchmarks - Thanks to Chris Chambreau @LLNL for the report For downloading OMB 6.0 and associated README instructions, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches, and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at lists.osu.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH2 libraries (and registered at the MVAPICH site) has crossed 3,275 worldwide (in 90 countries). The number of downloads from the MVAPICH site has crossed 1,618,000 (1.61 million). The MVAPICH team would like to thank all its users and organizations!! From panda at cse.ohio-state.edu Sun Aug 21 23:58:10 2022 From: panda at cse.ohio-state.edu (Panda, Dhabaleswar) Date: Mon, 22 Aug 2022 03:58:10 +0000 Subject: [Mvapich] Announcing the release of MVAPICH2-X-AWS 2.3.7 for Amazon AWS Cloud Message-ID: The MVAPICH team is pleased to announce the release of MVAPICH2-X-AWS 2.3.7. This version is targeted for Amazon AWS cloud with direct support for the Elastic Fabric Adapter (EFA). Features and enhancements for MVAPICH2-X-AWS 2.3.7 are listed here. MVAPICH2-X-AWS-2.3.7 (08/21/2022) *Features and Enhancements - Based on MVAPICH2-X - Improved inter-node latency & bandwidth performance - Add initial support for AWS hpc6a/c6a instances with 3rd generation AMD EPYC processors - Add support & performance optimization for AWS c6g/c7g with Amazon Graviton 2/3 ARM processors - Add initial support to rdma_read feature for AWS p4d instance type - Support for currently available basic OS types on AWS EC2 including: Amazon Linux 1/2, CentOS 7, Ubuntu 18.04/20.04 *Bug Fixes: - Fix issues of incorrectly detecting HCA devices on parallel cluster v3 - Fix issues with mpirun_rsh launcher - Fix issues related to qp creation on several instance types For downloading MVAPICH2-X-AWS-2.3.7 and associated user guide, please visit the following URL: http://mvapich.cse.ohio-state.edu All questions, feedback, bug reports, hints for performance tuning, patches and enhancements are welcome. Please post it to the mvapich-discuss mailing list (mvapich-discuss at cse.ohio-state.edu). Thanks, The MVAPICH Team PS: We are also happy to inform you that the number of organizations using MVAPICH2 libraries (and registered at the MVAPICH site) has crossed 3,275 worldwide (in 90 countries). The number of downloads from the MVAPICH site has crossed 1,618,000 (1.61 million). The MVAPICH team would like to thank all its users and organizations!!