[mvapich-discuss] mvapich-discuss Digest, Vol 176, Issue 6

[Ghazimirsaeed, Seyedeh Mahdieh]

Hi Grover,

We installed MVAPICH with the same configuration flags you mentioned on our local cluster but we couldn't reproduce it with WRF 3.6.

Can you please use this flag " --disable-mcast" for configuring MVAPICH and see if it fixes this failure.

Thanks,
Mahdieh

On 8/21/20, 12:00 PM, "mvapich-discuss on behalf of mvapich-discuss-request at cse.ohio-state.edu" <mvapich-discuss-bounces at cse.ohio-state.edu on behalf of mvapich-discuss-request at cse.ohio-state.edu> wrote:

    Send mvapich-discuss mailing list submissions to
    	mvapich-discuss at cse.ohio-state.edu
    
    To subscribe or unsubscribe via the World Wide Web, visit
    	https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.cse.ohio-state.edu%2Fmailman%2Flistinfo%2Fmvapich-discuss&data=02%7C01%7Cs.ghazimirsaeed%40queensu.ca%7C69256341b28740659d7708d845eb5d28%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C637336224470541394&sdata=lI5jq83YI5eEQCEWtLBY1A19INUF76NtNF4GBGUysss%3D&reserved=0
    or, via email, send a message with subject or body 'help' to
    	mvapich-discuss-request at cse.ohio-state.edu
    
    You can reach the person managing the list at
    	mvapich-discuss-owner at cse.ohio-state.edu
    
    When replying, please edit your Subject line so it is more specific
    than "Re: Contents of mvapich-discuss digest..."
    
    
    Today's Topics:
    
       1. mvapich2.3.4 mcast error (Gong-Do Hwang)
    
    
    ----------------------------------------------------------------------
    
    Message: 1
    Date: Thu, 20 Aug 2020 17:25:04 +0800
    From: Gong-Do Hwang <grover.hwang at gmail.com>
    To: mvapich-discuss at cse.ohio-state.edu
    Subject: [mvapich-discuss] mvapich2.3.4 mcast error
    Message-ID:
    	<CAFF9WUs=cJ87BM7EbXGkdZNvx5XHLQY4uxK3gERxiJkezzTaoQ at mail.gmail.com>
    Content-Type: text/plain; charset="utf-8"
    
    Hi,
    
    I was using mvapich2.3.4 to run WRF 4.1.5 over MLNX OFED 5.0. And I found
    it when using node larger than 8 I had error message below when the WRF
    integration started:
    
    [cn01:mpi_rank_0][mv2_mcast_resend_window] Failed to post mcast send
    errno:12
    [cn01:mpi_rank_0][mv2_mcast_resend_window] Failed to post mcast send
    errno:12
    [cn01:mpi_rank_0][mv2_mcast_resend_window] Failed to post mcast send
    errno:12
    
    mvapich2 was configured with the following flags:
    ./configure --prefix=$prefix --with-device=ch3:mrail  --with-rdma=gen2
    --enable-threads=multiple  --enable-rdma-cm --enable-threads=multiple
    --enable-romio --with-ch3-rank-bits=32  --with-ib-include=/usr/include
    --with-ib-libpath=/usr/lib64 --with-ibverbs-include=/usr/include
    --with-ibverbs-lib=/usr/lib64 CC=icc CFLAGS="-fPIC" F77=ifort
    FFLAGS="-fPIC" FC=ifort FCFLAGS="-fPIC" CXX=icpc CXXFLAGS="-fPIC"
    
    and run script and ARGS:
    
    export MV2_ENABLE_AFFINITY=0
    export MV2_IBA_HCA=mlx5_0
    mpiexec -rmk pbs  -np ${NPROCS}   ./wrf.exe
    
    I assigned the HCA because we have another mlx ehternet hca on the compute
    node.
    
    Is there any place I can find what error no =12 means? And is there any
    workaround for this?
    Thanks so much for your help!
    
    Grover
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.cse.ohio-state.edu%2Fpipermail%2Fmvapich-discuss%2Fattachments%2F20200820%2Ffdf09584%2Fattachment-0001.html&data=02%7C01%7Cs.ghazimirsaeed%40queensu.ca%7C69256341b28740659d7708d845eb5d28%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C637336224470541394&sdata=H%2Bm5ZkCh9YaoENuPEQtOdildM7xAvVjHDZXjD0oiY7s%3D&reserved=0>
    
    ------------------------------
    
    Subject: Digest Footer
    
    _______________________________________________
    mvapich-discuss mailing list
    mvapich-discuss at cse.ohio-state.edu
    https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.cse.ohio-state.edu%2Fmailman%2Flistinfo%2Fmvapich-discuss&data=02%7C01%7Cs.ghazimirsaeed%40queensu.ca%7C69256341b28740659d7708d845eb5d28%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C637336224470541394&sdata=lI5jq83YI5eEQCEWtLBY1A19INUF76NtNF4GBGUysss%3D&reserved=0
    
    
    ------------------------------
    
    End of mvapich-discuss Digest, Vol 176, Issue 6
    ***********************************************