[mvapich-discuss] mvapich2 error

[Dhabaleswar Panda]

Dr. Kandadai:

Thanks for your note. Do you see this error without KNEM?

Please note that MVAPICH2 does not officially support KNEM. It supports
LiMIC2 as the kernel module. For errors related to KNEM, please discuss it
with KNEM and MPICH2 groups. If you see any issues with MVAPICH2 and
MVAPICH2-LiMIC2, please let us know and we will be happy to extend help.

Thanks,

DK


On Tue, 22 Feb 2011, Swamy Kandadai wrote:

>
> Hi:
>
> I built MVAPICH2-1.6rc2 with KNEM support on a cluster of AMD Magny-Cours.
> I am running HPCC benchmark on 4 nodes (4*48=192 MPI tasks).
>
> The application runs successfully for all the benchmarks and fails while
> running the LINPACK benchmark.
> Here is the error:
>
> Fatal error in MPI_Recv: Message truncated, error stack:
> MPIDI_CH3U_Receive_data_found(257): Message from rank 0 and tag 1001
> truncated; 24576 bytes received but buffer size is 20736
>
> In one of the discussions, I saw that the problem is with the IB vendor
> stack and works well with Mellanox. Since, we are having
> Voltaire switch and Mellanox adapters, I tried to unset the environmental
> variable   MV2_USE_RDMA_FAST_PATH and still the
> application fails in the Linpack calculation.
>
> Any suggestions?
> Thanks
> Swamy Kandadai
>
> This is the ibstat info:
> CA 'mlx4_0'
>         CA type: MT26428
>         Number of ports: 2
>         Firmware version: 2.7.0
>         Hardware version: b0
>
> This is how I configured with KNEM:
>
> --with-device=ch3:nemesis \
>             --with-nemesis-local-lmt=knem \
>             --with-knem=/opt/knem
>
>
>
>
>
>
> Dr. Swamy N. Kandadai
> IBM Senior Certified Executive IT Specialist
> STG WW  Modular Systems Benchmark Center
> STG WW HPC and BI CoC Benchmark Center
> Phone:( 845) 433 -8429 (8-293) Fax:(845)432-9789
> swamy at us.ibm.com
> http://w3.ibm.com/sales/systems/benchmarks
>
>
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>