[mvapich-discuss] Performance issue

[Dhabaleswar Panda]

Hi,

Thanks for your note. Could you tell us whether you are using single-rail
or multi-rail environment. If you are using multi-rail environment, let us
know the number of rails you are using and the configuration/run-time
options you are using (there are several configurations and run-time
options available in 1.7a). Is this being run on the Tsukuba cluster?

Typically MPI library has a rendezvous threshold. Similarly, there are
thresholds for multi-rail configurations. These thresholds need to be
adjusted properly to get the best performance for different message sizes.
If you can provide answers to some of the above questions, it will be
helpful for us to analyze this issue further.

Thanks,

DK

On Sun, 5 Jun 2011, Masahiro Nakao wrote:

> Dear all,
>
> I use mvapich2-1.7a.
> I tried to measure a troughtput performance,
> but the value of performance I don't understand.
>
> The source code is as below.
> ---
> double tmp[SIZE];
>   :
> MPI_Barrier(MPI_COMM_WORLD);
> t1 = gettimeofday_sec();
> if(rank==0)   MPI_Send(tmp, SIZE, MPI_DOUBLE, 1, 999, MPI_COMM_WORLD );
> else  MPI_Recv(tmp, SIZE, MPI_DOUBLE, 0, 999, MPI_COMM_WORLD, &status );
> MPI_Barrier(MPI_COMM_WORLD);
> t2 = gettimeofday_sec();
>
> printf("%d Byte:%.1f MByte/s\n", sizeof(tmp), sizeof(tmp)/(t2-t1)/1000000);
> ---
> This program was run on 2 nodes.
>
> Results are as below. SIZE = 1, 2, 4, ... , 4096
> ---
>     8 Byte:  0.6 MByte/s
>    16 Byte:  1.1 MByte/s
>    32 Byte:  2.1 MByte/s
>    64 Byte:  4.5 MByte/s
>   128 Byte:  8.0 MByte/s
>   256 Byte: 15.1 MByte/s
>   512 Byte: 30.2 MByte/s
>  1024 Byte: 68.2 MByte/s
>  2048 Byte:102.3 MByte/s
>  4096 Byte:157.6 MByte/s
>  8192 Byte:195.2 MByte/s
> 16384 Byte: 80.8 MByte/s
> 32768 Byte:142.4 MByte/s
> ---
>
> Why is the troughtput down, when the value of transfar size is 16384 ?
>
> My environment is as below.
> ---
> - Quad-Core AMD Opteron(tm) Processor 8356
> - DDR Infiniband (Mellanox ConnectX)
> ---
>
> Regard,
> --
> Masahiro NAKAO
> Email : mnakao at ccs.tsukuba.ac.jp
> Researcher
> Center for Computational Sciences
> UNIVERSITY OF TSUKUBA
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>