[mvapich-discuss] Correct Configuration options for MVAPICH2 on AMD Opteron 6136(8-core) processor.

[Dhabaleswar Panda]

Hi,

Thanks for your note. Intra-node communication performance can be affected
by multiple factors:

1. Cache and Memory speed. This can vary across platforms.

2. CPU mapping to cores (Please see Section 6.4.3 of MVAPICH2 user guide
for details)

http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.7rc1.html#x1-480006.4.3

3. You can experiment with different combinations of CPU mapping to cores,
as indicated in Section 6.4.2 of the user guide, and see if it leads to
different performance numbers.

http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.7rc1.html#x1-470006.4.2

Let us know whether different CPU mappings lead you to different
performance numbers on your platform.

Thanks,

DK




On Fri, 16 Sep 2011, J. Lakshmi wrote:

> Dear sir,
>     I seem to be getting results below expectation on my cluster for the osu-benchmarks. I am enclosing the details for your comments. Am I missing something?
>
> SERC cluster with mvapich2-1.7.rc1 configuration with intel compiler: CC=icc CXX=icpc  FC=ifort –prefix=/opt/mv1.7 –with-device=ch3:mrail –with-rdma=gen2 –enable-shared
> Installed path -- /opt/mv1.7
>
>                     Benchmark(Ohio website*)              SERC-cluster
>
> Proc series          6174                                   6136
> No.of proc           24                                      32
> Freq.  (GHz)         2.2                                     2.4
> OS                   Red hat 5.5                            Cent OS 5.4
> Latency (4 bytes)   .68us                                 .77us     (intra-socket)
> Latency (4 bytes)    .88us                                1.02us   (inter-socket)
> Uni b/w(MB/sec)     1500for 1 MB                   970.52(for 1MB) intra
> Uni b/w(MB/sec)     1450for 1 MB                   984.80(for 1MB)inter
> bi b/w(MB/sec)       1900for 1 MB                   1291.34(for 1MB)intra
> bi b/w(MB/sec)       1850for 1 MB                   1093.30(for 1MB)inter
>
> cluster benchmark for intranode compared with osu_benchmark of ohio university
> *http://mvapich.cse.ohio-state.edu/performance/mvapich2/intra_magny.shtml
>
>
> Thankyou,
> with warm regards,
> Lakshmi J
>
> -----Mrs J. Lakshmi/serc/iisc wrote: -----To: panda at cse.ohio-state.edu
> From: Mrs J. Lakshmi/serc/iisc
> Date: 09/08/2011 10:24AM
> Cc: mvapich-discuss at cse.ohio-state.edu
> Subject: Re: [mvapich-discuss] Correct Configuration options for MVAPICH2 on	AMD Opteron 6136(8-core) processor.
>
> Thankyou for the information. I shall get the configuration done and execute the benchmarks for assessment.
>
> with warm regards,
> Lakshmi J
>
> -----mvapich-discuss-bounces at cse.ohio-state.edu wrote: -----To: "Mrs J. Lakshmi" <jlakshmi at serc.iisc.in>
> From: Dhabaleswar Panda
> Sent by: mvapich-discuss-bounces at cse.ohio-state.edu
> Date: 09/07/2011 06:37PM
> Cc: mvapich-discuss <mvapich-discuss at cse.ohio-state.edu>
> Subject: Re: [mvapich-discuss] Correct Configuration options for MVAPICH2 on	AMD Opteron 6136(8-core) processor.
>
> You do not need any special configuration for Opteron platform. MVAPICH2
> automatically detects the platform architecture, number of cores, etc. and
> carries out internal tuning. Since you are using Mellanox QDR adapters and
> switch, you can select the OFA-IB-CH3 interface to get the best set of
> features, performance and scalability. You can use the default
> configuration scheme (as given below or with the shared library option).
>
> ./configure --with-device=ch3:mrail --with-rdma=gen2
>
> This is outlined in Section 4.4 of the user guide at the following URL:
>
> http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.7rc1.html#x1-100004.4
>
> Regarding the compiler issue you have indicated, the performance of
> communication steps in a MPI library is not very sensitive to the compiler
> being used. However, computation steps can be optimized by different
> compilers. Thus, you can use either Intel or Open64.
>
> Hope this helps.
>
> Thanks,
>
> DK
>
> On Wed, 7 Sep 2011, Mrs J. Lakshmi wrote:
>
> > Hi,
> >   We are trying to install MVAPICH2 for our cluster built on 4-cpu, 8-core AMD 6136 processors interconnected with IB-HBAs through a Mellanox MIS5023 QDR switch. the cluster has 9 nodes. Can someone help us with the correct configuration options for using MVAPICH2 1.7 version on this cluster? Our environment requires use of different compilers like gnu, intel and open64. Which of these would run better for this cluster?
> >
> > Thankyou,
> > with warm regards,
> > Lakshmi J
> >
> > --
> > This message has been scanned for viruses and
> > dangerous content by MailScanner, and is
> > believed to be clean.
> >
> >
> > _______________________________________________
> > mvapich-discuss mailing list
> > mvapich-discuss at cse.ohio-state.edu
> > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
> >
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>
>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>