[mvapich-discuss] Parameters for build

[LEI CHAI]

Dr. Potts,

 > >    Is there more info or a writeup on the magnitude of the importance
 > >    of the VCLUSTER value?  I am trying to assist a computer
 > > manufacturer    in their adoption of MVAPICH and need to understand
 > > how dramatic an
 > >    effect users might observe in latency/BW if, e.g.,  we were to
 > > choose    SMALL and the system contained 2K nodes or, the reverse,
 > > we chose
 > >    LARGE for a 100 node cluster system.

In the case you have a large cluster and choose the
"small" configuration, there won't be performance
degradation in latency/bw, but will consume more
memory each process. In the opposite case if you
choose "large" configuration for a small cluster, bw
will drop about 20% for medium sized messages (say
about 1K-8K). Latency should not be affected very
much. The absolute numbers may not be accurate, it
also depends on the system. This is just to give a 
rough idea  :-)

 > >    BTW, did your numbers of <256, 256-1024, >1024 refer to nodes
 > > or to
 > >    cpus?  Given a system with 2-4 cpus per node, it makes a
 > > difference.

We refer to the number of CPUs.

 > >    I note that OFED builds of MVAPICH don't even ask one to
 > >    choose a size, so that implies a potentially bad default for any
 > >    given system.

The OFED builds use the "gen2" device of MVAPICH, in
which an adaptive approach is used to adjust the memory consumption at run time. So it should give
good performance and scalability for any given
system. This advanced approach is not available
in the "udapl" device yet. We will work on that in
the future.

Thanks.
Lei