[mvapich-discuss] MVAPICH2 Derived datatypes performance issue (vs. MVAPICH)

[Eric A. Borisch]

Good afternoon all,

I have been trying to understand some performance disparities between
two MPI installations that go well beyond the expected multirail
differences. The problem seems to come from derived datatype
performance. My two test conditions are:

 (A) MVAPICH (VAPI_multirail) and
 (B) MVAPICH2 (VAPI)

To make sure I wasn't just doing something odd in my code, and
hopefully make it easy for others to test as well, I've used the
"transp-datatype.c" test that can be found in an mpich2-1.0.5
distribution in "mpich2-1.0.5/test/mpi/perf/transp-datatype.c"

I bumped the SIZE up to 1024 on a side for the arrays to be transposed
& communicated -- other than that I've taken the code as is...

Running under MVAPICH, the communication & transpose via derived
datatypes takes ~55msec, while on MVAPICH2, it takes ~1300msec. This
is not a typo -- it takes 24x longer on MVAPICH2. If you try running
the code, this is the first value that is returned (in seconds) on
execution.

The straight-through (no transpose or derived datatypes) test shows
reasonable timings of 5.8ms and 8.9ms -- ~1350 MB/s and ~900 MB/s --
respectively.

These tests are performed on the same two machines in each case. Each
is a dual 3.4G P4, with a PCI-E dual-port SDR card and 4GB of ram;
running ROCKS 4.1.

Any thoughts or suggestions?

Thanks in advance,
 Eric Borisch
 Radiology Research
 Mayo Clinic

-- 
Eric A. Borisch
eborisch at ieee.org