[mvapich-discuss] Problem with multi-threaded performance

[David Minor]

I'm in the process of moving from MPICH2 to MVAPICH2 (in order to take
advantage of IB).  My program runs on a cluster of multi-core/smp
machines and runs multi-threaded on each of those machines. In MPICH2
I'm getting acceptable multi-threaded performance, but the moment I
compile/link with MVAPICH performance deteriorates drastically. I'm
comparing here only performance over a tcp/ip 1Gbt network.  Looking at
process usage it appears that all the threads are blocking on something
and doing very little work in parallel.  I'm compiling with
--enable-threads=multiple --with-pm=mpd --without-mpe
--enable-error-checking=no --enable-g=dbg --enable-fast, and calling
MPI::Init_thread(MPI_THREAD_MULTIPLE).  I'm compiling MPICH2 with
exactly the same parameters and running exactly the same code on the
same network. A simple nm on your libraries indicates you use
pthread_mutex_lock a lot more than MPICH2, but of course that may not be
relevant!  Please advise.

David Minor

Orbotech

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20070327/0175d7ed/attachment-0001.html