[mvapich-discuss] Problem running directly on IB with ofed

[Matthew Koop]

> I've got it MULTI_THREADED was the problem! I take it the openib
> driver is not multi-threaded and so when you do MPI::Init with the
> MULTIPLE_THREAD options it chooses the ipoib driver. Now does this
> mean that I have NO thread-safety whatsoever for MPI calls?  Is it
> possible to compile openib drivers thread-safe?  I've got a lot of
> questions about this. Regards, David

David,

Sorry to see the trouble you've been having. There was an issue with
multithreading and some optimizations we had made for collectives using
shared memory. We've since fixed this, however, the main tarball has not
yet been updated. Can you try this updated version?

You can download an updated tarball from the following location:
http://mvapich.cse.ohio-state.edu/nightly/mvapich2/branches/0.9.8/mvapich2-0.9.8-2007-03-13.tar.gz

Or, you can do the following in your existing copy of make.mvapich.gen2
(line 118)

COLL_FLAG="-D_SHMEM_COLL_"
to
COLL_FLAG="-"

The OpenFabrics drivers are thread-safe. At this point they are not
fork-safe though. We have tested with multi-threaded MPI tests and things
should operate correctly. Let us know if this does not solve your issue.

Thanks,
Matt