[mvapich-discuss] MPI_Info to register buffers at allocation time?

[sreeram potluri]

Jeff,

In MVAPICH2, window memory is registered with IB in MPI_Win_create and
MPI_Win_allocate calls. If you are communication from and to window memory,
you should not see the registration overhead in the first call.

Which benchmarks are you using when comparing RMA and SHMEM performance?
The synchronization semantics used could be the reason for difference in
performance.

Best
Sreeram Potluri




On Sat, Sep 14, 2013 at 4:46 PM, Jeff Hammond <jeff.science at gmail.com>wrote:

> I recall from looking at MPICH source that neither MPI_Alloc_mem nor
> MPI_Win_{create,allocate} do anything about memory registration.  Does
> MVAPICH do anything different in this respect or does it do IB registration
> on-the-fly for all communication (and presumably maintain a cache for
> future reuse)?  Is it possible to set an MPI_Info key that instructs the
> implementation to do IB registration immediately during these calls?
>
> My motivation is that I do a lot of RMA and I'd like to not see a latency
> hit the first time I communicate between processes.  Additionally, I see
> that MVAPICH-X SHMEM latency is much better than MPI-RMA latency, which
> does not make sense to me (there is no semantic reason for this as of
> MPI-3, but I recognize implementations do not necessarily reflect all that
> is possible in the specification) and I can only assume that some fraction
> of this improvement comes from the pre-registration of the SHMEM symmetric
> heap.
>
> If this features are not implemented, please consider this inquiry a
> feature request for future releases :-)
>
> Thanks,
>
> Jeff
>
> --
> Jeff Hammond
> jeff.science at gmail.com
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20130914/8727bbf7/attachment.html