[mvapich-discuss] MVAPICH2 2.3b Blocking communication progress

[Cabral, Matias A]

Hi Pau,

As Hari mentions, the libpsm2 communication path (Omni-Path communication library) does not support interrupt generation for completion events. Instead communication is requests driven, with a semantic very similar to non-blocking MPI. As with MPI, communication requests are posted to a queue and then polled/tested for completion, making progress while doing this.  Essentially, this is why MPI communication is so efficient over Omni-path and this semantic does not support generating costly interrupts (response time to handle them heavily impacts latency).

With an explicitly very clear disclaimer that this is not intended for message passing, the hfi1 driver (Omni-path driver) does support verbs communication API. Therefore, “standard IB communication API” is supported. However, I must insist that the performance seen for message passing will NOT be comparable to using PSM2 and this should be avoided.

Thanks,

_MAC

From: Subramoni, Hari [mailto:subramoni.1 at osu.edu]
Sent: Friday, October 20, 2017 12:02 PM
To: Pau Farré <pau.farre at bsc.es>; mvapich-discuss at cse.ohio-state.edu <mvapich-discuss at mailman.cse.ohio-state.edu>
Cc: Cabral, Matias A <matias.a.cabral at intel.com>; Subramoni, Hari <subramoni.1 at osu.edu>
Subject: RE: [mvapich-discuss] MVAPICH2 2.3b Blocking communication progress

Hi, Pau.

To the best of our knowledge, the Intel PSM2 interface does not support interrupt driven mode of progress. We need to poll all the time to check for progress. So we cannot implement the traditional “blocking” mode of progress for Intel Omni-Path HFIs.

We had some discussions with Intel folks about this in the past. I am not sure about the current status. I would recommend contacting Intel PSM2 developers for more information.

Thx,
Hari.

From: mvapich-discuss-bounces at cse.ohio-state.edu<mailto:mvapich-discuss-bounces at cse.ohio-state.edu> On Behalf Of Pau Farré
Sent: Friday, October 20, 2017 1:50 PM
To: mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu> <mvapich-discuss at mailman.cse.ohio-state.edu<mailto:mvapich-discuss at mailman.cse.ohio-state.edu>>
Subject: [mvapich-discuss] MVAPICH2 2.3b Blocking communication progress


Hi,

I amb interested in using MVAPICH 2.3b with the blocking communication progress mode and Intel Omnipath.

Is this actually supported? Since on the list of features says:

  *   Blocking (enables running multiple MPI processes/processor). Available for OpenFabrics (IB and iWARP) interfaces.

Thanks,

Pau Farre


WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.

http://www.bsc.es/disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20171020/02cb0d3a/attachment-0001.html>