[mvapich-discuss] Using DPM on desktop

[Subramoni, Hari]

Hi, Lana.

The interrupt drive mode of progression that is enabled by MV2_USE_BLOCKING is not required for DPM. So, that can be removed.

However, when DPM is enabled, the MPI library does not know ahead of time where the spawned process is going to be (i.e. the spawned process can be either on the local node or on any remote node) and thus falls back to using network based communication between parent and spawned processes. That is why the IB sub-channel is getting triggered.

We need to see what needs to be done in terms of design changes to enable non-IB based support for DPM. Off the top of my head, it may not be trivial.

Best,
Hari.

From: mvapich-discuss-bounces at cse.ohio-state.edu <mvapich-discuss-bounces at mailman.cse.ohio-state.edu> On Behalf Of Lana Deere
Sent: Wednesday, May 20, 2020 1:41 PM
To: mvapich-discuss at cse.ohio-state.edu <mvapich-discuss at mailman.cse.ohio-state.edu>
Subject: [mvapich-discuss] Using DPM on desktop

I'm using mvapich2 2.3.1 with mpiexec.hydra.  I have a simple MPI program and a complex one.  I'm not an MPI expert and have inherited these programs and their driving script.

The simple one I can run with something like
mpiexec.hydra -np 2 simple-program
and it will run on my desktop.  Or I can use qsub to submit the mpiexec.hydra command onto a cluster and it will run with the number of processes assigned by the qsub.  In the latter case it gets more mpiexec options:
-genv MV2_BCAST_HWLOC_TOPOLOGY 0
-genv MV2_NUM_HCAS 1
The simple-program runs fine either way, seemingly using InfiniBand on the cluster and I guess sockets on my desktop.

The complex program I can run under qsub on the cluster; it needs two additional options to operate:
-genv MV2_USE_BLOCKING 1
-genv MV2_SUPPORT_DPM 1
I would like to be able to run the complex program on my desktop as well, but the two extra options seem to force InfiniBand, which I don't have on the desktop.  At least theoretically the program could run with a single MPI process (plus one additional it spawns locally via DPM) provided I use only small datasets.

Is there any way to enable DPM which will allow it to run without InfiniBand?
If MV2_USE_BLOCKING is InfiniBand-specific, then I should be able to just skip that option on the desktop?

Thanks!

.. Lana (lana.deere at gmail.com<mailto:lana.deere at gmail.com>)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20200520/7315d3e7/attachment-0001.html>