[mvapich-discuss] No IB device found when running jobs

[Panda, Dhabaleswar]

Hi,

Thanks for your note. MVAPICH2 does not suppprt libfabric. Please use libibverbs. 

Thanks,

DK

Sent from my iPhone

> On Sep 12, 2019, at 12:13 PM, Arturo Fernandez <afernandez at odyhpc.com> wrote:
> 
> Hello MVAPICH2 team,
> I'm trying to use a combination of libfabric and the new MVAPICH2-2.3.2. If 'libibverbs' is not installed, MVAPICH2 will not build (similarly to the previous version). In a system where libibverbs is installed, and after installing libfabric, I configure MVAPICH2 with:
> ./configure --prefix=/opt/mvapich2 --with-device=ch3:mrail --with-rdma=gen2 --with-ib-include=/opt/libfabric/include --with-ib-libpath=/opt/libfabric/lib --disable-mcast
> It compiles and builds w/o any issue (unless I'm missing something), and 'mpirun_rsh -v' returns the expected outcome (2.3.2). However, MPI jobs fail because of no IB device:
> aborting job:
> Fatal error in MPI_Init:
> Other MPI error, error stack:
> MPIR_Init_thread(490)............:
> MPID_Init(396)...................: channel initialization failed
> MPIDI_CH3_Init(410).............: rdma_get_control_parameters
> rdma_get_control_parameters(1726):
> rdma_open_hca(575)...............: No IB device found
> My expectation was that using libfabric would suffice to fulfill communication requirements. My specific question would be: Is it possible to compile and build MVAPICH2 without a dependency on libibverbs? The configuration that I'm shooting for would be sort of a purely 'libfabric' mode (for lack of a better expression).
> Thanks,
> Arturo
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss