[mvapich-discuss] Problem initializing IB device (gen2)

[Roland Fehrenbacher]

Hi,

I have problems getting my IB adapters initialized when running an
mvapich binary:

$ mpiexec -comm mpich-ib ./cpi
[0] Abort: Error getting HCA context
 at line 260 in file viainit.c
[1] Abort: Error getting HCA context
 at line 260 in file viainit.c
[2] Abort: Error getting HCA context
 at line 260 in file viainit.c
[3] Abort: Error getting HCA context
 at line 260 in file viainit.c
mpiexec: Warning: tasks 0-3 exited with status 255.

The same happens when using mpirun_rsh.

I'm using mvapich 0.9.8 compiled against OFED 1.1.

Basic IB connectivity works as shown by a ping over the IB network
between the two test nodes I use. I can also run ib_rdma_bw,
ib_rdma_lat, etc. programs from the OFED release without any problem.

Loaded IB modules are:

beo-104:~# lsmod | grep ib_
ib_ipoib               49944  0
ib_sa                  17292  1 ib_ipoib
ib_uverbs              41520  0
ib_umad                18480  4
ib_mthca              120880  0
ib_mad                 39588  3 ib_sa,ib_umad,ib_mthca
ib_core                56192  6 ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad

and the following devices exist.

beo-104:~# ls -l /dev/infiniband/
total 0
crw-rw----  1 root root 231,  64 Jan 22 14:11 issm0
crw-rw----  1 root root 231,   0 Jan 22 14:11 umad0
crw-rw-rw-  1 root root 231, 192 Jan 22 14:11 uverbs0

I have read section "7.2.2  Error getting HCA Context" from the
Mvapich User Guide, but this didn't bring me any further.

What is going wrong?

Thanks,

Roland