[mvapich-discuss] Configure/make fails when using --with-shared-memory=sysv

[Åsmund Ervik]

Hello all,

I'm compiling mvapich2 on an old quirky cluster where I'm not root.
After having problems with shared memory using mvapich2 with plain
configure/make (I had to disable shared memory for any jobs doing
allreduce), I decided to try to configure/make using
--with-shared-memory=sysv . This failed at the "make" step, with the
following error:
   CC       src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.lo
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c: In function
'mv2_allocate_shm_local':
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:182: error:
'O_CREAT' undeclared (first use in this function)
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:182: error: (Each
undeclared identifier is reported only once
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:182: error: for each
function it appears in.)
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:182: error: 'O_RDWR'
undeclared (first use in this function)
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:182: error: 'O_EXCL'
undeclared (first use in this function)
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c: In function
'mv2_rma_allocate_shm':
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:375: error:
'O_CREAT' undeclared (first use in this function)
src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.c:375: error: 'O_RDWR'
undeclared (first use in this function)
make[2]: *** [src/mpid/ch3/channels/mrail/src/gen2/rdma_iba_1sc.lo] Error 1

The full configure command was
./configure --prefix=/home/asmunder/mvapich2-2.0
--with-ib-libpath=/cvos/shared/apps/ofed/1.3.1/lib64/
--with-ib-include=/cvos/shared/apps/ofed/1.3.1/include
--with-shared-memory=sysv --enable-libpci


I traced this down to lines 22 and 26 in rdma_iba_1sc.c ,
which is an #ifdef that only includes fcntl.h (containing O_CREAT et al)
when the flag _SMP_LIMIC_ is defined. Obviously this flag was not
defined in my case. I removed these two lines, and now configure/make
works fine.


Also, in the end, I found that the sysv thing was not necessary, and
that the
correct and much simpler solution was to set the environment variable
MV2_SHMEM_DIR to a tempdir I am allowed to write to. For future googlers,
the error message I originally had was e.g.

[node096:mpi_rank_0][mv2_shm_coll_init] shmem open failed for
file:/dev/shm/slot_shmem-coll-kvs_20417_0-node096-0-57926.tmp


Regards,
Åsmund


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20140911/a15bc4ee/attachment.sig>