[mvapich-discuss] mvapich2 -> rdma_setup_startup_ring(410): cannot create cq

[Jim Galarowicz]

Hi everyone,

I'm running on a small cluster that has slurm and mvapich2 version 2.1 
installed.
However, I'm seeing this error when I try to run a simple mpi application.

    /srun -n 2 --mpi=pmi2 ./nbody-mvapich2//
    / /
    //In: PMI_Abort(1, Fatal error in MPI_Init://
    //Other MPI error, error stack://
    //MPIR_Init_thread(514).......: //
    //MPID_Init(367)..............: channel initialization failed//
    //MPIDI_CH3_Init(492).........: //
    //MPIDI_CH3I_RDMA_init(224)...: //
    //rdma_setup_startup_ring(410): cannot create cq//
    //)//
    //In: PMI_Abort(1, Fatal error in MPI_Init://
    //Other MPI error, error stack://
    //MPIR_Init_thread(514).......: //
    //MPID_Init(367)..............: channel initialization failed//
    //MPIDI_CH3_Init(492).........: //
    //MPIDI_CH3I_RDMA_init(224)...: //
    //rdma_setup_startup_ring(410): cannot create cq//
    //)//
    /



I searched the internet and found this url 
(http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/2012-September/004027.html) 
on the "cannot create cq" issue, which suggests we need to set

ulimit -l unlimited  in  /etc/sysconfig/slurm

> If it doesn't show unlimited (or some other number much higher than 64)
> then you'll need to do something to update the limits slurm is using.
> On redhat systems you can put the following in /etc/sysconfig/slurm.
>
>      ulimit -l unlimited
So, I added that file with the "ulimit -l unlimited" statement added.
But, it didn't seem to make any difference on the issue.

Does anyone have any hints on what might be wrong?

Thank you,
Jim G




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20151101/271a6ac8/attachment.html>