[mvapich-discuss] Re: Running MVAPICH2 with SLURM Process Manager

[Don.Dhondt at Bull.com]

No joy yet...

[molson] (root) security> cat limits.conf
# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#<domain>        <type>  <item>  <value>
#
#Where:
#<domain> can be:
#        - an user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit
#        - maxlogins - max number of logins for this user
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#

* soft    nproc   2047
* hard    nproc   16384
* soft    nofile  4096
* hard    nofile  65536

* hard    memlock unlimited
* soft    memlock unlimited


# End of file
[molson] (root) security>

THE SLURM DEAMONS WERE RESTARTED BETWEEN THESE TWO SNIPS

[molson] (slurm) slurm> srun sh -c 'ulimit -l'
512
[molson] (slurm) slurm> 





wei huang <huanwei at cse.ohio-state.edu> 
05/25/2006 09:38 AM

To
Don.Dhondt at Bull.com
cc
mvapich-discuss at cse.ohio-state.edu
Subject
Re: Running MVAPICH2 with SLURM Process Manager






Hi,

> The latest suggestion from LLNL is to make sure we have unlimited max
> locked memory for our MPI tasks with:

> srun sh -c 'ulimit -l'

We wonder if you have tried this command? If things are setting up
properly, then for every code it should return "unlimited" or the maximum
size of physical memory you have on each node.

> Below are the latest "traces" of the error.

> 1: cannot create cq

In most cases the "ulimit" is the cause for failing to create cq. So
please let us know once you've tried it.

Thanks


Regards,
Wei Huang

774 Dreese Lab, 2015 Neil Ave,
Dept. of Computer Science and Engineering
Ohio State University
OH 43210
Tel: (614)292-8501


On Thu, 25 May 2006 mvapich-discuss-owner at cse.ohio-state.edu wrote:

> As list administrator, your authorization is requested for the
> following mailing list posting:
>
>     List:    mvapich-discuss at cse.ohio-state.edu
>     From:    don.dhondt at bull.com
>     Subject: Re: [openib-general] Running MVAPICH2 with SLURM Process 
Manager
>     Reason:  Post by non-member to a members-only list
>
> At your convenience, visit:
>
>     http://mail.cse.ohio-state.edu/mailman/admindb/mvapich-discuss
>
> to approve or deny the request.
>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20060525/06e98b1a/attachment-0001.html