[OOD-users] Setting up Proper Environments

[Nicklas, Jeremy]

What happens if you remove the `#PBS -V` line? I was under the assumption that if you don't specify `qsub -V` it will automatically source the .profile and set that all up. It works for me at one of the centers that use PBSPro.

Currently the per-user NGINX (PUN) processes are run under a very clean environment (as they were forked off from a `sudo` process). It is recommended the job itself should setup its own environment. Typically the resource managers (other than LSF) will run the batch job within a login shell.

One reason you wouldn't want to carry over environment from the OnDemand node is that it can be dramatically different than the cluster node the job is run on. Some of the environment variables PATH and LD_LIBRARY_PATH can point to directories that don't exist on the clusters.

Jeremy Nicklas
Web and Interface App Engineer
Ohio Supercomputer Center (OSC)<https://osc.edu/>
A member of the Ohio Technology Consortium<https://oh-tech.org/>
1224 Kinnear Road, Columbus, Ohio 43212
Office: (614) 292-6739<tel:+16142926739> • Mobile: (614) 316-6428<tel:+16143166428> • Fax: (614) 292-7168<tel:+16142927168>
jnicklas at osc.edu<mailto:jnicklas at osc.edu>

Learn more about OSC at https://osc.edu<https://osc.edu/>
________________________________
From: OOD-users [ood-users-bounces+jnicklas=osc.edu at lists.osc.edu] on behalf of Dominic Daninger [domd at nor-tech.com]
Sent: Thursday, October 12, 2017 12:59 PM
To: ood-users at lists.osc.edu
Subject: [OOD-users] Setting up Proper Environments

Hi All, We are going through our first setup of ood and have a question regarding environments.  If I were to submit a job outside of the ood browser using a submission script like this to PBS:

#!/bin/bash
#PBS -V
#PBS -l select=1:ncpus=2:mpiprocs=2
#PBS -l walltime=01:00
#PBS -N hostname.pbs
#PBS -j oe

module load prun/1.1 gnu7/7.1.0 openmpi/1.10.7 ohpc
mpirun hostname

Resulting in an output file similar to this:
compute-0-1
compute-0-1

If I submit the same job through the ood console using the Job Composer I end up getting this error:
/var/spool/pbs/mom_priv/jobs/29.vbcluster1.nor-tech.com.SC: line 9: module: command not found
/var/spool/pbs/mom_priv/jobs/29.vbcluster1.nor-tech.com.SC: line 11: mpirun: command not found

I have been able to get around this issue by sourcing the lmod init script inside my job script like this:

#!/bin/bash
#PBS -V
#PBS -l select=1:ncpus=2:mpiprocs=2
#PBS -N hostname.pbs
#PBS -j oe

export MODULEPATH=/opt/ohpc/pub/moduledeps/gnu7-openmpi:/opt/ohpc/pub/moduledeps/gnu7:/opt/ohpc/pub/modulefiles
source /opt/ohpc/admin/lmod/lmod/init/bash
module load prun/1.1 gnu7/7.1.0 openmpi/1.10.7 ohpc

mpirun hostname

Is there a way to pass the user’s full environment to the job so we don’t have to do the sourcing above?  Usually in PBS you’d use the ‘#PBS –V’ option but that is not working with the ood.


Dominic Daninger | domd at nor-tech.com<mailto:domd at nor-tech.com>
Vice President of Engineering
Direct 952-229-2070 | Cell 612 251 3505 | Fax 952-229-2061
[NT logo.jpg]Burnsville, Minn.
People Friendly Technology with a Global Reach

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/ood-users/attachments/20171012/2618a0f2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 1917 bytes
Desc: image003.jpg
URL: <http://lists.osu.edu/pipermail/ood-users/attachments/20171012/2618a0f2/attachment.jpg>