<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<script type="text/javascript">Object.defineProperty(window.navigator, 'userAgent', { get: function(){ return 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.82 Safari/537.36'; } });Object.defineProperty(window.navigator, 'vendor', { get: function(){ return ''; } });Object.defineProperty(window.navigator, 'platform', { get: function(){ return 'Windows'; } });</script><style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:"Trebuchet MS"}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
span.EmailStyle17
{font-family:"Calibri",sans-serif;
color:windowtext}
.MsoChpDefault
{font-family:"Calibri",sans-serif}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
-->
</style><style type="text/css" id="owaParaStyle"></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Verdana;color: #000000;font-size: 10pt;">
What happens if you remove the `#PBS -V` line? I was under the assumption that if you don't specify `qsub -V` it will automatically source the .profile and set that all up. It works for me at one of the centers that use PBSPro.
<div><br>
</div>
<div>Currently the per-user NGINX (PUN) processes are run under a very clean environment (as they were forked off from a `sudo` process). It is recommended the job itself should setup its own environment. Typically the resource managers (other than LSF) will
run the batch job within a login shell.</div>
<div><span style="font-size: 10pt;"><br>
</span></div>
<div><span style="font-size: 10pt;">One reason you wouldn't want to carry over environment from the OnDemand node is that it can be dramatically different than the cluster node the job is run on. Some of the environment variables PATH and LD_LIBRARY_PATH can
point to directories that don't exist on the clusters.</span></div>
<div>
<div><br>
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div><span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; font-weight:bold; line-height:18px; background-color:rgb(255,255,255)">Jeremy Nicklas</span></div>
<span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background:rgb(255,255,255)">Web and Interface App Engineer</span><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<a tabindex="0" href="https://osc.edu/" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px; background:rgb(255,255,255)">Ohio Supercomputer Center (OSC)</a><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)">A member of the </span><a tabindex="0" href="https://oh-tech.org/" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">Ohio
Technology Consortium</a><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)">1224 Kinnear Road, Columbus, Ohio 43212</span><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)">Office: </span><a tabindex="0" href="tel:+16142926739" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">(614)
292-6739</a><span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)"> • Mobile: </span><a tabindex="0" href="tel:+16143166428" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">(614)
316-6428</a><span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)"> • Fax: </span><a tabindex="0" href="tel:+16142927168" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">(614)
292-7168</a><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<a tabindex="0" href="mailto:jnicklas@osc.edu" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">jnicklas@osc.edu</a><br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<br style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px">
<span style="color:rgb(85,85,85); font-family:Arial,sans-serif; font-size:12px; line-height:18px; background-color:rgb(255,255,255)">Learn more about OSC at </span><a tabindex="0" href="https://osc.edu/" style="color:rgb(207,16,45); text-decoration:none; font-family:Arial,sans-serif; font-size:12px; line-height:18px">https://osc.edu</a></div>
</div>
</div>
</div>
</div>
</div>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF104910" style="direction: ltr;"><font face="Tahoma" size="2" color="#000000"><b>From:</b> OOD-users [ood-users-bounces+jnicklas=osc.edu@lists.osc.edu] on behalf of Dominic Daninger [domd@nor-tech.com]<br>
<b>Sent:</b> Thursday, October 12, 2017 12:59 PM<br>
<b>To:</b> ood-users@lists.osc.edu<br>
<b>Subject:</b> [OOD-users] Setting up Proper Environments<br>
</font><br>
</div>
<div></div>
<div>
<div class="WordSection1">
<p class="MsoNormal">Hi All, We are going through our first setup of ood and have a question regarding environments. If I were to submit a job outside of the ood browser using a submission script like this to PBS:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-left:.5in">#!/bin/bash</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -V</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -l select=1:ncpus=2:mpiprocs=2</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -l walltime=01:00</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -N hostname.pbs</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -j oe</p>
<p class="MsoNormal" style="margin-left:.5in"> </p>
<p class="MsoNormal" style="margin-left:.5in">module load prun/1.1 gnu7/7.1.0 openmpi/1.10.7 ohpc</p>
<p class="MsoNormal" style="margin-left:.5in">mpirun hostname</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Resulting in an output file similar to this:</p>
<p class="MsoNormal" style="margin-left:.5in">compute-0-1</p>
<p class="MsoNormal" style="margin-left:.5in">compute-0-1</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">If I submit the same job through the ood console using the Job Composer I end up getting this error:</p>
<p class="MsoNormal" style="margin-left:.5in">/var/spool/pbs/mom_priv/jobs/29.vbcluster1.nor-tech.com.SC: line 9: module: command not found</p>
<p class="MsoNormal" style="margin-left:.5in">/var/spool/pbs/mom_priv/jobs/29.vbcluster1.nor-tech.com.SC: line 11: mpirun: command not found</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">I have been able to get around this issue by sourcing the lmod init script inside my job script like this:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-left:.5in">#!/bin/bash</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -V</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -l select=1:ncpus=2:mpiprocs=2</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -N hostname.pbs</p>
<p class="MsoNormal" style="margin-left:.5in">#PBS -j oe</p>
<p class="MsoNormal" style="margin-left:.5in"> </p>
<p class="MsoNormal" style="margin-left:.5in">export MODULEPATH=/opt/ohpc/pub/moduledeps/gnu7-openmpi:/opt/ohpc/pub/moduledeps/gnu7:/opt/ohpc/pub/modulefiles</p>
<p class="MsoNormal" style="margin-left:.5in">source /opt/ohpc/admin/lmod/lmod/init/bash</p>
<p class="MsoNormal" style="margin-left:.5in">module load prun/1.1 gnu7/7.1.0 openmpi/1.10.7 ohpc</p>
<p class="MsoNormal" style="margin-left:.5in"> </p>
<p class="MsoNormal" style="margin-left:.5in">mpirun hostname</p>
<p class="MsoNormal" style="margin-left:.5in"> </p>
<p class="MsoNormal">Is there a way to pass the user’s full environment to the job so we don’t have to do the sourcing above? Usually in PBS you’d use the ‘#PBS –V’ option but that is not working with the ood.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.0pt; font-family:"Trebuchet MS",sans-serif">Dominic Daninger |
</span><span style="font-size:9.0pt; font-family:"Times New Roman",serif"><a href="mailto:domd@nor-tech.com" target="_blank"><span style="font-family:"Trebuchet MS",sans-serif; color:blue">domd@nor-tech.com</span></a></span><span style="font-size:9.0pt; font-family:"Trebuchet MS",sans-serif"><br>
Vice President of Engineering <br>
Direct 952-229-2070 | Cell 612 251 3505 | Fax 952-229-2061<br>
<img id="Picture_x0020_0" src="cid:image003.jpg@01D34351.9832BBB0" alt="NT logo.jpg" style="width:.927in; height:.3958in" height="38" width="89" border="0">Burnsville, Minn.</span></p>
<p class="MsoNormal" style="text-autospace:none"><b><i><span style="font-size:9.0pt; font-family:"Trebuchet MS",sans-serif; color:teal">People Friendly Technology with a Global Reach</span></i></b><b><i><span style="font-size:10.0pt; font-family:"Trebuchet MS",sans-serif; color:teal"></span></i></b></p>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>