<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Emoji";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.xm8653619640025940478msoautosig, li.xm8653619640025940478msoautosig, div.xm8653619640025940478msoautosig
{mso-style-name:x_m8653619640025940478msoautosig;
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-family:"Courier New"">Morgan,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">That sounds like something that would be quite helpful at our site.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Thanks!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Mike<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Rodgers, Morgan E. <mrodgers@osc.edu> <br>
<b>Sent:</b> Tuesday, December 11, 2018 11:53 AM<br>
<b>To:</b> Jeremy Nicklas <jeremywnicklas@gmail.com>; John-Paul Robinson <jprorama@gmail.com>; Michael Coleman <mcolema5@uoregon.edu>; User support mailing list for Open OnDemand <ood-users@lists.osc.edu><br>
<b>Subject:</b> Re: [OOD-users] no environment set for HPC desktop -- job fails<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div id="divtagdefaultwrapper">
<div id="divtagdefaultwrapper">
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Folks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I wanted to call out a feature in the forthcoming 1.4 release of OOD: resource manager client executable overrides. The cluster configuration YAML will support an option called "bin_overrides"
in the job configuration which allows you to specify the path to a resource manager executable such as sbatch or qsub that is in a different location than the default. We are hoping that this ability will solve a few problems for the community. First is we
have heard that at least one site wanted to use a client program that was not located in the default client executable directory. This will also give sites the ability to write custom wrappers such as submit filters, and environment setting wrappers to the
standard submission programs.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">This second ability is an advanced usage; for most of our adapters OOD expects to be able to capture return codes and parse STDOUT, but if return codes are properly handled, and the STDOUT formatting
is not changed, then anything should be possible. At this time I do not have any examples of this usage, but generating some is on my todo list.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><a href="https://osc.github.io/ood-documentation/develop/installation/resource-manager/slurm.html">https://osc.github.io/ood-documentation/develop/installation/resource-manager/slurm.html</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">- Morgan <o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;color:black">
<hr size="3" width="98%" align="center">
</span></div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> OOD-users <<a href="mailto:ood-users-bounces+mrodgers=osc.edu@lists.osc.edu">ood-users-bounces+mrodgers=osc.edu@lists.osc.edu</a>> on behalf of Michael Coleman via
OOD-users <<a href="mailto:ood-users@lists.osc.edu">ood-users@lists.osc.edu</a>><br>
<b>Sent:</b> Monday, December 10, 2018 8:04:34 PM<br>
<b>To:</b> Jeremy Nicklas; John-Paul Robinson; User support mailing list for Open OnDemand<br>
<b>Subject:</b> Re: [OOD-users] no environment set for HPC desktop -- job fails</span><span style="font-size:12.0pt;color:black">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Hi Jeremy,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Thanks for implementing the SLURM support. This minor setup issue notwithstanding, it's working great for us!</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Regarding the particulars of exactly how the environment gets set up on the compute nodes, I suspect this varies considerably from site to site. And many sites, definitely
including ours, cannot change the already established behavior without considerable distress on the users' parts.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">There are some different ideas in this ticket</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">
<a href="https://github.com/OSC/ood_core/issues/109">https://github.com/OSC/ood_core/issues/109</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">For us, the best solution would probably be a way to shim in a wrapper like</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> env -i HOME=~ bash -l -c 'sbatch yada yada...'</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">so that we can make job startup under OOD act as similarly as possible job startup without OOD.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Lacking that, or perhaps in addition, it'd also be nice to have a fix for script.job_environment parsing so that the 'ALL' (and 'NONE') keywords are supported, rather
than hacking with the 'native' feature.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Anyway, with a bit of hacking, we're going here.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black">Mike</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;font-family:"Courier New";color:black"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal"><b><span style="font-size:12.0pt;color:black">From:</span></b><span style="font-size:12.0pt;color:black"> Jeremy Nicklas <<a href="mailto:jeremywnicklas@gmail.com">jeremywnicklas@gmail.com</a>>
<br>
<b>Sent:</b> Saturday, December 8, 2018 03:51 PM<br>
<b>To:</b> John-Paul Robinson <<a href="mailto:jprorama@gmail.com">jprorama@gmail.com</a>>; User support mailing list for Open OnDemand <<a href="mailto:ood-users@lists.osc.edu">ood-users@lists.osc.edu</a>><br>
<b>Cc:</b> Michael Coleman <<a href="mailto:mcolema5@uoregon.edu">mcolema5@uoregon.edu</a>><br>
<b>Subject:</b> Re: [OOD-users] no environment set for HPC desktop -- job fails<o:p></o:p></span></p>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Hi all,<o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Original developer of the Slurm adapter in Open OnDemand here. I want first clear up any confusion about environment variables here. Behind the scenes when OnDemand calls the `sbatch` command
to submit a job it actually:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- sets the environment variable `SBATCH_EXPORT=NONE`<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- adds the command line argument `--parsable` for easier parsing of the job id<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">The relevant snippet of code can be seen here:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"><a href="https://github.com/OSC/ood_core/blob/866db98e3b3ba574a5799a3af84d2b5ffc010ff8/lib/ood_core/job/adapters/slurm.rb#L128-L138">https://github.com/OSC/ood_core/blob/866db98e3b3ba574a5799a3af84d2b5ffc010ff8/lib/ood_core/job/adapters/slurm.rb#L128-L138</a><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">I purposely set the `SBATCH_EXPORT=NONE` for a few reasons:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- as these are Passenger apps running below a `sudo` user switch, the environment is already drastically different from the user's login shell environment when they call `sbatch`<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- the library paths LD_LIBRARY_PATH and binary paths PATH could be different between the web node and the cluster node<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- if you use `SBATCH_EXPORT=NONE`, the Slurm documentation states:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"><br>
> Slurm will then implicitly attempt to load the user's environment on the node where the script is being executed<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> which is what we would want.<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">I do not recommend setting this environment variable to `ALL` for the reasons listed above. It may cause more headaches down the road.<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Now I am not entirely sure why you are seeing this issue after updating Slurm, but taking a look at the NEWS file for Slurm, one particular bugfix stands out:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"><a href="https://github.com/SchedMD/slurm/blob/d128eb21583fe80b808ce64667cd89d209cd62ab/NEWS#L532">https://github.com/SchedMD/slurm/blob/d128eb21583fe80b808ce64667cd89d209cd62ab/NEWS#L532</a><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">That particular commit in 17.11.11 fixes an issue in `src/common/env.c` which was introduced in 17.11.10. The exact commit is here:<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"><a href="https://github.com/SchedMD/slurm/commit/72b2355ca8d6f4381b4e417f76649712881f45b7">https://github.com/SchedMD/slurm/commit/72b2355ca8d6f4381b4e417f76649712881f45b7</a><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">I am not sure if this is what you are experiencing. But could you test OnDemand using Slurm 17.11.11?<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">- Jeremy Nicklas<o:p></o:p></span></p>
</div>
</div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">On Sat, Dec 8, 2018 at 5:33 PM John-Paul Robinson via OOD-users <<a href="mailto:ood-users@lists.osc.edu">ood-users@lists.osc.edu</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Ok. <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">My experience has been the behavior with the second export below and not the third. That is, spec an env and you only get those specific vars. Don’t spec any and you get the ALL behavior. <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">What’s odd is that desktop jobs did work before and now don’t, suggesting slurm has changed wrt the ood assumptions. <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Leaving the question, where are those ood assumptions made and can we just add ALL to the front of the list?<o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Is the conclusion of the github issue? <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;color:black"><br>
On Dec 8, 2018, at 1:54 PM, Michael Coleman <<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p><span style="font-size:12.0pt;color:black">The key change is what happens if you use a flag like<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> --export=MYVAR1,MYVAR2<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black">The behavior mentioned in the man page (see <a href="https://github.com/OSC/ood_core/issues/109" target="_blank">https://github.com/OSC/ood_core/issues/109</a>) says that no other environment variables will be passed.
In other words, this is equivalent to<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> --export=NONE,MYVAR1,MYVAR2<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black">The recent fix actually makes sbatch behave this way. Previously, the implementation of this case was incorrectly equivalent to<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> --export=ALL,MYVAR1,MYVAR2<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black">And just to make us all completely insane, it looks like they're changing this further in a newer release (that we don't yet use at our site). See the new man page here:<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <a href="https://slurm.schedmd.com/sbatch.html" target="_blank">https://slurm.schedmd.com/sbatch.html</a><o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:12.0pt;color:black">Mike<o:p></o:p></span></p>
<div id="x_m_8653619640025940478Signature">
<div name="x_divtagdefaultwrapper">
<p class="xm8653619640025940478msoautosig"><span style="font-size:10.0pt;font-family:Consolas;color:black">Michael Coleman (<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>), Computational Scientist</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xm8653619640025940478msoautosig"><span style="font-size:10.0pt;font-family:Consolas;color:black">Research Advanced Computing Services</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<div>
<p class="xmsonormal" style="background:white"><span style="font-size:10.0pt;font-family:Consolas;color:#212121">6235 University of Oregon</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal" style="background:white"><span style="font-size:10.0pt;font-family:Consolas;color:#212121">Eugene, OR 97403</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<p class="xm8653619640025940478msoautosig"><span style="font-size:12.0pt;color:black"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;color:#212121">
<hr size="3" width="98%" align="center">
</span></div>
<div id="x_m_8653619640025940478divRplyFwdMsg">
<p class="xmsonormal"><b><span style="font-size:12.0pt;color:black">From:</span></b><span style="font-size:12.0pt;color:black"> John-Paul Robinson <<a href="mailto:jprorama@gmail.com" target="_blank">jprorama@gmail.com</a>><br>
<b>Sent:</b> Saturday, December 8, 2018 10:54<br>
<b>To:</b> Michael Coleman<br>
<b>Cc:</b> User support mailing list for Open OnDemand<br>
<b>Subject:</b> Re: [OOD-users] no environment set for HPC desktop -- job fails</span><span style="font-size:12.0pt;color:#212121">
</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">I gotta say, I am trying to read it very very carefully but don’t seem to be able to parse out the subtlety.
</span><span style="font-size:12.0pt;font-family:"Segoe UI Emoji",sans-serif;color:#212121">😐</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Not sure how to miss read this, emphasis on all theirs:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><b><span style="font-size:12.0pt;color:#212121;border:none windowtext 1.0pt;padding:0in">--export</span></b><span style="font-size:12.0pt;color:#212121">=<<i><span style="border:none windowtext 1.0pt;padding:0in">environment variables
[ALL] | NONE</span></i>></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:#212121">Identify which environment variables from the submission environment are propagated to the launched application. By default, <i><span style="border:none windowtext 1.0pt;padding:0in">all</span></i> are
propagated. Multiple environment variable names should be comma separated. </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="xmsonormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;color:#212121"><br>
On Dec 7, 2018, at 10:24 PM, Michael Coleman <<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>> wrote:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="xmsonormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;color:#212121">If you read the man page very, very carefully, I think the new behavior actually matches what it always said. The old behavior didn't actually match. But I definitely
agree that it's a pretty significant change.<br>
<br>
Michael Coleman (<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>), Computational Scientist<br>
Research Advanced Computing Services<br>
6235 University of Oregon<br>
Eugene, OR 97403<br>
<br>
<br>
________________________________________<br>
From: John-Paul Robinson <<a href="mailto:jprorama@gmail.com" target="_blank">jprorama@gmail.com</a>><br>
Sent: Friday, December 7, 2018 17:19<br>
To: Michael Coleman<br>
Cc: User support mailing list for Open OnDemand<br>
Subject: Re: [OOD-users] no environment set for HPC desktop -- job fails<br>
<br>
I’m also scratching my head because documentation (even for the 18 release) still says the default of export is to propagate all environment variables. :/<br>
<br>
</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">On Dec 7, 2018, at 6:59 PM, John-Paul Robinson <<a href="mailto:jprorama@gmail.com" target="_blank">jprorama@gmail.com</a>> wrote:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Its a pretty alarming change to have happened in three ticks of the minor release number.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Any insights from others on this?</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">On Dec 7, 2018, at 6:37 PM, Michael Coleman <<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>> wrote:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Hi John-Paul,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">As you say, I believe the key event was the transition in SLURM versions. They apparently made a change in the behavior of export of environment variables from the submitting environment to
the job environment (the stuff --export controls). There was much wailing of our users here when their sbatch scripts broke as a result. Generally, the "fix" was simply for users that were already using the --export flag to add the "ALL" keyword to that
list, which seemed to restore the old behavior.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Ultimately, OOD is calling 'sbatch' to create jobs, and this change affects the environment those jobs see. At least in our environment, the --export=ALL flag seems to cure OOD issues. There
are probably other ways to change things, but this seemed the simplest.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Good luck,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Mike</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">-----Original Message-----</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">From: John-Paul Robinson <<a href="mailto:jprorama@gmail.com" target="_blank">jprorama@gmail.com</a>></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Sent: Friday, December 7, 2018 03:56 PM</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">To: Michael Coleman <<a href="mailto:mcolema5@uoregon.edu" target="_blank">mcolema5@uoregon.edu</a>>; User support mailing list for Open OnDemand <<a href="mailto:ood-users@lists.osc.edu" target="_blank">ood-users@lists.osc.edu</a>></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Subject: Re: [OOD-users] no environment set for HPC desktop -- job fails</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">MIke,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Thanks for pointing us to this issue.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">This does appear to be similar to what's happening in our dev</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">environment. (Note our still-working prod environment is Bright CM with</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">slurm 17.02.2).</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">The odd thing with our dev environment (built on OpenHPC) is that it was</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">working in October and only started failing in builds over the past</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">month. This appears to coincide with the OpenHPC 1.3.5 to 1.3.6 update</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">(going from slurm 17.11.7 to 17.11.10).</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">We've had some success in restoring the original working configuration</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">in one of our test stacks by reverting to the OpenHPC 1.3.5 release.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">What's odd is this implies the problem is not with OOD but in the</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">OpenHPC system env. As far as we can determine, our OOD remains</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">identical. We are setting up dev in vagrant with ansible provisioning</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">the openhpc + ood cluster based on the CRI_XSEDE work extended to add an</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">OOD node via vagrant + ansible (not as a warewulf provision).</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"><a href="https://github.com/jprorama/CRI_XCBC" target="_blank">https://github.com/jprorama/CRI_XCBC</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">I've read through the github issue below but haven't teased out all the</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">details.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Is there an obvious transition point where this export behavior could be</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">impacted by the underlying system versions OOD is running on?</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">We'll contribute insights on the github issue as we find them.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Thanks,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">John-Paul</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">On 12/5/18 5:15 PM, Michael Coleman wrote:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Hi John-Paul,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">We worked through something similar. You might find some useful hints on this ticket.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> <a href="https://github.com/OSC/ood_core/issues/109" target="_blank">https://github.com/OSC/ood_core/issues/109</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Cheers,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Mike</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">-----Original Message-----</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">From: OOD-users <<a href="mailto:ood-users-bounces+mcolema5=uoregon.edu@lists.osc.edu" target="_blank">ood-users-bounces+mcolema5=uoregon.edu@lists.osc.edu</a>> On Behalf Of John-Paul Robinson
via OOD-users</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Sent: Wednesday, December 5, 2018 02:49 PM</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">To: <a href="mailto:ood-users@lists.osc.edu" target="_blank">
ood-users@lists.osc.edu</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Subject: [OOD-users] no environment set for HPC desktop -- job fails</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">In our dev environment (slurm with ohpc) we have started to see this</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">error when trying to launch interactive desktops:</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">/tmp/slurmd/job00079/slurm_script: line 3: module: command not found</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Setting VNC password...</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Error: no HOME environment variable</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Starting VNC server...</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">vncserver: The HOME environment variable is not set.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">vncserver: The HOME environment variable is not set.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">vncserver: The HOME environment variable is not set. vncserver: The HOME</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">environment variable is</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">As we understand it, the PUN nginx worker launches the batch job that</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">starts the desktop batch job.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">The problem seems to be that the environment for the job is empty, hence</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">no module function or HOME env or anything else. We checked the env of</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">the users nginx worker under /proc and it is completely empty. Because</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">our job env is inherited from the caller (the nginx worker in this case)</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">the attempt to run the module command and vncserver commands naturally fail.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">When we launch an interactive terminal, it runs just fine, but I'm</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">guessing that's because the interactive session actually reads the</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">normal shell startup and builds its environment, even if it happened to</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">be missing in the proxy.</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Do you have any pointers on what could cause this situation. We</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">noticed it after we started adding additional interactive apps but don't</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">have a clear time point. It was working fine originally and still</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">functions fine in our prod env (without any of the additional</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">interactive apps).</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">Thanks,</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">John-Paul</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">_______________________________________________</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121">OOD-users mailing list</span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"><a href="mailto:OOD-users@lists.osc.edu" target="_blank">OOD-users@lists.osc.edu</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"><a href="https://lists.osu.edu/mailman/listinfo/ood-users" target="_blank">https://lists.osu.edu/mailman/listinfo/ood-users</a></span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal"><span style="font-size:12.0pt;color:#212121"> </span><span style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</blockquote>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">_______________________________________________<br>
OOD-users mailing list<br>
<a href="mailto:OOD-users@lists.osc.edu" target="_blank">OOD-users@lists.osc.edu</a><br>
<a href="https://lists.osu.edu/mailman/listinfo/ood-users" target="_blank">https://lists.osu.edu/mailman/listinfo/ood-users</a><o:p></o:p></span></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</body>
</html>