[OOD-users] Performance issues

[Dan Dietz]

Yes, our qlist will return different results between users and clusters.
When exactly does a "Dashboard load" happen? Would a user logging out and
reopening the browser trigger that or are we talking about at the server
level restart? Refreshing the cache at login for a user would be plenty
good enough I think. Either way, I think your patch will work for the time
being - we are likely 2-3 months away from going to any level of production
with this so no hurry on a robust solution for us.

I will give this a try and let you know.

Thanks again!

Dan

On Fri, Nov 9, 2018 at 1:55 PM Franz, Eric <efranz at osc.edu> wrote:

> Dan,
>
>
>
> If it does, this is one option. Add an
> /etc/ood/config/apps/dashboard/initializers/qlist.rb with contents like
> this:
>
>
>
> module Purdue
>
>   class Qlist
>
>     def self.qlist(cluster)
>
>       @qlist ||= Hash.new do |h, key|
>
>         Rails.logger.warn "executing expensive qlist"
>
>         h[key] = build_qlist(key)
>
>       end
>
>
>
>       @qlist[cluster]
>
>     end
>
>
>
>     def self.build_qlist(cluster)
>
>       ["#{cluster}-debug", "#{cluster}-serial", "#{cluster}-parallel" ]
>
>     end
>
>   end
>
> end
>
>
>
> # call to build cache on initialize (optional)
>
> Purdue::Qlist.qlist('owens')
>
> Purdue::Qlist.qlist('ruby')
>
> Purdue::Qlist.qlist('pitzer')
>
> Purdue::Qlist.qlist('oakley')
>
>
>
>
>
> You would replace build_qlist body with the method that executes qlist for
> the specified cluster. The other method qlist just employs per-argument
> memoization using Ruby’s Hash.new feature (the block provided to the hash
> is executed to generate the value if the value does not exist in the hash).
>
>
>
> Then you would add calls below to initialize the cache (though this might
> not be necessary, since memoziation would take care of this).
>
>
>
> In a form.yml.erb you would use <%
> Purdue::Qlist.qlist('your_cluster').each do |queue| %> to iterate over your
> queue list for that cluster.
>
>
>
> If you need something more robust because your requirements dictate you
> need to have the qlist executed at the start of the web form load, then
> I’ll have an alternative solution for you early next week.
>
>
>
> Thanks,
>
> Eric
>
>
>
> ---
>
> Eric Franz, Senior Web & Interface App Engineer
>
> Ohio Supercomputer Center
>
> An Ohio Technology Consortium (OH-TECH) Member
>
> 1224 Kinnear Road
>
> Columbus, OH 43212
>
> email: efranz at osc.edu
>
>
>
> *From: *OOD-users <ood-users-bounces+efranz=osc.edu at lists.osc.edu> on
> behalf of "Franz, Eric via OOD-users" <ood-users at lists.osc.edu>
> *Reply-To: *"Franz, Eric" <efranz at osc.edu>, User support mailing list for
> Open OnDemand <ood-users at lists.osc.edu>
> *Date: *Friday, November 9, 2018 at 1:29 PM
> *To: *Dan Dietz <ddietz at purdue.edu>
> *Cc: *"ood-users at lists.osc.edu" <ood-users at lists.osc.edu>
> *Subject: *Re: [OOD-users] Performance issues
>
>
>
> Dan,
>
>
>
> Does the value of qlist vary from cluster to cluster/user to user, and is
> it something that could be cached on Dashboard load instead of something
> that is executed every time you open an iHPC app’s web form?
>
>
>
> Thanks,
>
> Eric
>
>
>
> ---
>
> Eric Franz, Senior Web & Interface App Engineer
>
> Ohio Supercomputer Center
>
> An Ohio Technology Consortium (OH-TECH) Member
>
> 1224 Kinnear Road
>
> Columbus, OH 43212
>
> email: efranz at osc.edu
>
>
>
> *From: *Dan Dietz <ddietz at purdue.edu>
> *Date: *Friday, November 9, 2018 at 1:10 PM
> *To: *"Franz, Eric" <efranz at osc.edu>
> *Cc: *"ood-users at lists.osc.edu" <ood-users at lists.osc.edu>
> *Subject: *Re: [OOD-users] Performance issues
>
>
>
> Eric,
>
>
>
> Thanks for the quick response. You are right, it does not affect job
> composer, although there is a bit of delay opening it at first.
>
>
>
> A temporary patch would be most appreciated.
>
>
>
> Thanks,
>
> Dan
>
>
>
> On Fri, Nov 9, 2018 at 11:29 AM Franz, Eric <efranz at osc.edu> wrote:
>
> Dan,
>
>
>
> From what you are describing, I would not expect it to affect the job
> composer. But I apologize, this is a big oversight on our part. I opened
> issues https://github.com/OSC/ood-dashboard/issues/417 and
> https://github.com/OSC/ood-dashboard/issues/418 to track this work.
>
>
>
> We knew that the dashboard is inefficient in the menu building process and
> it is something we were planning to fix in the next three months. But it
> looks like this is higher priority.
>
>
>
> The dashboard builds the menu dynamically on each request, instead of
> caching the information. This is great when you have multiple people doing
> app sharing or app development as the users PUN doesn’t have to be
> restarted to see a new app. It is also useful for an interactive app where
> description or title information changes system state. But those are edge
> cases, not the common case for most Open OnDemand sites. And as you observe
> it is problematic when the form.yml.erb contains expensive code.
>
>
>
> I can provide a patch you can use till the next release. I’ll follow up
> with that in another email.
>
>
>
> Thanks,
>
> Eric
>
>
>
> ---
>
> Eric Franz, Senior Web & Interface App Engineer
>
> Ohio Supercomputer Center
>
> An Ohio Technology Consortium (OH-TECH) Member
>
> 1224 Kinnear Road
>
> Columbus, OH 43212
>
> email: efranz at osc.edu
>
>
>
> *From: *OOD-users <ood-users-bounces+efranz=osc.edu at lists.osc.edu> on
> behalf of Dan Dietz via OOD-users <ood-users at lists.osc.edu>
> *Reply-To: *Dan Dietz <ddietz at purdue.edu>, User support mailing list for
> Open OnDemand <ood-users at lists.osc.edu>
> *Date: *Friday, November 9, 2018 at 10:59 AM
> *To: *"ood-users at lists.osc.edu" <ood-users at lists.osc.edu>
> *Subject: *[OOD-users] Performance issues
>
>
>
> I'm currently testing implementing OOD here and all has gone
> quite smoothly so far (really, very nice work, I'm really enjoying working
> with it). I'm having one issue now though. As I started making the apps
> (I've got 6 so far) use form.yml.erb and dynamically display the user's
> available queues (select instead of free-form text box) and dynamically
> pull in versions from modules, things have gotten progressively slower in
> the dashboard (in apps, job composer, just loading the home page, anything)
> as I've improved more apps. It probably takes 10 seconds to load any page
> now.
>
>
>
> Watching `top` on the server running OOD what I see are a ton of `qlist`
> (in-house command that lists eligible queues for the user the form template
> is using) processes flying by. What it feels like is the dashboard is
> building every dynamic form on every page load, even if I'm nowhere near
> any of the interactive apps.
>
>
>
> Am I doing something wrong or has anyone else seen this?
>
>
>
> Thanks!
>
> Dan
>
>
>
> --
>
> Dan Dietz
> Senior Scientific Applications Analyst
> ITaP Research Computing, Purdue University
>
>
>
>
> --
>
> Dan Dietz
> Senior Scientific Applications Analyst
> ITaP Research Computing, Purdue University
>


-- 
Dan Dietz
Senior Scientific Applications Analyst
ITaP Research Computing, Purdue University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/ood-users/attachments/20181109/1600b45c/attachment-0001.html>