[OOD-users] Jupyter status?

[Nicklas, Jeremy]

Hi Jason,

As Basil gave a preview of, we are currently working on adding interactive HPC (iHPC) support to the Dashboard app that comes with a default OOD install (hopefully we will roll it out in the next couple of weeks, maybe earlier for some centers who are brave enough to test highly experimental features). Basil only demonstrated launching a Gnome desktop + VNC server within a batch job, but this feature isn't limited to only VNC servers.

At the OSC OnDemand portal in particular we added a Jupyter Notebook Server and an RStudio Server app.

A basic rundown of how the Dashboard handles our Jupyter Notebook setup:

1. The Dashboard has a form that the user can use to set up # cores & # hours they want the batch job to use
2. User clicks "Launch" which generates and submits a "fancy" batch script to the cluster
3. User waits until batch job runs...
4. When the batch job runs it generates a random password, scans for an available port, builds a Jupyter notebook config file, and then launches the Jupyter Notebook server
5. The batch job also outputs all of the necessary connection information to a file with 600 permissions
6. The Dashboard scans for the existence of this file, when it finds this file it parses it to generate the buttons that a user can click and be redirected to the Jupyter notebook server running on the compute node

The Jupyter notebook will have full access to all of the nodes (including GPUs for CUDA support) requested within the batch job.

To get around needing an SSH tunnel, the included OOD Apache proxy has the capability to proxy to any web server on the internal network. By default this feature is turned off as we recommend sys admins configure a whitelist of hosts for it before turning it on:

https://osc.github.io/ood-documentation/infrastructure/ood-portal-generator/configuration.html#configure-reverse-proxy

I may have provided more information than necessary but I figured it may benefit someone else in the mailing list. Hopefully I somewhat answered your questions, otherwise feel free to ask more.

Jeremy Nicklas
Web and Interface App Engineer
Ohio Supercomputer Center (OSC)
A member of the Ohio Technology Consortium
1224 Kinnear Road, Columbus, Ohio 43212
Office: (614) 292-6739 • Mobile: (614) 316-6428 • Fax: (614) 292-7168
jnicklas at osc.edu

Learn more about OSC at https://osc.edu

________________________________________
From: OOD-users [ood-users-bounces at lists.osc.edu] on behalf of Jason F Sommerfield [jasons at psc.edu]
Sent: Wednesday, June 07, 2017 3:48 PM
To: ood-users at lists.osc.edu
Subject: [OOD-users] Jupyter status?

[Again--thanks for the excellent existing functionality in Open OnDemand!]

PSC has a burning need for simple** Jupyter support and OOD would be an
ideal interface for such--that I'd like to encourage development on this
front.  If a general-purpose web proxy of sorts, it might be useful for
other web-interfaced applications (e.g. Hadoop) as well.  The feedback
I'm getting from our user consultants is that even something
Jupyter-specific would be pretty very impactful.

Has this effort yet been planned that there's an ETA, a likely direction
(custom app or fork of JupyerHub?) or any identified opportunity to
contribute?


**we have an awkward work-around and have been investigating JupyterHub

Thanks,
--
Jason Sommerfield | jasons at psc.edu | 1-412-268-9606
Coordinator, Systems & Operations
Pittsburgh Supercomputing Center, 300 S. Craig Street, Pittsburgh PA, 15213
_______________________________________________
OOD-users mailing list
OOD-users at lists.osc.edu
https://lists.osu.edu/mailman/listinfo/ood-users