JupyterHub in Kubernetes

So I am not sure that I understand the question. I.e. I would like to separate the online notebook development (which is done against jupyterhub) - I.e. an interactive session against a container, from the batch job execution of the notebook. Usually you would like to convert the notebook to python code in the batch case.

Are you looking for EXECUTION of the notebook on a cloud VM? I.e. as a k8s job?

We were in almost exactly the same situation. Workstations at the office. GCP account. We started pushing for more remote work about two years ago, and we built our internal machine learning platform (https://iko.ai) around our workflow solving one bit after the other, precisely because we didn't want to be tied to one library/framework, and without having to pollute our notebooks.

Here's what we're doing:

- No-setup, fresh, notebook environments with the most popular libraries pre-installed: this saves a lot of time, avoids having environments that break, etc. We start afresh with large Docker images, and then people can install libraries.

- Real-time collaboration on notebooks: so we can troubleshoot and pair-program on the same notebook at the same time, we can see cursors, selections, changes, etc. We also use that for our weekly calls where we have the agenda, code snippets, etc. all in the same place. We can talk through agenda items, edit in real-time, add snippets of code, and brainstorm.

- Multiple versions of your notebooks: there's a regression in JupyterLab that we fixed. We're working to make it available in upstream JupyterLab[0].

- Long-running notebook scheduling with output that survives closed tabs and network disruptions: we select the Docker image and the output file. This solves the common problem of launching a long-running notebook, and having to keep the browser tab open and hope the network connection doesn't break. Some people circumvent that porblem by scheduling the notebook and saving the artifacts right from the notebook, but we wanted to be able to have the output streamed whether the tab was closed or not. Bonus: we can watch the same notebook running on multiple laptops.

- Automatic experiment tracking: automatically detects your models, parameters, and metrics and saves them without you remembering to do so or polluting your notebook with tracking code.

- Easily deploy your model and get a "REST endpoint" so data scientists don't tap on anyone's shoulder to deploy their model, and developers don't need to worry about ML dependencies to use the models, and be dragged into the "ML realm". The models also have a page where you can invoke the model by entering JSON and get predictions, or uploading a CSV file and get predictions.

- Build Docker images for your model and push it to a registry to use it wherever you want: currently we push to DockerHub and GitLab

- Monitor your models' performance on a live dashboard: requests, latency, performance, etc.

- Publish notebooks as AppBooks: automatically parametrize a notebook to enable clients to interact with it without exporting PDFs or having to build an application or mutate the notebook. This is very useful when you want to expose some parameters that are very domain specific to a domain expert. For example, some features in a nuclear engineering problem are very domain specific, and a nuclear engineer can bring in a lot of value tweaking the parameters themselves. It also spares our people from spinning up yet another VM on GCP, loading the model, creating a Flask application, setting authentication, etc. The AppBook runs are also tracked, of course.

Much more on our roadmap. We're only focusing on actual problems we have faced serving our clients, and problems we are facing now.

The notebook servers are run on an arbitrary cluster that happens to be ours for now, but we're pretty much approaching it with the following mindset: we are a company that uses this platform, and the platform just happens to be ours.

- [0]: https://github.com/jupyterlab/jupyterlab/issues/5923

Have you taken a look at the "Zero to JupyterHub with Kubernetes" guide? https://zero-to-jupyterhub.readthedocs.io/en/latest/index.ht...

Alternatively, you could check out the QHub project from Quansight. https://qhub.dev/

I think these are less relevant to you since you already have in-house k8s resources, but Coiled and Saturn Cloud offer services that might be interesting to other people with similar goals but who don't have a kubernetes cluster already.

https://coiled.io https://www.saturncloud.io/

How about using Kubeflow? I am not a user of Kubeflow, so might be misinterpreting it. But here is my understanding.

Requirement: Use the notebooks used on laptops in production. Be able to scale as needed. Multi-cloud.

Kubeflow: Kubeflow basically breaks down the notebook into chunks of images, which run as separate pods. These pods are scheduled by Argo workflow - called pipeline. Kubeflow serving enables you to expose the note book as an API. You can schedule different pods w/ different specs, including GPU.

If that meets your need, then you need a simple way to run on laptops - look for miniflow.

There is a tool that is translate from notebook to images directly, so your data scientists do not need to write YAML. Look for miniKF + Kale.

Hope this helps.