Howdy! I’m Richard.
I have been quietly learning OOD for about a year and joined discourse today to ask this question.
Rant
I am a user support specialist for a research university. I try to bridge the gap between the clueless Users and the overworked Administrators.
I want the Users to adopt the new, better ways of doing things, but they won’t do it unless I can make it very easy. They don’t want to learn how the cluster works, and they shouldn’t have to. You can open up a Jupyter Notebook in about two clicks on Google Colab with zero understanding of Google Cloud services. That’s what we are competing with now. Every time I open up the Jupyter Notebook interactive app on our OnDemand instance and see how many form elements I have to scroll past (without reading) to find the the submit button, it makes me sad. And the cluster hardware sits idle. We can do better than this!
On the other hand, the Administrators don’t have time to keep updating each and every App form whenever a new software update is available. All the Apps work differently, and all the users want different environments, so the workload grows exponentially. Developing these Apps can get really hard because the more things we try to support the more complex the Apps become. Another university was impressed with something I built and it was just one feature for just one app. And we can’t even share it with them! Because no two sites are doing things the same way. We find ourselves re-developing the same things over and over while other people do the same. It’s madness.
So I’m basically trying to solve all the world’s problems. Thank you for attending my TED talk.
…
Two Problems
I noticed that Open OnDemand has a “Job Composer” feature. I think it’s really neat that we can interact with our scheduler without using the command line. The problem with Job Composer that it doesn’t actually succeed at reducing the user’s workload. In order to make the Job Composer do anything, you have to first write a functional batch script by hand. Doesn’t that kind of defeat the purpose? The user still has to learn how to use the scheduler and the module system, and they still have to type code by hand. Yes, this solution reduces the amount of manual labor - but it does nothing to reduce the barrier to entry for a beginner user. The New Template form in the Job Composer should have form fields for selecting scheduler options and modules. We already know this is possible, because our Interactive Apps do it that way.
I noticed that Interactive Apps have forms that allow the user to select their scheduler options and modules. I think it’s really neat that the user can select curated default values for quickly getting a functional job started. The problem is that this type of interface isn’t scalable. There’s no way for the User to save their combination of choices, if they want to switch back and forth between two combinations, between two apps, across clusters, or share with other Users. And every time the User wants something new that the current form field doesn’t support, they have to wait around until the administrators feel like updating the form. Yes, this solution reduces the barrier to entry - but it does nothing to reduce the manual labor. The Interactive Apps should have a consistent method of saving and loading scheduler and module options. We already know this is possible, because our Job Composer does it that way.
…
One Stone
What if… ? We could “Factor Out” the common scheduler and module form fields from the Interactive Apps, replacing them with a uniform “select a template” field. We then collect all of that logic and put it in the New Template form in the Job Composer. Maybe there are multiple New Template forms to help distribute the logic. In this paradigm, the Administrators would provide their Useful Preset Combinations for the Interactive Apps in the form of Default Templates. Each App would allow the User to choose from the list of default templates and user templates, perhaps filtered to show the relevant ones. The App Form wouldn’t function much differently for a casual user (who doesn’t read anyway).
…
…
TLDR: What if we insert “Job Template” way of doing things between the “select your options” and “submit interactive app”? This would unify lots of apps and help on-ramp users.
I understand that there are both technical and conceptual difficulties, but I do not fear them. My questions are:
-
Is there a planned change to the core engine that will render this discussion obsolete?
-
Is anybody already working on a similar solution to these problems?
-
Does anybody have a better idea?