If you are using Slurm, have you considered https://slurm.schedmd.com/elastic_computing.html?
Our current focus is on adding to OnDemand a Kubernetes adapter by July and Cloudy Cluster adapter by a date @jeff.ohrstrom could talk to, and there are discussions about adding an adapter to ISSQ’s Sid cloud platform but that timeline is undetermined. We also did some work on an OpenStack adapter but that work was stalled due to some decisions that still need to be made and a lack of sites to collaborate with.
With any adapter to a cloud resource there are still two problems:
- how do you configure that cloud resource to make it available as a resource to utilize to unprivileged users, say from a login node
- how do you alter OnDemand to utilize this resource
The one site we collaborate with who has an OpenStack instance is interested in using OnDemand strictly as a tool for provisioning VMs for long term use to specific users and providing streamlined access to those VMs, where the users are granted root access and can install whatever software they want and manage as they will. The typical use case was a professor setting up a persistent VM with some software and accounts so students could access and use during a class. Such a VM may exist for months at a time, with the user turning on and shutting down periodically.
This model is very different from using OpenStack as a cloud burst tool for doing interactive work, where the VMs may be ephemeral or even hidden to the user.
One thing that is not clear is whether OnDemand should utilize a service account to interact with the OpenStack API, or whether API credentials should be provisioned automatically for every OnDemand user. We are taking the latter approach with our Kubernetes implementation.
Do you have any interest in participating in or supporting the development of an OpenStack adapter? It isn’t on our timeline in the near term but I’ll make note of you in the draft pull request so we remember when we pick this up.