Checking for apps functioning/status


I’m just wrapping up an upgrade to and ondemand server and am about to begin the tedious process of starting and verifying that each app works as expected. As I do that it made me wonder how that might be automated. Does anyone have any elegant solutions to monitoring ondemand to make sure it is online and functional? Or just something more robust and less boring than manually starting each app and testing it?

Our base of installed apps is steadily growing and is likely to grow much larger over time as we begin to deploy user developed applications. Doing this manually won’t be an option for much longer. My default approach will be to watch logs more closely, but that leaves us in a situation of having broken things discovered by users, which is not optimal (but is the default way we monitor a lot of stuff, so …)


We have started working on building up a set of automated acceptance tests to run against OSC’s OnDemand install and against docker images that we can use daily on certain branches of the code base. Right now they are using selenium but in the future they may use a mix of tools. I wonder if this work could be done in a way to enable other sites like yours to have a test suite.

What is an example test that you want to do? Are you mostly interested in testing the interactive apps i.e. actually starting MATLAB or the Desktop with different sets of user inputs and verifying that the apps actually launch without error? For that, there might be another solution for that using ReFrame that we are also exploring.