[Edit: this is my personal opinion about VMs. Installing it on an existing login node is OK. It seems to be a matter of organizational preference. Whatever the choice, the host machine (virtual or not) should be treated just like a login node from a security perspective - where any sensitive files/binaries/etc have to be locked down with the appropriate file permissions and ACLs.]
The best practice is to stand up a VM to install on for isolation reasons. Not only user/memory/network/process/maintenance isolation but also to isolate your login node from all the rpm installations.
The added benefit, just from VMs is, less resource usage. OSC’s login nodes are huge with 250 GB of RAM and 28 cores. OOD doesn’t need nearly that much, though someone else will have to chime in with what it does need. (I’m guessing out of thin air here that ~20-30 GB and ~4 cores is enough but again, just a guess, I’ll try to confirm what an appropriate size is).
Oh! and upgrades. A lot of folks have a prod and test instances, where they can test out configs or changes before deploying them to their users. Again, VM isolation is very good for this.
Like everything else in this world it’s a trade-off. You’re trading isolation for Hardware. Smaller sites may need to install directly on the login or head nodes simply because they can’t spare the hardware for a VM. The VM approach gives you isolation in all sorts of dimensions but at the cost of hardware and often underutilized resources.