FYI: Testing the reverse proxy configuration with netcat fails if you have IPv6 disabled.
On our cluster all our nodes explicitly have IPv6 disabled. When trying to test the reverse proxy setup for OOD, we get this type of failure:
[user@node ~]$ nc -l 5432
close: Bad file descriptor
This is due to netcat attempting to close a non-existent IPv6 listening socket. The work around is to issue the command with the “-4” parameter to force nc to only open an IPv4 listening socket.
So in the batch connect jobs, we launch the app, and use netcat to verify the port is open through the wait_until_port_used function.
Here’s the nc command we’re trying to run for reference.
$1 is the host we passed in. Maybe if it was an ip (a version 4 ip) it would work. Where are you using wait_until_port_used function?
Instead of passing in ${host} you can try $(hostname -i | awk '{print $2}') or even just 127.0.0.1 if you’re sure it’ll be local to the node. Here’s where we use it in our jupyter app for reference.
Oh sorry, I’m sorry, are you saying your having an issue with an actual app or with some trival test to make sure things are setup right?
It could cause issues as I’ve indicated above. But as you can see the nc command there, you may try running it on a compute node and see if it errors with the same complaint as listen does.
Thanks for the update, we are still in the process of setting up the interactive app portion, so nothing breaking yet, just the test.
If I am understanding correctly, OOD will be using ‘nc’ to call the node for connectivity verification vs. ‘nc -l’ being launched as a listener on the node. (Like the example for testing reverse proxy shows.)
If that’s correct, then I doubt there will be an issue. From what I can find, this is only an error that happens with netcat is started as a listener and tries to clean up a socket that was never started. (e.g. the IPv6 socket)