Ondemand Web interface times out

Get Help
1

Previously, our OOD server was on a different VLAN. I moved the server to the new VLAN and did a uninstall / reinstall:

$ sudo yum remove ondemand-python ondemand ondemand-nodejs ondemand-nginx ondemand-apache ondemand-gems ondemand-rubygem-bundler ondemand-passenger ondemand-runtime ondemand-gems ondemand-dex ondemand-ruby

then

$ sudo yum install ondemand ondemand-dex

Here’s what’s installed now:

# rpm -qa | grep ondemand
ondemand-ruby-1.8-1.el7.x86_64
ondemand-python-1.8-1.el7.x86_64
ondemand-nodejs-1.8-1.el7.x86_64
ondemand-rubygem-bundler-1.17.3-1.el7.noarch
ondemand-dex-2.24.0-8.el7.x86_64
ondemand-runtime-1.8-1.el7.x86_64
ondemand-1.8.18-1.el7.x86_64
ondemand-passenger-6.0.4-7.el7.x86_64
ondemand-gems-1.8.18-1.8.18-1.el7.x86_64
ondemand-release-web-1.8-1.noarch
ondemand-gems-1.8.12-1.8.12-1.el7.x86_64
ondemand-nginx-1.17.3-7.p6.0.4.el7.x86_64
ondemand-apache-1.8-1.el7.x86_64

Here’s the portal config:

# cat /etc/ood/config/ood_portal.yml
# Ansible managed
---
servername: ondemand.jhuapl.edu
port: '443'

ssl:
- 'SSLCertificateFile /etc/pki/tls/certs/aplcdhen01.jhuapl.edu.pem'
- 'SSLCertificateKeyFile /etc/pki/tls/private/aplcdhen01.jhuapl.edu.key'
- 'SSLCertificateChainFile /etc/pki/tls/certs/aplcdhen01.jhuapl.edu.pem'

auth:
- 'AuthType openid-connect'
- 'Require valid-user'

dex:
  client_secret: 334389048b872a533002b34d73f8c29fd09efc50
  connectors:
    - type: ldap
      id: ldap
      name: LDAP
      config:
        host: aplid.jhuapl.edu:636
        insecureSkipVerify: false
        userSearch:
          baseDN: cn=users,dc=jhuapl,dc=edu
          filter: "(objectClass=person)"
          username: uid
          idAttr: uid
          emailAttr: mail
          nameAttr: displayName
          preferredUsernameAttr: uid
        groupSearch:
          baseDN: ou=Groups,dc=dom1-proxy,dc=apl-staging
          filter: "(objectClass=group)"
          userMatchers:
            - userAttr: DN
              groupAttr: member
          nameAttr: cn
  frontend:
    theme: ondemand

Here’s the dex config

cat /etc/ood/dex/config.yaml
---
issuer: https://ondemand.jhuapl.edu:5554
storage:
  type: sqlite3
  config:
    file: "/etc/ood/dex/dex.db"
web:
  http: 0.0.0.0:5556
  https: 0.0.0.0:5554
  tlsCert: "/etc/ood/dex/aplcdhen01.jhuapl.edu.pem"
  tlsKey: "/etc/ood/dex/aplcdhen01.jhuapl.edu.key"
telemetry:
  http: 0.0.0.0:5558
staticClients:
- id: ondemand.jhuapl.edu
  redirectURIs:
  - https://ondemand.jhuapl.edu/oidc
  name: OnDemand
  secret: 334389048b872a533002b34d73f8c29fd09efc50
connectors:
- type: ldap
  id: ldap
  name: LDAP
  config:
    host: aplid.jhuapl.edu:636
    insecureSkipVerify: false
    userSearch:
      baseDN: cn=users,dc=jhuapl,dc=edu
      filter: "(objectClass=person)"
      username: uid
      idAttr: uid
      emailAttr: mail
      nameAttr: displayName
      preferredUsernameAttr: uid
    groupSearch:
      baseDN: ou=Groups,dc=dom1-proxy,dc=apl-staging
      filter: "(objectClass=group)"
      userMatchers:
      - userAttr: DN
        groupAttr: member
      nameAttr: cn
oauth2:
  skipApprovalScreen: true
enablePasswordDB: false
frontend:
  dir: "/usr/share/ondemand-dex/web"
  theme: ondemand

After configuring, I updated the configs with the generator:

# /opt/ood/ood-portal-generator/sbin/update_ood_portal -f
cp -p /etc/pki/tls/certs/aplcdhen01.jhuapl.edu.pem /etc/ood/dex/aplcdhen01.jhuapl.edu.pem
chown ondemand-dex:ondemand-dex /etc/ood/dex/aplcdhen01.jhuapl.edu.pem
cp -p /etc/pki/tls/private/aplcdhen01.jhuapl.edu.key /etc/ood/dex/aplcdhen01.jhuapl.edu.key
chown ondemand-dex:ondemand-dex /etc/ood/dex/aplcdhen01.jhuapl.edu.key
Backing up previous Apache config to: '/opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf.20201208T183117'
Generating new Apache config at: '/opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf'
chown root:apache /opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf
chmod 640 /opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf
Generating Apache config checksum file: '/etc/ood/config/ood_portal.sha256sum'
Backing up previous Dex config to: '/etc/ood/dex/config.yaml.20201208T183117'
mv /etc/ood/dex/config.yaml /etc/ood/dex/config.yaml.20201208T183117
Generating new Dex config at: /etc/ood/dex/config.yaml
mv /tmp/dex_config20201208-35617-v69ad3 /etc/ood/dex/config.yaml
chown ondemand-dex:ondemand-dex /etc/ood/dex/config.yaml
chmod 600 /etc/ood/dex/config.yaml
Completed successfully!

Restart the httpd24-httpd service now.

Suggested command:
    sudo systemctl try-restart httpd24-httpd.service httpd24-htcacheclean.service


Restart the ondemand-dex service now.

Suggested command:
    sudo systemctl restart ondemand-dex.service

Then restarted the services.

When trying to access with any browser, it times out (nothing shown on screen).

Here’s the access logs showing the timeout:

10.100.8.123 - - [08/Dec/2020:18:32:07 +0000] "GET / HTTP/1.1" 302 229 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
10.100.8.123 - - [08/Dec/2020:18:32:07 +0000] "GET /pun/sys/dashboard HTTP/1.1" 302 459 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
10.100.8.123 - - [08/Dec/2020:18:32:27 +0000] "-" 408 - "-" "-"

Here’s what’s in the error log:

[Tue Dec 08 18:31:41.728788 2020] [auth_openidc:warn] [pid 35695] [client 10.100.8.123:32786] oidc_clean_expired_state_cookies: state (mod_auth_openidc_state_b7pXnlgkCYX0D4Jc62wN7HyM0vc) has expired (original_url=https://ondemand.jhuapl.edu/pun/sys/dashboard)
[Tue Dec 08 18:31:41.728863 2020] [auth_openidc:warn] [pid 35695] [client 10.100.8.123:32786] oidc_clean_expired_state_cookies: state (mod_auth_openidc_state_sIVHYZLrqBkigdzwRYYu532Mf6U) has expired (original_url=https://ondemand.jhuapl.edu/pun/sys/dashboard)

Any other suggestions on how to track down why it’s timing out?

2

Hey sorry for the delay! As a guess, I’d bet you can’t reach ondemand.jhuapl.edu:5554. You’ve probably opened up 443 only or something and that strange odic port isn’t accessible by your client.

You can open you’re browsers dev tools and see how you’re being redirected and what url is timing out.

1 Like
3

No worries, thanks for the response. I guess I glanced over the need for 5554 for the client. Will report back when that gets opened up (this is behind a firewall I don’t control).