HAProxy with Dex issue

Hi,

I’ve been trying to get OnDemand 2.0.20 with Dex to work with HAProxy by following this thread Putting an entire OOD setup behind a web proxy? - #5 by wdpypere
The proxy is ondemand.example.org and ondemand is ondemand000.example.org

At the moment I’m getting the following error(fqdn domain changed):

oidc_util_http_call: curl_easy_perform() failed on: https://o
ndemand.example.org:5554/.well-known/openid-configuration (Failed connect to <ondemand000_ip>:5554; Connection refused)
[Thu Dec 16 11:46:21.434240 2021] [auth_openidc:error] [pid 5028] [client <haproxy_ip>:57804] oidc_provider_static_config: could not retrieve metadata from
url: https://ondemand.example.org:5554/.well-known/openid-configuration

The ood_portal.yml has the following relevant information:

servername: ondemand000.example.org
proxy_server: ondemand.example.org
ssl:

  • ‘SSLCertificateFile “/etc/pki/tls/certs/ondemand000-example-org-chain.pem”’
  • ‘SSLCertificateKeyFile “/etc/pki/tls/private/ondemand000.key”’
  • ‘SSLCertificateChainFile “/etc/pki/tls/certs/ondemand000-example-org-chain.pem”’

auth:

  • ‘AuthType openid-connect’
  • ‘Require valid-user’

dex:
ssl: true
connectors:
- type: ldap
id: ldap
name: LDAP
config:
----redacted config

logout_redirect: /oidc?logout=https%3A%2F%2Fondemand.example.org

The haproxy config:>

frontend main
bind *:80
bind <haproxy_ip>:443 crt /etc/pki/tls/certs/ondemand_example_org.pem ssl
default_backend app
mode http
stats uri /haproxy?stats

backend app
balance roundrobin
mode http
timeout tunnel 7200s
option forwardfor
http-request redirect scheme https unless { ssl_fc }
server ondemand000 <ondemand000_ip>:443 check ssl verify none

Also set the following in /opt/rh/httpd24/root/etc/httpd/conf.d/auth_openidc.conf:

OIDCRedirectURI https://ondemand.example.org/oidc

I’ve tried setting a separate frontend and backend in haproxy for Dex(port 5554) and the message then changes to:

oidc_authenticate_user: the URL hostname (ondemand000.example.org) of the configured OIDCRedirectURI does not match the URL hostname of the URL being accessed (ondemand.example.org): the “state” and “session” cookies will not be shared between the two!

We also have an OnDemand 1.7 server which works fine with haproxy but that doesn’t have Dex and the hostname is the same as the haproxy(ondemand.example.org), if that makes a difference. The HAProxy is version 1.8.

Any ideas or help is appreciated.

Thanks,
Chris

Hi Chris,

I haven’t looked into it specifically but could it be that your problem is the same as in: field "issuer" in generated dex config should be configurable · Issue #1101 · OSC/ondemand · GitHub ?

Regards,

Wouter

Hi Wouter

I had a look into that as well and it seems the issuer in dex/config.yaml has the correct value pointing to the proxy:

issuer: https://ondemand.example.org:5554

Further down there’s also the following which I don’t know if it’s correct:

 web:
   http: 0.0.0.0:5556
   https: 0.0.0.0:5554
   tlsCert: "/etc/ood/dex/ondemand000-example-org-chain.pem"
   tlsKey: "/etc/ood/dex/ondemand000.key"
 telemetry:
   http: 0.0.0.0:5558
 staticClients:
 - id: ondemand000.example.org
   redirectURIs:
   - https://ondemand000.example.org/oidc
   name: OnDemand

Thanks,
Chris

You’re getting connection refused. Can you access this URL through a web browser? Seems like you need to open that port 5554 on your proxy.

Hi Jeff,

Thanks for the hint, I wasn’t able to access the page although the port was open in the firewall. I’ve had to switch to a test proxy server for now but managed to solve it so here’s what I did in case it helps anyone.
I got the openid-configuration page in the browser by adjusting haproxy.cfg but it was still giving a curl error in the httpd log when accessing the proxy base url due to the self-signed cert.
Once the certificate was fixed the following error came up again:

haproxy.example.org_error_ssl.log:

[Wed Dec 22 12:37:06.104516 2021] [auth_openidc:error] [pid 14632] [client <haproxy-ip:33596] oidc_authenticate_user: the URL hostname (ondemand000.example.org) of the configured OIDCRedirectURI does not match the URL hostname of the URL being accessed (haproxy.example.org): the "state" and "session" cookies will not be shared between the two!

To fix it I had to change /opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf and adjust the OIDCRedirectURI to:

  OIDCRedirectURI https://haproxy-test.example.org/oidc

That brings up a page with the error Bad request Unregistered redirect_uri ("https://haproxy-test.example.org/oidc").
The RedirectURIs options had to be adjusted as well in /etc/ood/dex/config.yaml:

  redirectURIs:
  - https://haproxy-test.example.org/oidc

Login page comes up and the only issue remaining was logout which needed the following change in /opt/rh/httpd24/root/etc/httpd/conf.d/ood-portal.conf:

  Redirect "/logout" "/oidc?logout=https%3A%2F%2Fhaproxy-test.example.org"

Were these changes meant to be handled automatically with the logout_redirect variable in ood_portal.yml? I still need to switch back to the production haproxy but :crossed_fingers: no issues come up.

Also here’s the haproxy frontend/backend config for reference:

frontend main
    bind *:80
    bind *:5554 crt /etc/pki/tls/certs/haproxy-test.pem ssl
    bind *:443 crt  /etc/pki/tls/certs/haproxy-test.pem ssl
    default_backend app
    mode http
    stats refresh 10s
    stats show-node
    stats auth <admin_user>:<admin_pass>
    stats uri   /haproxy?stats

backend app
    balance     roundrobin
    mode http
    timeout  tunnel 7200s
    option forwardfor
    http-request redirect scheme https unless { ssl_fc }
    server  ondemand <ondemand_ip> ssl check port 443 verify none

Regards,
Chris