Long loading time

Hi All,
On our OOD instance, it may occur that users need to wait some time for OOD to fully load (see example below).


Corresponding entry in /var/log/httpd24/ood-server_access_ssl.log:

- - [15/May/2020:12:47:57 -0700] “GET / HTTP/1.1” 301 249 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- - [15/May/2020:12:47:57 -0700] “GET / HTTP/1.1” 302 238 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- - [15/May/2020:12:47:57 -0700] “GET /pun/sys/dashboard HTTP/1.1” 302 524 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- user1 [15/May/2020:12:47:59 -0700] “GET /pun?code=0.ATEAg2IDy[…]&session_state=6c02[…] HTTP/1.1” 302 238 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- user1 [15/May/2020:12:48:01 -0700] “GET /pun/sys/dashboard HTTP/1.1” 200 10202 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
~51 seconds loading
- user1 [15/May/2020:12:48:52 -0700] “GET /pun/sys/dashboard/assets/application-05ef8da[…].css HTTP/1.1” 200 199278 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- user1 [15/May/2020:12:48:52 -0700] “GET /pun/sys/dashboard/assets/application-18125[…].js HTTP/1.1” 200 238848 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- - [15/May/2020:12:48:52 -0700] “GET /public/Blue_Roche_logo_h30px@2x.png HTTP/1.1” 200 1991 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- user1 [15/May/2020:12:48:52 -0700] “GET /pun/sys/dashboard/apps/icon/shared_jupyter HTTP/1.1” 200 5004 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- user1 [15/May/2020:12:48:53 -0700] “GET /pun/sys/dashboard/apps/icon/shared_rstudio-srv HTTP/1.1” 200 12828 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- - [15/May/2020:12:48:53 -0700] “GET /public/ec-hub.png?timestamp=1589572131 HTTP/1.1” 200 375363 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”
- - [15/May/2020:12:48:53 -0700] “GET /public/OpenOnDemand_powered_by_RGB.svg HTTP/1.1” 200 159559 “https://ood-server/pun/sys/dashboard” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”

There are no relevant details in /var/log/ondemand-nginx/.
This issue only appears when /pun?code=[...] gets called.
The actual authentication took 2.3 seconds (we use OpenID-Connect for authentication) - most of the delay seems to come from the PUN process initialization.
Is this a known issue?
Let me know in case you need more information about any specific configuration file.
In case the openidc-auth configuration file helps:

# openauth configurations
OIDCProviderMetadataURL https://login.microsoftonline.com/[...]/.well-known/openid-configuration
OIDCClientID […]
OIDCClientSecret […]

OIDCRedirectURI https://ood-server/pun
OIDCCryptoPassphrase […]

OIDCScope “openid email”
OIDCRemoteUserClaim […]

Thanks

Do you have any interactive apps that use a form.yml.erb and whose erb does something expensive?