Rstudio rstudio-server-2021.09.0+351-1.x86_64 issues

Hello. This issue might be that I’m running too new a version of rstudio server (I grabbed the latest available), but I get an error launching it through ondemand where the login screen appears with a Sign-In failed page and the warning message: Error: Temporary server error, please try again.

I’ve been following the directions on this page to get past some of the errors in configuration: Unable to launch fully containerized Rstudio - #20 by dugan as well as RStudio when launched without Singularity is having strange troubles with authentication - #32 by fenz

I configured the view.html.erb, submit.yml.erb, and before.sh.erb files as described in that second document, and when I view the page with the Inspector in firefox, I see

<input type="hidden" name="csrf-token" value="long hex string">

So, I think the settings that people are getting for the token are working correctly, but something else is clearly wrong that I can’t find. Is this just a function of the newest version of Rstudio, or something else possibly? Where might I look for more debugging? There’s nothing in the system log or the output log file from slurm.

Thanks.

I just tried this new version using the same config that works with 1.4.x and found that rserver failed to
start. But it did log a message to syslog:

rserver[7757]: ERROR Could not find details for server user ‘rstudio-server’

so I added one more server option: “–server-user $USER” and now everything seems to be working
with this version of Rstudio.

You might have a different problem because it sounds like you got the server to start but authentication
is failing. You might want to try getting 1.4.x working first.

My first solution was not using token generated in before.sh.
If you change before and submit files make sure the variable names are correct.
In the view.html I use ‘value="<%= csrftoken %>"/>’ but in the before.sh example (RStudio when launched without Singularity is having strange troubles with authentication - #42 by jeff.ohrstrom) is using ‘<%= csrf_token %>’
If you share all the parts you added we can check if there’s any issue.

Thanks everyone. I don’t know how to get a hold of the RH7 version of rstudio 1.4, so I haven’t gone anywhere with that piece. Is there a link somewhere to it? The only one I saw on the rstudio site was for Debian.

Let me try to post the options I’ve got in my files (if I can get the discourse formatting right)

In before.sh.erb

<%-
  require 'securerandom'
  csrf_token = SecureRandom.uuid
-%>
export csrf_token="<%= csrf_token %>"

In script.sh.erb:

#!/usr/bin/env bash
module purge

# load RStudio Server

# load R
module load <%= context.Rapp %>

#
# Start RStudio Server
#

# PAM auth helper used by RStudio
export RSTUDIO_AUTH="${PWD}/bin/auth"

# Generate an `rsession` wrapper script
export RSESSION_WRAPPER_FILE="${PWD}/rsession.sh"
(
umask 077
sed 's/^ \{2\}//' > "${RSESSION_WRAPPER_FILE}" << EOL
  #!/usr/bin/env bash

  # Log all output from this script
  export RSESSION_LOG_FILE="${PWD}/rsession.log"
  exec &>>"\${RSESSION_LOG_FILE}"

  # Launch the original command
  echo "Launching rsession..."
  set -x
  exec rsession --r-libs-user "${R_LIBS_USER}" "\${@}"
EOL
)
chmod 700 "${RSESSION_WRAPPER_FILE}"

# Set working directory to home directory
cd "${HOME}"

# Create a unique $TMPDIR for runtime files
#export TMPDIR="$(mktemp -d)"

mkdir $TMPDIR/rstudio

# Create a new database.conf file to live in $TMPDIR
echo "directory=${TMPDIR}/rstudio" > $TMPDIR/rstudio/database.conf
# Output debug info
module list

# Launch the RStudio Server
echo "Starting up rserver..."
set -x
/usr/lib/rstudio-server/bin/rserver \
  --www-port ${port} \
  --auth-none 0 \
  --auth-pam-helper-path "${RSTUDIO_AUTH}" \
  --auth-encrypt-password 0 \
  --rsession-path "${RSESSION_WRAPPER_FILE}" \
  --server-data-dir "${TMPDIR}/rstudio" \
  --server-working-dir "${TMPDIR}/rstudio" \
  --server-user "${USER}" \
  --database-config-file "${TMPDIR}/rstudio/database.conf" \
  --secure-cookie-key-file "${TMPDIR}/rstudio-server/secure-cookie-key"

In view.html.erb:

<script>
  document.cookie = "csrf-token=<%= csrf_token %>; path=/rnode/<%= host %>/<%= port %>; secure";
</script>
<form action="/rnode/<%= host %>/<%= port %>/auth-do-sign-in" method="post" target="_blank">
  <input type="hidden" name="username" value="<%= ENV["USER"] %>">
  <input type="hidden" name="password" value="<%= password %>">
  <input type="hidden" name="staySignedIn" value="1">
  <input type="hidden" name="appUri" value="">
  <input type="hidden" name="csrf-token" value="<%= csrf_token %>"/>
  <button class="btn btn-primary" type="submit">
    <i class="fa fa-registered"></i> Connect to RStudio Server
  </button>
</form>

well, I just kinda made up a url based on the rstudio github releases and found the rpm for 1.4.1717, so I installed that one but get the same error.

If I view the connection.yml file and enter my username and the generated password. I get an rstudio server screen that says The requested page was not found. I’m thinking I have a general rstudio problem. I’m going to try launching it manually from the command line and see if I can get in.

I’ve managed to get rstudio server working on the command line and was able to connect to it locally. I launched it through OnDemand and get the “Temporary Server Error” message. If I manually enter the username/password here (using the connection.yml info), I get the requested page error. If I then try to connect locally on the system (without going through ondemand), using those credentials and port information, I am able to log in and get the Rstudio session. So, I believe that OnDemand is launching the application correctly, but something about the way the connection is forwarded is not working correctly.

Maybe that helps focus the troubleshooting a little.

It really sounds like the csrf token isn’t getting passed. Does the generated connection.yml file have a “csrf_token: …” line?

It does not. I have a host, port, and password line.

What does your submit.yml.erb file look like?

I was just looking at that as you were replying.

---
conn_params:
  - csrf_token
batch_connect:
  template: "basic"
script:
  queue_name: <%= custom_queue %>
  native: [ "--gres=gpu:<%= gpus.blank? ? 0 : gpus %>", "--mem=<%= memory %>G", "--cpus-per-task=<%= bc_num_slots.blank? ? 1 : bc_num_slots %>" ]

also, just to check, I did an echo of csrf_token in the script file just to see if that environment variable was being set to anything, and it is:
c8fbe8ec-27c2-471d-8101-9317e20e53a4

OK, conn_params needs to be inside batch_connect

---
batch_connect:
  conn_params:
    - csrf_token
  template: "basic"
...

Try that.

Aha, that was it. I was reviewing the other posts and the batch connect options page at the same time and was about to post that when you did it. That was definitely the problem. I’ve got csrf_token in the connection.yml file, and it’s working in Ondemand as well now.

Thanks for your help on that.

FWIW, Rstudio-server 2021.09.0-351 also works.

There’s one more rserver option you will want to add: “–auth-timeout-minutes 0”. This defaults to 60 and
affects the lifetime of the csrf-token cookie. We don’t want our cookie to expire because if it does the server will generate a new one and we won’t be able to get back in. Setting this option to 0 causes the
cookie lifetime to be controlled by another option “–auth-stay-signed-in-days” which defaults to 30 days.

Fantastic. Thanks. I updated that as well.