Add custom options to bsub (vnc)

Get Help
1

Hi All,
Based on this page, one can add custom options to the job submission command.

Our form.yml contains the following:

---
attributes:
  desktop: "mate"
  bc_vnc_idle: 0
  bc_vnc_resolution:
    required: true
  node_type: null
  ood_var_mem:
    required: true
    label: "Memory"
    value: 1

form:
  - bc_vnc_idle
  - desktop
  - bc_account
  - bc_num_hours
  - bc_num_slots
  - ood_var_mem
  - node_type
  - bc_queue
  - bc_vnc_resolution
  - bc_email_on_started

submit.yml.erb:

---
batch_connect:
  template: vnc
script:
  native:
    - "-M"
    - "<% ood_var_mem.blank? ? 1 : ood_var_mem.to_i %>"

However, just the -n option from bc_num_slots gets added to the native array.

{
  "job_name": "ondemand/sys/dashboard/sys/bc_desktop/ondemand",
  "workdir": "/home/user/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/ondemand/output/adc83917-7613-47e8-94e1-65b19296d26d",
  "output_path": "/home/user/ondemand/data/sys/dashboard/batch_connect/sys/bc_desktop/ondemand/output/adc83917-7613-47e8-94e1-65b19296d26d/output.log",
  "shell_path": "/bin/bash",
  "wall_time": 3600,
  "native": [
    "-n",
    "1"
  ],
  "queue_name": "interactive",
  "email_on_started": false

Even if we hard-code the value of -M in submit.yml.erb or if we hard-code it to bc_num_slots.rb as shown below, only the -n option gets passed to native.

[...]
      def submit(fmt: nil)
        slots = value.blank? ? 1 : value.to_i
        case fmt
        when "torque"
          native = { resources: { nodes: slots } }
        when "slurm"
          native = ["-N", slots]
        when "pbspro"
          native = ["-l", "select=1:ncpus=#{slots}"]
        when "lsf"
          native = ["-n", slots, "-M", "1"]
        else
          native = nil
        end
        native ? { script: { native: native } } : {}
[...]

Does anyone know how to add custom options?
Thanks and best regards

2

I’m looking into why this may not be working for you. Can you confirm that it doesn’t actually submit with the new parameters? It may be submitting the right thing, just not writing out the details correctly.

When you grep execve /var/log/ondemand-nginx/$USER/error.log you’ll see the actual command you submit. (You can see at the end I added some environment variables as a sanity check in my submit.yml.erb as well)

App 1961 output: [2020-03-13 10:12:59 -0400 ]  INFO "execve = [{\"PBS_DEFAULT\"=>\"owens-batch.ten.osc.edu\", \"LD_LIBRARY_PATH\"=>\"/opt/torque/lib64:/opt/rh/rh-nodejs10/root/usr/lib64:/opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/httpd24/root/usr/lib64:/opt/ood/ondemand/root/usr/lib64\"}, \"/opt/torque/bin/qsub\", \"-d\", \"/users/PZS0714/johrstrom/ondemand/data/sys/dashboard/batch_connect/dev/jupyter/output/72d06ecd-2d1b-419c-9786-8e0c2895f55b\", \"-N\", \"ondemand/sys/dashboard/dev/jupyter\", \"-S\", \"/bin/bash\", \"-o\", \"/users/PZS0714/johrstrom/ondemand/data/sys/dashboard/batch_connect/dev/jupyter/output/72d06ecd-2d1b-419c-9786-8e0c2895f55b/output.log\", \"-j\", \"oe\", \"-l\", \"walltime=01:00:00\", \"-l\", \"nodes=1:ppn=1\", \"-v\", \"FOO=BAR,BOCA=LLENA\", \"/tmp/qsub.20200313-1961-n4mfv0\"]"
3

There could be an actual bug here in how ruby deals with hash merging. I took a long look at it and it seems what it writes is what it submits. It’s unclear why nobody has ever ran into this before but it’s either a very serious but or there’s something else going on here. What you’ve given in your submit.yml.erb and form.yml seem just fine.

I’d say try to stop using the smart attribute bc_num_slots and instead use something like ood_var_slots and so you’re native attributes aren’t being merged with anything else.

See if this has any affect.

---
batch_connect:
  template: vnc
script:
  native:
    - "-M"
    - "<% ood_var_mem.blank? ? 1 : ood_var_mem.to_i %>"
    - "-n"
    - "<% ood_var_slots.blank? ? 1 : ood_var_slots.to_i %>"
4

Hi Jeff,
Thank you for investigating. I confirm that it doesn’t submit with the specified options (as checked in /var/log/ondemand-nginx/$USER/error.log).
I’ve also tried to replace bc_num_slots with a custom variable as you proposed - however I get the following error message.

image
image1451×265 28.3 KB

Do I have to replace bc_num_slots in some other files too?
So far I have just replaced them in form.yml and submit.yml.erb.

form.yml:

---
attributes:
  desktop: "mate"
  bc_vnc_idle: 0
  bc_vnc_resolution:
    required: true
  node_type: null
  ood_var_mem:
    required: true
    label: "Memory"
    value: 1
  ood_var_slots:
    required: true
    label: "Slots"
    value: 1

form:
  - bc_vnc_idle
  - desktop
  - bc_account
  - bc_num_hours
  - ood_var_mem
  - ood_var_slots
  - node_type
  - bc_queue
  - bc_vnc_resolution
  - bc_email_on_started

submit.yml.erb:

---
batch_connect:
  template: vnc
script:
  native:
    - "-M"
    - "<% ood_var_mem.blank? ? 1 : ood_var_mem.to_i %>"
    - "-n"
    - "<% ood_var_slots.blank? ? 1 : ood_var_slots.to_i %>"
5

Yea, if it’s not in your form.yml then you can’t use it anywhere.

6

We replaced bc_num_slots with a custom attribute - that worked.
Thanks