File Upload Issue

We’re having issues with file upload. It appears that file upload proceeds to 100%, and then fails. We tested this with 1gb, 2gb, 3gb, etc. Upload works up to a 2gb file, but 3gb and larger fails. We’ve tried this on both OOD 2.0.8 and 2.0.13. Failed upload results in a partial file upload. Also, the resulting partial file cannot be deleted via the Files app.

Are you able to share the logs for this? The relevant logs would be:
/var/log/ondemand-nginx/$user/access.log
/var/log/ondemand-nginx/$user/error.log

Seeing any errors or strange info messages there would be a start.

Also, curious if you’ve replicated on more than one browser or if this is happening across them for you.

Also, to be sure for troubleshooting, have you set the nginx_file_upload_max to 2 in /etc/ood/config/nginx_stage.yml or changed the value from default in any way?

Could that be a filesystem limit?

Can you copy a file > 2 GB using some other means to the same filesystem?

That number sticks in my head from long ago as a maximum file size.

Thanks for all the questions. @travert there aren’t any telling errors in the logs and we’ve tried chrome and firefox with same result.

I do find that after the file upload is reported failed, I close the file upload window and the web server hangs, and is only resolvable by restarting pun. Oddly enough, after a failed 3Gib file upload, I find a 3Gib file called “RackMultipart…” in /tmp.

Also, the ngnix_file_upload_max is set to 10737420000.

@bennet yes we tried SFTP and that works just fine.

The only thing I can suppose is that the file is uploaded successfully to memory, but during transfer to disk it times out or fails for some reason. Another question, is there a memory dependency for file upload?

Thanks for the answers @maflister.

The way the file uploads is to first upload to /tmp and then mv that file. So, that you see the file in /tmp but it never gets to the correct location seems to indicate something is going wrong at the mv stage.

I’m curious, when it fails does it throw any kind of message? Do you see anything like what’s in this GitHub issue which sounds similar to what you are seeing:

@travert I think that is the exact issue I have. I will wait to see resolution of that github issue. I’m curious how this hasn’t been reported more often. Perhaps most users don’t work with large files?

Thanks for letting us know you are also experiencing so we can prioritize it.

I’m not sure why more reports of this hasn’t come up, but I would guess the same reason of most users just not tending to need larger uploads possibly. But, I do know we need to get this patched quickly so we really appreciate you bringing this up and apologize for the issue.

1 Like