Hello everybody!
our group recently encountered an issue while performing a big number of sequential DataSHIELD commands.
Our programs would usually run for around 1.5 hours but are occasionally stopped at random points during the run by a message starting with “Waiting…”.
I’m not sure whether this occurs with other commands but we observed the issue while using dataFrameSubset, which is also by far the most used command in our script.
After a random time (till now something between 10min and 3h) the run will continue but every command afterwards will result in a error.
When looking at the error via datashield.errors() I get “Cannot connect: connection refused (Connection refused)”.
Has something like this occured before? And if yes, what would a solution look like?
Asking users in our consortium didn’t result in a solution so I’d like to increase the range.
Till now we could ensure that our server is definitely not overwhelmed by the workload.
Best regards,
Stephan
Hi,
That means that the R server session has died, most likely because of (server) memory shortage. That is a common issue when computing with all data in the R memory. Either increase the memory of the (hardware) server. Or find a way to make your computations in a memory efficient way (depends on your data structure, data source, and type of computations). You can also remove the unsused R objects from your server-side R sessions with ds.rm(), to release some memory.
Regards
Yannick
Hi Yannick! Thank you for your reply!
You’re right that we never used ds.rm() somewhere so that actually might be a solution!
We previously tried changing our RAM but that didn’t change the issue.
I’ll try to increase it tomorrow by more than last time and insert ds.rm() in the code.
I’ll give an update if the issue will persist.
Thank you very much!
Regarding the random time after which it fails: are there several users running analysis at the same time (i.e. concurrently)? That would explain why it fails faster some time.
FYI I have started writing a cookbook, and you can have a look at the How to Improve Performance topic.
Regards
Yannick