New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PSOCK fails with slower functions #524
Comments
It sounds like you're implying that a "slow" function gives the error. I don't see how the processing time would result in Instead, I guess that one of your parallel workers crashes/core dumps for one reason or the other, e.g. running out of memory. If that happens, you'll get the same error message, e.g. > cl <- parallelly::makeClusterPSOCK("remote.worker.org")
> cl
Socket cluster with 1 nodes where 1 node is on host 'hb-x1-2016' (R version 3.6.3 (2020-02-29), platform x86_64-pc-linux-gnu)
> library(future)
> plan(cluster, workers = cl)
> f <- future(quit("no"))
> value(f)
Error in unserialize(node$con) :
ClusterFuture (<none>) failed to receive results from cluster RichSOCKnode #1 (PID 4893 on 'hb-x1-2016'). The reason reported was 'error reading from connection'
> traceback()
6: stop(ex)
5: receiveMessageFromWorker(future, ...)
4: result.ClusterFuture(future)
3: result(future)
2: value.Future(f)
1: value(f) BTW, what does I recommend that you try to reproduce the problem with a smaller example using a smaller data set with fewer package dependencies. You could also add near-live progress updates using progressr to narrow how far the function calls get before you get that error. |
Describe the bug
When I try a fast function, like a regression with mtcars, PSOCK woks well on DigitalOcean droplets. But, if I connect to an S3 space, PSOCK returns this error
Reproduce example
Expected behavior
It should have returned lm object. I tested the code by running it from ssh (terminal) on each droplet , and it works.
Session information
I tested both on laptop and server
Laptop: Pop!_OS 21.04
Server: Ubuntu Server 20.04
The text was updated successfully, but these errors were encountered: