Replies: 1 comment
-
Couple other things observed since posting this: Occasionally rqinfo will look like the following (None None None). I have never observed this with Worker:
Here is an example of a job that hit the job_timeout but appears to still have finished after the fact:
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Running the following in my environment:
Python 3.11.5
rq==1.15.1
redis==5.0.1
I need to be able to share database connections between my jobs for each worker. Initially i was using the default Worker but was seeing lots of unexplainable rq.timeouts.JobTimeoutException. I suspected this was due to all of my jobs using the same database engine/connections and Worker forking.
In trying to isolate this further i switched to using SimpleWorker. While i still see some unexplainable timeouts (300 seconds for a query that normally runs in 5 seconds) i am now getting a lot of AbandonedJobError's. The worker appears to be dying but I cannot seem to debug this further.
Here is an example of a job that had a AbandonedJobError. Whats interesting is the amount of time between the job starting and the exception being raised is significantly more then the job_timeout of 300.
I also occasionally see this complaining about my on_failure callback when running rq info and when using SimpleWorker. This is perplexing as the worker and job modules are reside in the same folder.
The worker definitely seems to be dying as it disappears from rqinfo and dosent restart until manually restarting my systemd daemon. The code/infrastructure/jobs are all the same between using Worker and SimpleWorker. Any advice/direction would be appreciated.
Beta Was this translation helpful? Give feedback.
All reactions