billiard.exceptions.WorkerLostError: Worker exited prematurely: exitcode 1

[shaMiaoMiao]

celery billiard error

I found a billiard error when use celery with version == '5.1.2' . main msg like:

Traceback (most recent call last): File "/home/work/pyenv/celery/lib/python3.7/site-packages/billiard/pool.py", line 1267, in mark_as_worker_lost human_status(exitcode), job._job), billiard.exceptions.WorkerLostError: Worker exited prematurely: exitcode 1 Job: 172.

Detail version is :

billiard == 3.6.4.0

celery == 5.1.2

And start script is :

celery multi restart secondary_work -c 30 -A metric_celery -Q secondary_queue --pidfile="$HOME/run/celery/%n.pid" --logfile="$HOME/log/celery/%n%I.log" --max-tasks-per-child=50

I use two queue on different server.

Looks like a billiard process problem and its a occasional anomaly, i try to set -c 20 and max-tasks-per-child=50, but it's helpless, i am not sure whether need to set a large num.

Does anyone has a good idea?

Thanks !

[open-collective-bot]

Hey @shaMiaoMiao 👋,
Thank you for opening an issue. We will get back to you as soon as we can.
Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors.
If you require immediate assistance please consider sponsoring us.

[auvipy]

probably this is a billiard issue, can you try 5.2.0rc1 and report again? can you check billiard repo? i could migrate this issue there. also please share more detail about your deployment and codes according to issue template

[shaMiaoMiao]

probably this is a billiard issue, can you try 5.2.0rc1 and report again? can you check billiard repo? i could migrate this issue there. also please share more detail about your deployment and codes according to issue template

Thanks for ur reply. It's useless to update celery version to 5.2.0rc1. I update codes according and some informations in issue template, looking forward to your help. Thank you very very much!