-
-
Notifications
You must be signed in to change notification settings - Fork 375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error #01: Needed resources are used by other tasks. #1276
Comments
Hi, thanks for reporting ! yes, this might be fixed with #1125, which is currently being rebased on master. the ?_async=true will make the API calls return a task_id. the caller should wait for the task to complete, before issuing a next api call. this can be done with GET /tasks/:id/wait |
Thanks for the reply! Yes, I can. But let me clarify: do I understand correctly that I need to write a shell handler that will process each curl request, take the ID and put it into GET /tasks/:ID/wait ? |
yes, something like this: |
Hello, @neolynx! I apologize for the delay in replying, it took a while to check everything out. I've added
Aptly version 1.5.0+ds1-1+b4 . This is stdout of shell script with
And do I understand correctly that I can't get ID and STATE on this request?
And is it correct that on |
Hi ! I looked in the problem and I think aptly does not support parallel operation, i.e. different clients sending requests at the same time. For this reason in the molior project there is a queue before all aptly operations, making in work with parallel request. There are ideas to introduce a mutex to wait until the resources are available, but I am not sure this is safe and requests will be executed in order (imagine creating, deleting and recreating same repo. those should be executed in order as they arrive). So probably aptly should queue requests and process them in order, which whould be a bigger implementation. In the meanwhile, I there a way to wait for all jenkins tasks to complete, and then do all aptly operations in one go, not using async ? |
I implemented a queue in #1271, this should allow to publish from different clients instead of returning 409. Something like this should work for all commands which support async:
Could you give this a try? Regarding your questions:
The question now is, how to deal with non async requests. Maybe they should go through the queue as well and additionally just wait for the task to finish before returning 200 ? |
The normal non-async requests are not also queued in #1271, and also file uploads should work concurrently now. Could you please give it a try? |
@neolynx Hi! Sorry for the wait and thank you so much for fix! Builds have been falling less frequently. I checked parallel builds with aptly version: 1.5.0+207+g372ce3c4. Error I keep getting an error when doing parallel builds on Jenkins with new error: Maybe, I must open new issue? Jenkins log from the broken job:
aptly json log:
And I want to clarify: when I query curl I get json
Do I understand correctly that this query does not need to reverse |
if you are using parallel uploads / add files, make sure a unique upload directory is used:
if the same folder is used from multiple sources, you might get the the background operation via for the subsequent publishing of the repo, you could use the async operating and poll the corresponsing task until successful (State: 2). |
@neolynx thanks! 😃 The doc says that the directory will be created if it doesn't exist, and a successfully downloaded package will be deleted. So the directory will be deleted too.... Got it. I'll check it out right now. That is, in parallel builds, when one has already uploaded the package and is publishing, the file and directory are deleted. At the same time a file from another build was added to this directory... |
@neolynx it's worked! Your support and advice have been incredible helpful 😃 |
Hi! Need help with publishing packages!
Packages are assembled on Jenkins, and then they are further uploaded and published to the repository using the aptly api:
The problem is that there may be parallel builds, downloads, and publications at the same time, and I get the following error:
aptly — "Error #1: Needed resources are used by other tasks."
curl — (22) The requested URL returned error: 409 Conflict
Have I tried using this version:
#1125 (comment)
and using ?_async=true, but the error persists.
I tried to build aptly from the master branch, errors persist.
What else can I do, or maybe I'm doing something wrong?
The text was updated successfully, but these errors were encountered: