pip-compile: Files can be re-compiled in the wrong order when using package groups #28518
Replies: 1 comment 9 replies
-
|
yes, there's no guarantee for updates order. the artifacts function needs to resolve the input files order, not the extract function |
Beta Was this translation helpful? Give feedback.
-
|
Isn't it arbitrary to sort it alphabetically today? Of course we could also add a "packageFilePriority" concept instead but seems unnecessary if we let managers return the order of files themselves. |
Beta Was this translation helpful? Give feedback.
-
|
The pip-compile manager's This ordering just gets lost when multiple packages are updated as a group. It looks like the order of the extracted packages gets reversed here in |
Beta Was this translation helpful? Give feedback.
-
|
This could be used to preserve the index number: Then the pip-compile update command could use this field for sorting. |
Beta Was this translation helpful? Give feedback.
-
|
That seems like a reasonable solution. That index could then be used to re-sort the lists of package files in lib/workers/repository/update/branch/get-updated.ts. The managers themselves then wouldn't need to do anything special; their |
Beta Was this translation helpful? Give feedback.
-
|
I opened #28671 to fix this. It turns out the |
Beta Was this translation helpful? Give feedback.
-
What would you like help with?
I think I found a bug
How are you running Renovate?
Mend Renovate hosted app on github.com
If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.
No response
Please tell us more about your question or problem
When using multiple package files that depend on each other along with package rules that group updates, Renovate can re compile the input files in the wrong order.
Example:
requirements.in:
requirements-dev.in:
And a package rule in the renovate config:
Because awscli 1.32.86 depends on exactly botocore 1.34.86 and requirement-dev.in includes the lock file requirements.txt, when awscli and botocore are updated requirements.in must be re-compiled before requirements-dev.in.
The problem is that the order of package files returned by the pip-compile manager's
extractAllPackageFilesis lost. In the above example renovate attempts to update awscli first, and therefore re-compiles requirements-dev.in first. This causes Renovate to report an error and block merging the pull request; see mbudnek/renovate-test#12.It looks like this happens because the list of package updates gets sorted at some point, and since "awscli" sorts before "botocore" renovate attempts to upgrade them in that order, instead of the order in which the package files were returned by the manager.
Logs (if relevant)
Logs
Beta Was this translation helpful? Give feedback.
All reactions