Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Incorrect wheel downloaded #646

Open
nishantvarma opened this issue Mar 7, 2024 · 5 comments
Open

[BUG] Incorrect wheel downloaded #646

nishantvarma opened this issue Mar 7, 2024 · 5 comments

Comments

@nishantvarma
Copy link

nishantvarma commented Mar 7, 2024

Buildout downloads Python 3.8 wheel in Python a 3.10 environment.

We have no distributions for pkg that satisfies 'pkg==7.3.0'.
root: Reading http://admin:[email protected]/pkg/
root: Found link: http://admin:[email protected]/pkg/pkg-7.3.2-py38-none-any.whl
root: Found link: http://admin:[email protected]/pkg/pkg-7.3.0-py310-none-any.whl
root: Found link: http://admin:[email protected]/pkg/pkg-7.3.0-py38-none-any.whl
root: Found link: http://admin:[email protected]/pkg/pkg-7.3.0-py3-none-any.whl
root: pkg==7.3.0 does not match pkg 7.3.2
root: pkg==7.3.0 does not match pkg 7.3.1
Getting distribution for 'pkg==7.3.0'.
Fetching pkg 7.3.0 from: http://admin:[email protected]/pkg/pkg-7.3.0-py38-none-any.whl
root: Downloading http://admin:[email protected]/pkg/pkg-7.3.0-py38-none-any.whl
Got pkg 7.3.0.

Note: We use bytecode-wheels, so version is important.

@nishantvarma
Copy link
Author

nishantvarma commented Mar 7, 2024

I also asked in pypa/setuptools#4264. However, it seems to be an issue with how Buildout does things -- it should be able to filter out irrelevant packages much earlier (it does so for C-Python packages etc.). Pip also downloads packages correctly.

@icemac
Copy link
Contributor

icemac commented Mar 8, 2024

This seems to happen since Python 3.8 – for yet unknown reasons.

Please try the following setting, either in ~/.buildout/default.cfg or in your project's buildout.cfg:

[buildout]
abi-tag-eggs = true

It puts the eggs into separate directories depending on their ABI version.

@nishantvarma
Copy link
Author

nishantvarma commented Mar 8, 2024

Thanks, @icemac. I tried that -- it still downloads the Python 38 wheel. It does create a folder called eggs/cp310/pkg-7.3.0-py3.10.egg but it the Python 38 contents. I think it should be solved during resolution/download (like pip does for the same package)? Let me know if you have any other suggestions.

As of now, the only workaround I can think of is to pre-fetch the wheels using Pip or something; but that will have to be done outside of Buildout. A simple example is:

for pkg in $(buildout -c buildout.cfg annotate versions | grep "=" |sed "s/= /==/g"); do
    pip download --no-deps --ignore-requires-python -d downloads "$pkg" || true
done

It can even be a plugin.

@nishantvarma nishantvarma changed the title Incorrect wheel downloaded (wrong python version) [BUG] Incorrect wheel downloaded (wrong python version) Mar 9, 2024
@nishantvarma nishantvarma changed the title [BUG] Incorrect wheel downloaded (wrong python version) [BUG] Incorrect wheel downloaded Mar 9, 2024
@gotcha
Copy link
Member

gotcha commented Mar 19, 2024

Did you check if you would have a cached wrong download ?

This could happen if you have set download-cache in your .buildout/download-cache.

@nishantvarma
Copy link
Author

nishantvarma commented Mar 20, 2024

I always set download-cache to downloads/dist. When it is empty, it downloads pkg-py38.whl to cache and converts it as eggs/pkg-py310.eggs. If I manually download pkg-py310.whl to cache, it picks that and converts correctly (it still checks server for something else -- despite disabling newest -- but that doesn't seem to impact) . We wrote a wrapper that populates the cache correctly ATM. I think the download logic has some problems. Didn't get time to check further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants