feat(manifest): change pypi-options to function more like channels and platforms

[tdejager]

In the #1385 we've decided to give this a try.

This PR makes pypi-options work more like channels and platforms. They can now not be specified for the default feature so it's more like:

[project]
name = "pypi-custom-registry"
authors = ["Tim de Jager <[email protected]>"]
channels = ["conda-forge"]
platforms = ["osx-arm64", "osx-64", "linux-64", "win-64"]

[project.pypi-options]
index-url = "https://pypi.org/simple"

They are now still inherited by features even when no-default-feature is enabled. Couple of things we need to decide

1. Do we like this nested structure of should we flatten to: pypi-index-url or something.
2. I've now given a warning when parsing that the old version does nothing. It's not as nice, but I can't do it in validation as I do not have the information anymore. Is this behavior okay, or should we allow adding it to the default feature anyways. Kind of like system-requirements?
3. Does this work as expected? Maybe @zen-xu can verify?

[olivier-lacroix]

On 2/, for me, it makes sense that the default feature works as closely as possible as other named features. So in my mind, defining pypi-options on the default feature should override the project level ones, as it would on other features.
This has the side benefit of make this backward compatible 😄

[olivier-lacroix]

On 1/ I think the nested structure as you propose makes sense

[tdejager]

On 2/, for me, it makes sense that the default feature works as closely as possible as other named features. So in my mind, defining pypi-options on the default feature should override the project level ones, as it would on other features.

This has the side benefit of make this backward compatible 😄

Makes sense!

Except currently you cannot define channels and platforms on the default feature level. I think what you are suggesting makes sense though. That these project settings are inherited by default but can be overidden on a feature level, even the default (because they can be overriden in non-defaults). So that would need to change for platforms and channels.

Currently doing something like:

[project]
name = "bar"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim de Jager <[email protected]>"]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[feature.default]
channels = ["bioconda"]

Gives you:

  × failed to parse project manifest
    ╭─[pixi.toml:9:10]
  8 │
  9 │ [feature.default]
    ·          ───┬───
    ·             ╰── The name 'default' is reserved for the default feature
 10 │ channels = ["bioconda"]
    ╰────

[tdejager]

I just don't know how to specify that nicely though.

This:

[project]
name = "bar"
version = "0.1.0"
description = "Add a short description here"
authors = ["Tim de Jager <[email protected]>"]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

# Would fall under project again
channels = ["bioconda"]

will not work.

[nichmor]

just my 1 cent - I also think that nested structure look more appropriate and intuitive for me

[zen-xu]

It may be more appropriate to rename the default env as the base env.

The config under project is the default config. In this way, there will be no semantic conflict with the base env.

[tdejager]

It may be more appropriate to rename the default env as the base env.

The config under project is the default config. In this way, there will be no semantic conflict with the base env.

Sure, we can talk about it but might be for a different PR :)

[olivier-lacroix]

How about deserialising using uv struct directly @tdejager ? or at least aligning with it somehow. that struct handles merge, which should also simplify things in pixi

[tdejager]

How about deserialising using uv struct directly @tdejager ? or at least aligning with it somehow. that struct handles merge, which should also simplify things in pixi

We should use, the sub-fields directly indeed. I was already doing that in a seperate PR, adding pre-release support, which I will get back to after his. I would like to support Serialize and a subset of the fields. But good pointing out that we want the naming to be the same, for easy conversion and switching if needed.

[tdejager]

@olivier-lacroix I've made the change backwards compatible like you've suggested. So you can also add it to the default feature still.

[zen-xu]

It may be more appropriate to rename the default env as the base env.
The config under project is the default config. In this way, there will be no semantic conflict with the base env.

Sure, we can talk about it but might be for a different PR :)

I create issue in #1416

[ruben-arts]

I'll re-add the [pypi-options] as a working table including a deprecation warning. Taking a lesson from Numpy's deprecation system, and giving that a try.

[ruben-arts]

About the logic, what should happen with this case?:

[project.pypi-options]
index-url = "https://pypi.org/simple"
extra-urls = ["https://pypi.com/simple/extra"]
find-links = [{url = "https://pypi.bla/simple/flat"}, {path = "/path/to/flat"}]

[feature.test.pypi-options]
index-url = "https://prefix.pypi.com/test"
extra-urls = ["https://pypi.com/simple/test"]
find-links = [{url = "https://pypi.bla/simple/test"}, {path = "/path/to/test"}]

[feature.test1.pypi-options]
index-url = "https://prefix.pypi.com/test1"
extra-urls = ["https://pypi.com/simple/test1"]
find-links = [{url = "https://pypi.bla/simple/test1"}, {path = "/path/to/test1"}]

[environments]
env = ["test", "test1"]

What would the pypi-options look like for the env environment?

[zen-xu]

I think only extra-urls and find-links can be set under [feature] as extra candidates

[tdejager]

I think only extra-urls and find-links can be set under [feature] as extra candidates

I don't know... I kind of like the ability to make an environment based on an alternative index, although I must agree with you I do not see a direct use-case for it.

What do you think about the extra-urls and find-links as uv looks from top to bottom? Currently, the first defined feature is top of the list passed to uv.

[zen-xu]

The uv document said

When uv searches for a package across multiple indexes, it will iterate over the indexes in order (preferring the --extra-index-url over the default index)

[olivier-lacroix]

@ruben-arts I think in your example, your project level pypi option should be ignored at feature level (like channels and platform are today).
The project level options will however be picked up by the default feature, and merged with others at environment level.

[olivier-lacroix]

I'll re-add the [pypi-options] as a working table including a deprecation warning. Taking a lesson from Numpy's deprecation system, and giving that a try.

I don’t think it should be deprecated. It should allow one to override the project level option s for the default feature.

[tdejager]

The plan in my head, let's see what @olivier-lacroix and @ruben-arts thinks:

• 1. For pypi-options make it so that we can add it to the project and the default feature.
• 2. In a subsequent PR, add the ability to use [feature.default.channels] etc. (now you cant do this) so that it functions similarly to pypi-options in the default feaure.