use-only-tar-bz2: true for package cache results in error "nothing provides requested"

[scottyhq]

I just spent a lot of time getting to the bottom of the following issue that showed up last week with v2 of this action, originally I thought it had to do with v2.1 -> v2.2 but this comment in a separate issue from @bollwyvl seems to hold the answer!

unless specifically overridden per-feedstock, conda-forge now now only generates .conda files (conda-forge/conda-forge.github.io#1586) if you've configured use-only-tar-bz2: true for caching in the workflow, it's not going to find any new packages.

Encountered problems while solving:
  - nothing provides requested apache-beam-with-gcp 2.43.0**

It's not clear to me from readme alone why use-only-tar-bz2: true is necessary for caching to work properly? I suspect this will come up quite a bit if new package versions don't have a .tar.gz, so if there is no easy technical solution for caching .conda files a note in the readme.md would be helpful.

[goanpeca]

@scottyhq thanks for raising the issue. When the action was created some time ago, the packages downloaded by conda when using the .conda format where being always redownloaded, that is why the use-only-tar-bz2: true was necessary.

I have not tested (will create a new test to check this) but maybe more recent versions of conda have fixed this issue

@jezdez do you know if this is still the case?

Thanks!

[michaelosthege]

Heads-up: Since conda-forge recently changed their defaults, more and more packages are no longer available as *.tar.bz2 artifacts, including for example numpy >=1.23.5.

When the action was created some time ago, the packages downloaded by conda when using the .conda format where being always redownloaded, that is why the use-only-tar-bz2: true was necessary.

I'm going to try use-only-tar-bz2: true in pymc-devs/pymc#6431 but how can I diagnose if caching of .conda artficats works as it should?

[xylar]

Yes, also seeing this problem. It would be nice to know if no longer using use-only-tar-bz2: true is safe with caching.

[michaelosthege]

Yes, also seeing this problem. It would be nice to know if no longer using use-only-tar-bz2: true is safe with caching.

It is safe for installation, but users must update to recent conda/mamba versions.
My base env for example was still on Python 3.7 and could not update to recent conda/mamba versions supporting .conda artifacts.

For caching I don't know.

[xylar]

Thank you, that's helpful. I imagine python 3.7 will become increasingly hard to support because conda-forge is no longer building for it.

[ctb]

Maybe related (and similar to scientificcomputing/mpi-tutorial#23 I think) - with use-only-tar-bz2 set to true, I was unable to install rust pinned to 1.75.0 (on conda-forge, with linux-64) because it wasn't showing up in the catalog (with mamba search). Setting use-only-tar-bz2 to false made the install work.

[jezdez]

@goanpeca @dbast Do you have a suggestion how to proceed with this? It sounds like we want to more aggressively discourage the use of use-only-tar-bz2 given the existence of the more modern .conda files, and in case of conda-forge the lack of support in the first place.

What is the reason for the following statement?

use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

[xylar]

Yes, I don't think conda environments will work with use-only-tar-bz2 anymore. Many, many conda-forge packages are only available as .conda files.

[goanpeca]

@jezdez at some point in the past when testing caching for this action, when packages where .conda files, conda (at that moment) was redownloading everything, no matter that the pkg_dirs folder containing the .conda files was there.

I have not recently checked if this still the case. If it is not, then we can update those instructions, or even deprecate that flag on the action