Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

conda-build version 24.5.0 assigns values to frozendict keys: results in internal errors #5342

Closed
2 tasks done
Tracked by #5319
jakirkham opened this issue May 14, 2024 · 3 comments · Fixed by #5345
Closed
2 tasks done
Tracked by #5319

conda-build version 24.5.0 assigns values to frozendict keys: results in internal errors #5342

jakirkham opened this issue May 14, 2024 · 3 comments · Fixed by #5345
Assignees
@kenodegard
Labels
in-progress issue is actively being worked on severity::1 blocker; broken functionality with no workaround source::community catch-all for issues filed by community members type::bug describes erroneous operation, use severity::* to classify the type
Milestone
24.5.x

Comments

@jakirkham
Copy link
Member

jakirkham commented May 14, 2024 •
edited

Checklist

  • I added a descriptive title
  • I searched open reports and couldn't find a duplicate

What happened?

Starting in conda-build version 24.5.0, frozendicts have been used in a few places. For example PR ( #5284 ) looks potentially relevant. Unfortunately it appears they are being used in places where assignments happen, which causes errors

Fore example, recently tried to build this recipe using this variant file on Windows. The exact same recipe worked a month ago with conda-build version 24.3.0, but with conda-build version 24.5.0 it encounters the following error:

Traceback (most recent call last):
  File "C:\Miniforge\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(execute())
  File "C:\Miniforge\lib\site-packages\conda_build\cli\main_build.py", line 590, in execute
    api.build(
  File "C:\Miniforge\lib\site-packages\conda_build\api.py", line 250, in build
    return build_tree(
  File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3638, in build_tree
    packages_from_this = build(
  File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 2696, in build
    output_d["files"] = set(output_d["files"]) - to_remove
TypeError: 'frozendict.frozendict' object does not support item assignment

More details (with full logs) about both conda-build version 24.3.0 and 24.5.0 builds in this comment: conda-forge/cuda-nvtx-feedstock#16 (comment)

Conda Info

active environment : base
    active env location : C:\Miniforge
            shell level : 1
       user config file : C:\Users\VssAdministrator\.condarc
 populated config files : C:\Miniforge\.condarc
                          C:\Users\VssAdministrator\.condarc
          conda version : 24.3.0
    conda-build version : 24.5.0
         python version : 3.10.14.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=x86_64_v4
                          __conda=24.3.0=0
                          __win=0=0
       base environment : C:\Miniforge  (writable)
      conda av data dir : C:\Miniforge\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : C:\Miniforge\pkgs
                          C:\Users\VssAdministrator\.conda\pkgs
                          C:\Users\VssAdministrator\AppData\Local\conda\conda\pkgs
       envs directories : C:\Miniforge\envs

Conda Config

==> C:\Miniforge\.condarc <==
aggressive_update_packages:
  - ca-certificates
  - certifi
channels:
  - conda-forge
show_channel_urls: True
conda_build:
  error_overlinking: False
  pkg_format: 2
  zstd_compression_level: 19

==> C:\Users\VssAdministrator\.condarc <==
add_pip_as_python_dependency: False
auto_update_conda: False
channel_priority: strict
channels:
  - conda-forge
show_channel_urls: True
conda_build:
  error_overlinking: False
  pkg_format: 2
  zstd_compression_level: 19

==> envvars <==
allow_softlinks: False
solver: libmamba
bld_path: D:\\bld\\

Conda list

# packages in environment at C:\Miniforge:
#
# Name                    Version                   Build  Channel
anaconda-client           1.12.3             pyhd8ed1ab_0    conda-forge
anaconda-project          0.11.1             pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310h00ffb61_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py310h8d17308_0    conda-forge
chardet                   5.2.0           py310h5588dad_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.3.0          py310h5588dad_0    conda-forge
conda-build               24.5.0          py310h5588dad_1    conda-forge
conda-env                 2.6.0                         1    conda-forge
conda-forge-ci-setup      4.4.7           py310h82d9320_100    conda-forge
conda-forge-metadata      0.8.0              pyhd8ed1ab_0    conda-forge
conda-index               0.4.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-oci-mirror          0.1.0              pyhd8ed1ab_0    conda-forge
conda-pack                0.7.1              pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.14             pyh1a96a4e_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
filelock                  3.14.0             pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h181d51b_0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
frozendict                2.4.4           py310ha8f682b_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py310h5588dad_3    conda-forge
jsonschema                4.22.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py310h5588dad_0    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
lcms2                     2.15                 h3e3b177_1    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libarchive                3.7.2                h313118b_1    conda-forge
libcurl                   8.7.1                hd5e4a3a_0    conda-forge
libdeflate                1.18                 hcfcfb64_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libjpeg-turbo             2.1.5.1              hcfcfb64_1    conda-forge
liblief                   0.14.1               he0c23c2_1    conda-forge
libmamba                  1.5.8                h3f09ed1_0    conda-forge
libmambapy                1.5.8           py310h04f2035_0    conda-forge
libpng                    1.6.43               h19919ed_0    conda-forge
libsolv                   0.7.28               h12be248_2    conda-forge
libsqlite                 3.45.2               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libtiff                   4.5.1                h6c8260b_1    conda-forge
libwebp-base              1.4.0                hcfcfb64_0    conda-forge
libxcb                    1.13                          0    conda-forge
libxml2                   2.12.6               hc3477c8_1    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              he774522_1000    conda-forge
m2-bash                   5.2.015.1            hd8ed1ab_0    conda-forge
m2-brotli                 1.1.0.1              hd8ed1ab_0    conda-forge
m2-ca-certificates        20230311.1           hd8ed1ab_1    conda-forge
m2-conda-epoch            20230914               0_x86_64    conda-forge
m2-coreutils              8.32.5               hd8ed1ab_0    conda-forge
m2-curl                   8.2.1.1              hd8ed1ab_0    conda-forge
m2-db                     5.3.28.4             hd8ed1ab_0    conda-forge
m2-file                   5.45.1               hd8ed1ab_0    conda-forge
m2-filesystem             2023.02.07.1         hd8ed1ab_0    conda-forge
m2-findutils              4.9.0.3              hd8ed1ab_0    conda-forge
m2-gcc-libs               11.3.0.4             hd8ed1ab_0    conda-forge
m2-gdbm                   1.23.1               hd8ed1ab_0    conda-forge
m2-gettext                0.22.1               hd8ed1ab_0    conda-forge
m2-git                    2.42.0.2             hd8ed1ab_0    conda-forge
m2-gmp                    6.3.0.1              hd8ed1ab_0    conda-forge
m2-gzip                   1.13.1               hd8ed1ab_0    conda-forge
m2-heimdal                7.8.0.4              hd8ed1ab_0    conda-forge
m2-heimdal-libs           7.8.0.4              hd8ed1ab_0    conda-forge
m2-info                   7.0.3.3              hd8ed1ab_0    conda-forge
m2-less                   643.1                hd8ed1ab_0    conda-forge
m2-libasprintf            0.22.1               hd8ed1ab_0    conda-forge
m2-libbz2                 1.0.8.4              hd8ed1ab_0    conda-forge
m2-libcbor                0.9.0.1              hd8ed1ab_0    conda-forge
m2-libcurl                8.2.1.1              hd8ed1ab_0    conda-forge
m2-libdb                  5.3.28.4             hd8ed1ab_0    conda-forge
m2-libedit                20221030_3.1.1       hd8ed1ab_0    conda-forge
m2-libexpat               2.5.0.1              hd8ed1ab_0    conda-forge
m2-libffi                 3.4.4.1              hd8ed1ab_0    conda-forge
m2-libfido2               1.13.0.1             hd8ed1ab_0    conda-forge
m2-libgcrypt              1.9.4.1              hd8ed1ab_0    conda-forge
m2-libgdbm                1.23.1               hd8ed1ab_0    conda-forge
m2-libgettextpo           0.22.1               hd8ed1ab_0    conda-forge
m2-libgpg-error           1.47.1               hd8ed1ab_0    conda-forge
m2-libiconv               1.17.1               hd8ed1ab_0    conda-forge
m2-libidn2                2.3.4.2              hd8ed1ab_0    conda-forge
m2-libintl                0.22.1               hd8ed1ab_0    conda-forge
m2-liblzma                5.4.4.1              hd8ed1ab_0    conda-forge
m2-libnghttp2             1.56.0.1             hd8ed1ab_0    conda-forge
m2-libopenssl             3.1.2.1              hd8ed1ab_0    conda-forge
m2-libp11-kit             0.25.0.1             hd8ed1ab_0    conda-forge
m2-libpcre2_8             10.42.1              hd8ed1ab_0    conda-forge
m2-libpsl                 0.21.2.1             hd8ed1ab_0    conda-forge
m2-libreadline            8.2.001.3            hd8ed1ab_0    conda-forge
m2-libsqlite              3.42.0.2             hd8ed1ab_0    conda-forge
m2-libssh2                1.11.0.1             hd8ed1ab_0    conda-forge
m2-libtasn1               4.19.0.1             hd8ed1ab_0    conda-forge
m2-libunistring           1.1.2                hd8ed1ab_0    conda-forge
m2-libxcrypt              4.4.36.1             hd8ed1ab_0    conda-forge
m2-libxml2                2.9.14.5             hd8ed1ab_0    conda-forge
m2-libxslt                1.1.38.1             hd8ed1ab_0    conda-forge
m2-libzstd                1.5.5.1              hd8ed1ab_0    conda-forge
m2-msys2-runtime          3.4.9.1              hd8ed1ab_0    conda-forge
m2-nano                   7.2.1                hd8ed1ab_0    conda-forge
m2-ncurses                6.4.1                hd8ed1ab_0    conda-forge
m2-openssh                9.4p1.1              hd8ed1ab_0    conda-forge
m2-openssl                3.1.2.1              hd8ed1ab_0    conda-forge
m2-p11-kit                0.25.0.1             hd8ed1ab_0    conda-forge
m2-patch                  2.7.6.2              hd8ed1ab_0    conda-forge
m2-perl                   5.36.1.2             hd8ed1ab_0    conda-forge
m2-perl-authen-sasl       2.16.3               hd8ed1ab_0    conda-forge
m2-perl-clone             0.46.1               hd8ed1ab_0    conda-forge
m2-perl-convert-binhex    1.125.2              hd8ed1ab_0    conda-forge
m2-perl-encode-locale     1.05.2               hd8ed1ab_0    conda-forge
m2-perl-error             0.17029.1            hd8ed1ab_0    conda-forge
m2-perl-file-listing      6.16.1               hd8ed1ab_0    conda-forge
m2-perl-html-parser       3.81.1               hd8ed1ab_0    conda-forge
m2-perl-html-tagset       3.20.3               hd8ed1ab_0    conda-forge
m2-perl-http-cookies      6.10.2               hd8ed1ab_0    conda-forge
m2-perl-http-daemon       6.16.1               hd8ed1ab_0    conda-forge
m2-perl-http-date         6.06.1               hd8ed1ab_0    conda-forge
m2-perl-http-message      6.44.1               hd8ed1ab_0    conda-forge
m2-perl-http-negotiate    6.01.3               hd8ed1ab_0    conda-forge
m2-perl-io-html           1.004.2              hd8ed1ab_0    conda-forge
m2-perl-io-socket-ip      0.41.2               hd8ed1ab_0    conda-forge
m2-perl-io-socket-ssl     2.083.1              hd8ed1ab_0    conda-forge
m2-perl-io-stringy        2.113.1              hd8ed1ab_0    conda-forge
m2-perl-libwww            6.67.1               hd8ed1ab_0    conda-forge
m2-perl-lwp-mediatypes    6.04.1               hd8ed1ab_0    conda-forge
m2-perl-mailtools         2.21.1               hd8ed1ab_0    conda-forge
m2-perl-mime-tools        5.510.1              hd8ed1ab_0    conda-forge
m2-perl-net-http          6.23.1               hd8ed1ab_0    conda-forge
m2-perl-net-smtp-ssl      1.04.2               hd8ed1ab_0    conda-forge
m2-perl-net-ssleay        1.92.5               hd8ed1ab_0    conda-forge
m2-perl-termreadkey       2.38.5               hd8ed1ab_0    conda-forge
m2-perl-timedate          2.33.2               hd8ed1ab_0    conda-forge
m2-perl-try-tiny          0.31.1               hd8ed1ab_0    conda-forge
m2-perl-uri               5.21.1               hd8ed1ab_0    conda-forge
m2-perl-www-robotrules    6.02.2               hd8ed1ab_0    conda-forge
m2-sed                    4.9.1                hd8ed1ab_0    conda-forge
m2-zlib                   1.3.1                hd8ed1ab_0    conda-forge
mamba                     1.5.8           py310hd9d798f_0    conda-forge
markupsafe                2.1.5           py310h8d17308_0    conda-forge
menuinst                  2.0.2           py310h00ffb61_0    conda-forge
miniforge_console_shortcut 1.0                  h57928b3_0    conda-forge
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.0                ha2aaf27_2    conda-forge
openssl                   3.3.0                hcfcfb64_0    conda-forge
oras-py                   0.1.14             pyhd8ed1ab_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pillow                    9.5.0           py310h3dcae36_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkginfo                   1.10.0             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.4.0              pyhd8ed1ab_0    conda-forge
psutil                    5.9.8           py310h8d17308_0    conda-forge
py-lief                   0.14.1          py310h9e98ed7_1    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py310h8d17308_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
pytest                    8.1.2              pyhd8ed1ab_0    conda-forge
python                    3.10.14         h4de0772_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py310h5588dad_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywin32                   306             py310h00ffb61_2    conda-forge
pyyaml                    6.0.1           py310h8d17308_1    conda-forge
rattler-build             0.16.0               ha08ef0e_0    conda-forge
rattler-build-conda-compat 0.0.6              pyh267e887_0    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hcfcfb64_1    conda-forge
reproc-cpp                14.2.4.post0         h63175ca_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
ripgrep                   11.0.2                        1    conda-forge
rpds-py                   0.18.1          py310hc226416_0    conda-forge
ruamel.yaml               0.18.6          py310h8d17308_0    conda-forge
ruamel.yaml.clib          0.2.8           py310h8d17308_0    conda-forge
ruamel_yaml               0.15.80         py310h8d17308_1009    conda-forge
setuptools                69.5.1             pyhd8ed1ab_0    conda-forge
shyaml                    0.6.2              pyhd3deb0d_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.4             py310h8d17308_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.11.0               hd8ed1ab_0    conda-forge
typing_extensions         4.11.0             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
vc                        14.3                hcf57466_18    conda-forge
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
wrapt                     1.16.0          py310h8d17308_0    conda-forge
xorg-libxau               1.0.8                         2    conda-forge
xorg-libxdmcp             1.1.2                         2    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yaml-cpp                  0.8.0                h63175ca_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstandard                 0.22.0          py310h0009e47_0    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge

Additional Context

Here is the PR used for testing: conda-forge/cuda-nvtx-feedstock#16

Here is the state of the feedstock during the build: https://github.com/conda-forge/cuda-nvtx-feedstock/tree/f4f4da08ab69ed665ddbf4d062088a9fd8af0d7b

Should be able to reproduce by running: conda-build -m .ci_support\win_64_.yaml recipe
@jakirkham jakirkham added the type::bug describes erroneous operation, use severity::* to classify the type label May 14, 2024
This was referenced May 14, 2024
Closed
Merged
@jakirkham
Copy link
Member Author

cc @kenodegard

@jakirkham jakirkham mentioned this issue May 14, 2024
Merged
5 tasks
@kenodegard kenodegard self-assigned this May 14, 2024
@kenodegard kenodegard added severity::1 blocker; broken functionality with no workaround source::community catch-all for issues filed by community members labels May 14, 2024
@kenodegard
Copy link
Contributor

@jakirkham thanks for reporting, taking a look

@jakirkham jakirkham mentioned this issue May 15, 2024
Closed
kenodegard added a commit that referenced this issue May 18, 2024
@kenodegard
Minimal OS agnostic reproducer for gh-5342
60bc111
@kenodegard kenodegard mentioned this issue May 18, 2024
Merged
3 tasks
@kenodegard kenodegard added this to the 24.5.x milestone May 19, 2024
@kenodegard kenodegard added the in-progress issue is actively being worked on label May 19, 2024
@kenodegard kenodegard mentioned this issue May 19, 2024
Closed
31 tasks
@beeankha
Copy link
Contributor

Resolved by merge of #5345

@beeankha beeankha mentioned this issue May 24, 2024
Merged
@jakirkham jakirkham mentioned this issue May 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kenodegard kenodegard

Labels
in-progress issue is actively being worked on severity::1 blocker; broken functionality with no workaround source::community catch-all for issues filed by community members type::bug describes erroneous operation, use severity::* to classify the type
Projects
🧭 Planning
Archived in project
Milestone
24.5.x
Development

Successfully merging a pull request may close this issue.

Fix frozendict usage conda/conda-build
3 participants
@jakirkham @kenodegard @beeankha