Can't install pypi-dependencies into path with spaces.

[jacobj45]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pixi init --import environment.yml

(Check below for environment.yml)

Issue description

I am trying to create a new Pixi environment with Sentry dependencies installed. The dependencies are mentioned in this requirements file. I have also created the below environment YAML file with the same dependencies for convenience:

environment.yml

name: sentry-env
channels:
- conda-forge
dependencies:
- python>=3.11
- pip
- pip:
  - beautifulsoup4>=4.7.1
  - boto3>=1.34.8
  - botocore>=1.34.8
  - cachetools>=5
  - celery>=5
  - click>=8.1
  - confluent-kafka>=2.1.1
  - croniter>=1.3.10
  - cssselect>=1.0.3
  - datadog>=0.44
  - django-crispy-forms>=1.14.0
  - django-csp>=3.7
  - django-pg-zero-downtime-migrations>=0.13
  - Django>=5.0.2,<6
  - djangorestframework>=3.14.0
  - drf-spectacular>=0.26.3
  - email-reply-parser>=0.5.12
  - google-api-core>=2.15.0
  - google-auth>=2.25.2
  - google-cloud-bigtable>=2.22.0
  - google-cloud-build>=3.22.0
  - google-cloud-core>=2.4.1
  - google-cloud-functions>=1.14.0
  - google-cloud-kms>=2.20.0
  - google-cloud-pubsub>=2.19.0
  - google-cloud-spanner>=3.40.1
  - google-cloud-storage>=2.14.0
  - googleapis-common-protos>=1.62.0
  - google-crc32c>=1.5.0
  - isodate>=0.6.1
  - jsonschema>=3.2.0
  - lxml>=4.9.3
  - maxminddb>=2.3
  - mistune>=2.0.3
  - mmh3>=4.0.0
  - packaging>=21.3
  - parsimonious>=0.10.0
  - petname>=2.6
  - phonenumberslite>=8.12.32
  - Pillow>=10.2.0
  - progressbar2>=3.41.0
  - python-rapidjson>=1.4
  - psycopg2-binary>=2.9.9
  - PyJWT>=2.4.0
  - pydantic>=1.10.9
  - python-dateutil>=2.8.2
  - pymemcache
  - python-u2flib-server>=5.0.0
  - fido2>=0.9.2
  - python3-saml>=1.15.0
  - PyYAML>=6.0.1
  - rb>=1.9.0
  - redis-py-cluster>=2.1.0
  - redis>=3.4.1
  - requests-oauthlib>=1.2.0
  - requests>=2.25.1
  - rfc3339-validator>=0.1.2
  - rfc3986-validator>=0.1.1
  - sentry-arroyo>=2.16.0
  - sentry-kafka-schemas>=0.1.50
  - sentry-ophio==0.1.5
  - sentry-redis-tools>=0.1.7
  - sentry-relay>=0.8.45
  - sentry-sdk>=1.39.2
  - snuba-sdk>=2.0.25
  - simplejson>=3.17.6
  - sqlparse>=0.4.4
  - statsd>=3.3
  - structlog>=22
  - symbolic==12.8.0
  - toronado>=0.1.0
  - typing-extensions>=4.9.0
  - ua-parser>=0.10.0
  - unidiff>=0.7.4
  - urllib3[brotli]>=2
  - brotli>=1.0.9
  - pyuwsgi==2.0.23
  - zstandard>=0.18.0
  - sentry-usage-accountant==0.0.10
  - msgpack>=1.0.7
  - cryptography>=38.0.3
  - grpcio>=1.59.0
  - hiredis>=0.3.1
  - phabricator>=0.7.0
  - openai==1.3.5
  - setuptools

However, trying to run pixi init --import throws the following error:

  × Failed to download distributions
  ├─▶ Failed to fetch wheel: python-rapidjson==1.8
  ├─▶ Failed to build: `python-rapidjson==1.8`
  ╰─▶ Build backend failed to build wheel through `build_wheel()` with exit status: 1
      --- stdout:
      running bdist_wheel
      running build
      running build_ext
      building 'rapidjson' extension
      gcc -pthread -B /home/myuser/Documents/dummy3/.pixi/envs/default/compiler_compat -fno-strict-overflow
      -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/myuser/Documents/dummy3/.pixi/envs/default/include -fPIC -O2
      -isystem /home/myuser/Documents/dummy3/.pixi/envs/default/include -fPIC -DPYTHON_RAPIDJSON_VERSION=1.8
      -DRAPIDJSON_EXACT_VERSION=v1.1.0-706-g232389d4 -I./rapidjson/include -I/home/myuser/.cache/rattler/cache/uv-cache/.tmpypJjLN/.venv/include "-I/home/myuser/Documents/
      dummy3/.pixi/envs/default/include/python3.12" -c ./rapidjson.cpp -o build/temp.linux-x86_64-cpython-312/./
      rapidjson.o -pedantic -Wno-long-long -std=c++11
      --- stderr:
      gcc: fatal error: cannot specify ‘-o’ with ‘-c’, ‘-S’ or ‘-E’ with multiple files
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      ---

Expected behavior

A new Pixi environment with all the dependencies installed.

[ruben-arts]

Hey @jacobj45,

Because some of your dependencies are not build for "older" glibc versions pixi will try to build the pypi packages from scratch. Adding a system requirement on an older version of libc will allow pixi to download more resent versions.

Add this to the generated pixi.toml

[system-requirements]
libc = "2.32" # verify your `__glibc` version by running `pixi info`

This will result in a much faster solve and no need for any compiler to run.

The issue related to improving this workflow is #346

[jacobj45]

@ruben-arts (Sorry for the late reply 😅)

I have updated my pixi.toml as per your suggestion...

pixi.toml

[project]
name = "sentry-env"
version = "0.1.0"
description = "Add a short description here"
authors = ["J <[email protected]>"]
channels = ["conda-forge"]
platforms = ["linux-64"]

[tasks]

[dependencies]

[target.linux-64.dependencies]
python = ">=3.11"
pip = "*"

[system-requirements]
libc = "2.32"

... but pixi info shows that the version of glibc is unchanged ...

output

      Pixi version: 0.23.0
          Platform: linux-64
  Virtual packages: __unix=0=0
                  : __linux=6.0.6=0
                  : __glibc=2.35=0
                  : __archspec=1=icelake
         Cache dir: /home/myuser/.cache/rattler/cache
      Auth storage: /home/myuser/.rattler/credentials.json

Project
------------
              Name: sentry-env
           Version: 0.1.0
     Manifest file: /home/myuser/Documents/dummy3/pixi.toml
  Config locations: 
      Last updated: 28-05-2024 22:34:16

Environments
------------
       Environment: default
          Features: default
          Channels: conda-forge
  Dependency count: 2
      Dependencies: python, pip
  Target platforms: linux-64

... and I get the same error as before while adding packages.

---

I also tried adding the package as per the docs:

[system-requirements]
libc = { family="glibc", version="2.32" }

That didn't work either.

[ruben-arts]

This pixi.toml installs correctly for me.

But afterwards you need to have a compiler installed.

It is hard to test for me locally as my base system has some comilers. But I believe you can add:

pixi add compilers
pixi add --pypi python-rapidjson==1.8

[jacobj45]

@ruben-arts Unfortunately, that didn't work either.

But I was able to find out the root cause of the issue. It was due to the presence of space characters in the directory path. For example, I can't successfully build a Pixi env from ~/Documents/gh - issue/.

I think this is a Pixi issue because I am able to install all dependencies when using pip install

[jacobj45]

I am also curious why setting system-requirements and adding the compilers dependency didn't change glibc version. You mentioned that it would result in a faster solve, which I am very much interested in achieving.

[baszalmstra]

The virtual packages reported by pixi info are based on your system and not on the system requirements of your project.

[ruben-arts]

But I was able to find out the root cause of the issue. It was due to the presence of space characters in the directory path. For example, I can't successfully build a Pixi env from ~/Documents/gh - issue/.

Glad you found that, that indeed seems like something we should fix.

[tdejager]

Would be nice to create a reproducer, as for me adding spaces to a directory does not make a difference on osx. :)

[jacobj45]

@tdejager

Simplest sequence of steps that I can think of in order to reproduce the issue:

1. Create directory named Sample Dir (Note the space character)
2. Run pixi init from within the directory
3. Run pixi add python
4. Run pixi add --pypi python-rapidjson==1.8 to reproduce the error
5. Run the above steps in another directory named SampleDir to see expected results

I am on Linux Pop!OS and using Python 3.12

[twrightsman]

This seems to be a more general issue (like conda/conda#186) with spaces in the path to the environment. I can reproduce this with just [dependencies] and a package that installs an executable with a shebang line.

$ mkdir 'sub directory'
$ cd 'sub directory'
$ pixi init
$ vim pixi.toml  # see below
$ pixi run marimo --version
Error launching 'marimo': No such file or directory (os error 2)
$ pixi shell
(myEnv) $ marimo --version
bash: /home/twrightsman/sub directory/.pixi/envs/default/bin/marimo: cannot execute: required file not found
$ head -n1 '/home/twrightsman/sub directory/.pixi/envs/default/bin/marimo'
#!/home/twrightsman/sub directory/.pixi/envs/default/bin/python

Perhaps pixi should issue a warning (if not prevent) when creating environments in a path with spaces?

pixi.toml

[project]
name = "myEnv"
version = "0.1.0"
channels = ["conda-forge"]
platforms = ["linux-64"]

[dependencies]
marimo = "~=0.6.19"