Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace pdm dependency solver with uv #1891

Merged
merged 5 commits into from
Nov 18, 2024
Merged

Conversation

spwoodcock
Copy link
Member

@spwoodcock spwoodcock commented Nov 17, 2024

What type of PR is this? (check all applicable)

  • πŸ• Feature
  • πŸ› Bug Fix
  • πŸ“ Documentation
  • πŸ§‘β€πŸ’» Refactor
  • βœ… Test
  • πŸ€– Build or CI
  • ❓ Other (please specify)

Describe this PR

uv:

  • uv was released not too long ago and has since almost gained feature parity with other tools like pdm.
  • It's written in rust and is orders of magnitude faster at locking dependencies: a minor win, but a nice one nonetheless.
  • It also allows you to run different versions of Python on your system very easily.
  • Thanks @Pradip-p for pointing this tool out!

What I did:

  • Replaced all usage of pdm with uv.
  • Overhauled the backend dockerfile to work with uv and be a bit cleaner.
  • Updated docs around using pdm.

Minor caveats:

  • We can no longer use the dynamic field in pyproject.toml: dynamic=["version"].
  • This isn't a big deal - we can just hardcode it. The version updating is done by commitizen anyway, so nothing lost.

Bonus:

  • There is also a pre-commit hook for uv, allowing us to re-lock automatically if someone updates the a backend dependency and forgets to lock πŸŽ‰ (which would break the docker build otherwise).

Key things of note:

  • uv lock re-locks dependencies, including all optional groups.
  • uv sync installs dependencies locally.
  • uv run ... runs a command using installed dependencies venv.
  • It's also possible to manage multiple python versions on the system with uv python list and uv python install.
  • pdm publish --> uv publish for publishing to pypi.

Note: I plan to talk about using this tool with the dev teams when I visit NAXA πŸ‘

Screenshots

A fresh lock takes 1 second:

image

Checklist before requesting a review

[optional] What gif best describes this PR or how it makes you feel?

@spwoodcock spwoodcock added backend Related to backend code devops Related to deployment or configuration labels Nov 17, 2024
@spwoodcock spwoodcock added this to the Backend Dependency Updates milestone Nov 17, 2024
@spwoodcock spwoodcock self-assigned this Nov 17, 2024
@github-actions github-actions bot added docs Improvements or additions to documentation contrib labels Nov 17, 2024
@Pradip-p
Copy link

When I heard about UV, it always triggered one question: what exactly makes it faster than other tools like pip, PDM, or Poetry?

it seems UV uses heavy parallelization, advanced caching, and an optimized way to access metadata from PYPI. Instead of downloading entire package wheels, it queries the zip file’s Central Directory and fetches only the needed metadata using file offsets, avoiding unnecessary downloads and speeding up dependency resolution.

Am I understanding this correctly? Let me know your thoughts! @spwoodcock @nrjadkry @Sujanadh

@spwoodcock
Copy link
Member Author

You are definitely right when I comes the install part!

When it comes to speeding up the locking, the key factor here is that this is a library written in rust, a much lower level programming language with better parallelization than python (as you mention).

They most likely use pyo3 to generate Python bindings for the underlying rust library, meaning we get to install and use it from Python πŸ˜„

Same concept for ruff, one of astral's other tools

@spwoodcock spwoodcock merged commit d85abcf into development Nov 18, 2024
5 checks passed
@spwoodcock spwoodcock deleted the build/pdm-to-uv branch November 18, 2024 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to backend code contrib devops Related to deployment or configuration docs Improvements or additions to documentation
Projects
Development

Successfully merging this pull request may close these issues.

2 participants