v0.4.5

Installation

pip install openllm==0.4.5

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.4.5

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.4.5 start opt

To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.4.5

Find more information about this release in the CHANGELOG.md

What's Changed

• refactor(cli): move out to its own packages by @aarnphm in #619
• fix(cli): correct set working_dir by @aarnphm in #620
• chore(cli): always show available models by @aarnphm in #621
• fix(sdk): make sure build to quiet out stdout by @aarnphm in #622
• chore: update jupyter notebooks with new API by @aarnphm in #623
• fix(ruff): correct consistency between isort and formatter by @aarnphm in #624
• feat(vllm): support passing specific dtype by @aarnphm in #626
• chore(deps): bump taiki-e/install-action from 2.21.8 to 2.21.11 by @dependabot in #625
• feat(cli): --dtype arguments by @aarnphm in #627
• fix(cli): make sure to pass the dtype to subprocess service by @aarnphm in #628
• ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #629
• infra: removing clojure frontend from infra cycle by @aarnphm in #630
• fix(torch_dtype): load eagerly by @aarnphm in #631

Full Changelog: v0.4.4...v0.4.5