TorchServe v0.5.3 Release Notes
This is the release of TorchServe v0.5.3.
New Features
- KServe V2 support - Added support for KServe V2 protocol.
- Model customized metadata support - Extended managementAPI to support customized metadata from handler.
Improvements
- Upgraded log4j2 version to 2.17.1 - Added log4j upgrade to address CVE-2021-44832.
- Upgraded pillow to 9.0.0, python support upgraded to py3.8/py3.9 - Added docker, install dependency upgrade.
- GPU utilization and GPU memory usage metrics support - Added support for GPU utilization and GPU memory usage metrics in benchmarks.
- Workflow benchmark support - Added support for workflow benchmark.
- benchmark-ab.py warmup support - Added support for warmup in benchmark-ab.py.
- Multiple inputs for a model inference example - Added example to support multiple inputs for a model inference.
- Documentation refactor - Improved documention.
- Added API auto-discovery - Added support for API auto-discovery.
- Nightly build support - Added support for Github action nightly build
pip install torchserve-nightly
Platform Support
Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows 10 Pro, Windows Server 2019, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4). TorchServe now requires Python 3.8 and above.
GPU Support
Torch 1.10+ Cuda 10.2, 11.3
Torch 1.9.0 + Cuda 11.1
Torch 1.8.1 + Cuda 9.2