Skip to content

TorchServe v0.5.0 Release Notes
Compare
Choose a tag to compare
@lxning lxning released this 18 Nov 19:18
· 1608 commits to master since this release
v0.5.0
532ae41

This is the release of TorchServe v0.5.0.

New Features

  • PyTorch 1.10.0 support - TorchServe is now certified working with torch 1.10.0 torchvision 0.11.1, torchtext 0.11.0 and torchaudio 0.10.0
  • Kubernetes HPA support - Added support for Kubernetes HPA.
  • Faster transformer example - Added example for Faster transformer for optimized transformer model inference.
  • (experimental) torchprep support - Added experimental CLI tool to prepare Pytorch models for efficient inference.
  • Custom metrics example - Added example for custom metrics with mtail metrics exporter and Prometheus.
  • Reactjs example for Image Classifier - Added example for Reactjs Image Classifier.

Improvements

  • Batching inference exception support - Optimized batching to fix a concurrent modification exception that was occurring with batch inference.
  • k8s cluster creation support upgrade - Updated Kubernetes cluster creation scripts for v1.17 support.
  • Nvidia devices visibility support - Added support for NVIDIA devices visibility.
  • Large image support - Added support for PIL.Image.MAX_IMAGE_PIXELS.
  • Custom HTTP status support - Added support to return custom http status from a model handler.
  • TS_CONFIG_FILE env var support - Added support for setting TS_CONFIG_FILE as env var.
  • Frontend build optimization - Optimized frontend to reduce build times by 3.7x.
  • Warmup in benchmark - Added support for warmup in benchmark scripts.

Platform Support

Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows 10 Pro, Windows Server 2019, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4)

GPU Support

Torch 1.10+ Cuda 10.2, 11.3
Torch 1.9.0 + Cuda 11.1
Torch 1.8.1 + Cuda 9.2

Assets 17
Loading