Skip to content

TorchServe v0.2.0 Release Notes (Beta)
Compare
Choose a tag to compare
@maaquib maaquib released this 11 Aug 21:08
v0.2.0
b89d1ca

This is the release of TorchServe v0.2.0

Highlights:

  • Kubernetes Support - Torchserve deployment in Kubernetes using Helm Charts and a Persistent Volume
  • Prometheus metrics - Added Prometheus as the default metrics framework
  • Requirements.txt support​ - Added support to specify model specific dependencies as a requirements file within a mar archive; Cleanup of unused parameters and addition of relevant ones for torch-model-archiver
  • Pytorch Scripted Models Support - Scripted model versions added to model zoo; Added testing for scripted models
  • Default Handler Refactor: (breaking changes) The default handlers have been refactored for code reuse and enhanced post-processing support. More details in Backwards Incompatible Changes section below
  • Windows Support - Added support for torchserve on windows subsystem for Linux
  • AWS Cloud Formation Support - Added support for multi-node AutoScaling Group deployment, behind an Elastic Load Balancer using Elastic File System as the backing store
  • Benchmark and Testing Enhancements - Added models in benchmark and sanity tests, support for throughput with batch processing in benchmarking, support docker for jmeter and apache benchmark tests
  • Regression Suite Enhancements - Added new POSTMAN based test cases for API and pytest based intrusive test cases
  • Docker Improvements - Consolidated dev and codebuild dockerfiles
  • Install and Build Script Streamlining - Unified install scripts, added code coverage and sanity script
  • Python Linting - More exhaustive python linting checks across Torchserve and Model Archiver

Backwards Incompatible Changes

  • Default Handler Refactor:
    • The default handlers have been refactored for code reuse and enhanced post-processing support. The output format for some of the following examples/models has been enhanced to include additional details like score/class probability.
    • The following default-handlers have been equipped with batch support. Due to batch support, resnet_152_batch example is not a custom handler example anymore.
      • image_classifier
      • object_detector
      • image_segmenter
    • The index_to_name.json file use for the class to name mapping has been standardized across vision/text related default handlers
    • Refactoring and code reuse have resulted into reduced boilerplate code in all the serve/examples.
    • Custom handler documentation has been restructured and enhanced to facilitate the different possible ways to build simple or complex custom handlers

Other PRs since v0.1.1

Bug Fixes:

  • Fixed NameError in default image_classifier handler #489
  • Fixed timeout errors during build #420 and unit tests #493
  • Fixed model loading error on cpu which was saved on gpu #444
  • Fixed Snapshot not being emitted after unregistering model with no workers #491
  • Inference API description conformant to OpenAPI #372
  • Removed duplicate snapshot server property #318
  • Fixed tag for latest CPU version in README #452
  • Added check for no objects detected in object detector #447
  • Fixed incorrect set up of default workers per model #513
  • Fixed model-archiver to accept handler name or handler_name:entry_pnt_func combinations #472

Others

  • Netty dependencies update #487
  • Updates to install documentation and contribution guidelines #527

Platform Support

Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4)

Getting Started with TorchServe

Additionally, you can get started at https://pytorch.org/serve/ with installation instructions, tutorials and docs.
Lastly, if you have questions, please drop it into the PyTorch discussion forums using the ‘deployment’ tag or file an issue on GitHub with a way to reproduce.

Assets 10
Loading