Compatibility with 3D inputs and improved documentation

This release extends TorchCAM compatibility to 3D inputs, and improves documentation.

Note: TorchCAM 0.2.0 requires PyTorch 1.5.1 or higher.

Highlights

Compatibility for inputs with more than 2 spatial dimensions

The first papers about CAM methods were built for classification models using 2D (spatially) inputs. However, the latest methods can be extrapolated to higher dimension inputs and it's now live:

import torch
from torchcam.cams import SmoothGradCAMpp
# Define your model compatible with 3D inputs
video_model = ...
extractor = SmoothGradCAMpp(video_model)
# Forward your input
scores = model(torch.rand((1, 3, 32, 224, 224)))
# Retrieve the CAM
cam = extractor(scores[0].argmax().item(), scores)

Multi-version documentation

While documentation was up-to-date with the latest commit on the main branch, previously if you were running an older release of the library, you had no corresponding documentation.

As of now, you can select the version of the documentation you wish to access (stable releases or latest commit):

Demo app

Since spatial information is at the very core of TorchCAM, a minimal Streamlit demo app was added to explore the activation of your favorite models. You can run the demo with the following commands:

streamlit run demo/app.py

Here is how it renders retrieving the heatmap using SmoothGradCAMpp on a pretrained resnet18:

New features

CAMs

Implementations of CAM method

• Enabled CAM compatibility for inputs with more than 2 spatial dimensions #45 (@frgfm)
• Added support of XGradCAM #47 (@frgfm)

Test

Verifications of the package well-being before release

• Added unittests for XGradCAM #47 (@frgfm)

Documentation

Online resources for potential users

• Added references to XGradCAM in README and documentation #47 (@frgfm)
• Added multi-version documentation & added github star button #53, #54, #55, #56 (@frgfm)
• Revamped README #59 (@frgfm) focusing on short easy code snippets
• Improved documentation #60 (@frgfm)

Others

Other tools and implementations

• Added issue templates for bug report and feature request #49 (@frgfm)
• Added option to specify a single CAM method in example script #52 (@frgfm)
• Added minimal demo app #59 (@frgfm)

Bug fixes

CAMs

• Fixed automatic layer resolution on GPU #41 (@frgfm)
• Fixed backward hook warnings for Pytorch >= 1.8.0 #58 (@frgfm)

Utils

• Fixed RGBA -> RGB conversion in overlay_mask #38 (@alexandrosstergiou)

Test

• Fixed overlay_mask unittest #38 (@alexandrosstergiou)

Documentation

• Fixed codacy badge in README #46 (@frgfm)
• Fixed typo in documentation #62 (@frgfm)

Others

• Fixed CI job for conda build #34 (@frgfm)
• Fixed model mode in example script #37 (@frgfm)
• Fixed sphinx version #40 (@frgfm)
• Fixed usage instructions in README #43 (@frgfm)
• Fixed example script for local image input #51 (@frgfm)

Improvements

CAMs

• Added NaN check in gradcams #37 (@frgfm)

Test

• Added NaN check unittest for gradcam #37 (@frgfm)
• Switched from unittest to pytest #45 (@frgfm) and split test files by module

Documentation

• Updated README badges #34, illustration #39 and usage instructions #41 (@frgfm)
• Added instructions to run all CI checks locally in CONTRIBUTING #34, #45 (@frgfm)
• Updated project hierarchy description in CONTRIBUTING #43 (@frgfm)
• Added minimal code snippet in documentation #41 (@frgfm)

Others

• Updated version in setup #34 and requirements #61 (@frgfm)
• Leveraged automatic layer resolution in example script #41 (@frgfm)
• Updated CI job to run unittests #45 (@frgfm)