docs: Added detailed explanation on CAM computation (#8)

* docs: Added installation instructions

* docs: Fixed typo

* docs: Added detailed explanations for each CAM method

* docs: Clarified explanations of Score-CAM

docs/source/cams.rst

@@ -7,7 +7,7 @@ torchcam.cams
 
 CAM
 --------
-Related to activation-based class activation maps.
+Methods related to activation-based class activation maps.
 
 
 .. autoclass:: CAM
@@ -17,12 +17,11 @@ Related to activation-based class activation maps.
 
 Grad-CAM
 --------
-Related to gradient-based class activation maps.
+Methods related to gradient-based class activation maps.
 
 
 .. autoclass:: GradCAM
 
-
 .. autoclass:: GradCAMpp
 
 .. autoclass:: SmoothGradCAMpp

docs/source/index.rst

@@ -3,9 +3,15 @@ Torchcam documentation
 
 The :mod:`torchcam` package gives PyTorch users the possibility to visualize the spatial influence on classification outputs.
 
+.. toctree::
+ :maxdepth: 2
+ :caption: Getting Started
+
+ installing
+
 .. toctree::
  :maxdepth: 1
- :caption: Package Reference
+ :caption: Package Documentation
 
  cams
  utils

docs/source/installing.rst

@@ -0,0 +1,36 @@
+
+************
+Installation
+************
+
+This library requires Python 3.6 or newer.
+
+Via Python Package
+==================
+
+Install the last stable release of the package using pip:
+
+.. code:: bash
+
+ pip install torchcam
+
+
+Via Conda
+=========
+
+Install the last stable release of the package using conda:
+
+.. code:: bash
+
+ conda install -c frgfm torchcam
+
+
+Via Git
+=======
+
+Install the library in developper mode:
+
+.. code:: bash
+
+ git clone https://github.com/frgfm/torch-cam.git
+ pip install -e torch-cam/.

torchcam/cams/cam.py

@@ -117,7 +117,18 @@ def __repr__(self):
 
 
 class CAM(_CAM):
- """Implements a class activation map extractor as described in https://arxiv.org/abs/1512.04150
+ """Implements a class activation map extractor as described in `"Learning Deep Features for Discriminative
+ Localization" <https://arxiv.org/pdf/1512.04150.pdf>`_.
+
+ The Class Activation Map (CAM) is defined for image classification models that have global pooling at the end
+ of the visual feature extraction block. The localization map is computed as follows:
+
+ .. math::
+ L^{(c)}_{CAM}(x, y) = ReLU\\Big(\\sum\\limits_k w_k^{(c)} A_k(x, y)\\Big)
+
+ where :math:`A_k(x, y)` is the activation of node :math:`k` in the last convolutional layer of the model at
+ position :math:`(x, y)`,
+ and :math:`w_k^{(c)}` is the weight corresponding to class :math:`c` for unit :math:`k`.
 
  Example::
  >>> from torchvision.models import resnet18
@@ -150,7 +161,29 @@ def _get_weights(self, class_idx, scores=None):
 
 
 class ScoreCAM(_CAM):
- """Implements a class activation map extractor as described in https://arxiv.org/abs/1910.01279
+ """Implements a class activation map extractor as described in `"Score-CAM:
+ Score-Weighted Visual Explanations for Convolutional Neural Networks" <https://arxiv.org/pdf/1910.01279.pdf>`_.
+
+ The localization map is computed as follows:
+
+ .. math::
+ L^{(c)}_{Score-CAM}(x, y) = ReLU\\Big(\\sum\\limits_k w_k^{(c)} A_k(x, y)\\Big)
+
+ with the coefficient :math:`w_k^{(c)}` being defined as:
+
+ .. math::
+ w_k^{(c)} = softmax(Y^{(c)}(M) - Y^{(c)}(X_b))
+
+ where :math:`A_k(x, y)` is the activation of node :math:`k` in the last convolutional layer of the model at
+ position :math:`(x, y)`, :math:`Y^{(c)}(X)` is the model output score for class :math:`c` before softmax
+ for input :math:`X`, :math:`X_b` is a baseline image,
+ and :math:`M` is defined as follows:
+
+ .. math::
+ M = \\Big(\\frac{M^{(d)} - \\min M^{(d)}}{\\max M^{(d)} - \\min M^{(d)}} \\odot X \\Big)_{1 \\leq d \\leq D}
+
+ where :math:`\\odot` refers to the element-wise multiplication, :math:`M^{(d)}` is the upsampled version of
+ :math:`A_d` on node :math:`d`, and :math:`D` is the number of channels on the target convolutional layer.
 
  Example::
  >>> from torchvision.models import resnet18

torchcam/cams/gradcam.py

@@ -55,7 +55,23 @@ def _get_weights(self, class_idx, scores):
 
 
 class GradCAM(_GradCAM):
- """Implements a class activation map extractor as described in https://arxiv.org/pdf/1710.11063.pdf
+ """Implements a class activation map extractor as described in `"Grad-CAM: Visual Explanations from Deep Networks
+ via Gradient-based Localization" <https://arxiv.org/pdf/1610.02391.pdf>`_.
+
+ The localization map is computed as follows:
+
+ .. math::
+ L^{(c)}_{Grad-CAM}(x, y) = ReLU\\Big(\\sum\\limits_k w_k^{(c)} A_k(x, y)\\Big)
+
+ with the coefficient :math:`w_k^{(c)}` being defined as:
+
+ .. math::
+ w_k^{(c)} = \\frac{1}{H \\cdot W} \\sum\\limits_{i=1}^H \\sum\\limits_{j=1}^W
+ \\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)}
+
+ where :math:`A_k(x, y)` is the activation of node :math:`k` in the last convolutional layer of the model at
+ position :math:`(x, y)`,
+ and :math:`Y^{(c)}` is the model output score for class :math:`c` before softmax.
 
  Example::
  >>> from torchvision.models import resnet18
@@ -86,7 +102,32 @@ def _get_weights(self, class_idx, scores):
 
 
 class GradCAMpp(_GradCAM):
- """Implements a class activation map extractor as described in https://arxiv.org/pdf/1710.11063.pdf
+ """Implements a class activation map extractor as described in `"Grad-CAM++: Improved Visual Explanations for
+ Deep Convolutional Networks" <https://arxiv.org/pdf/1710.11063.pdf>`_.
+
+ The localization map is computed as follows:
+
+ .. math::
+ L^{(c)}_{Grad-CAM++}(x, y) = \\sum\\limits_k w_k^{(c)} A_k(x, y)
+
+ with the coefficient :math:`w_k^{(c)}` being defined as:
+
+ .. math::
+ w_k^{(c)} = \\sum\\limits_{i=1}^H \\sum\\limits_{j=1}^W \\alpha_k^{(c)}(i, j) \\cdot
+ ReLU\\Big(\\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)}\\Big)
+
+ where :math:`A_k(x, y)` is the activation of node :math:`k` in the last convolutional layer of the model at
+ position :math:`(x, y)`,
+ :math:`Y^{(c)}` is the model output score for class :math:`c` before softmax,
+ and :math:`\\alpha_k^{(c)}(i, j)` being defined as:
+
+ .. math::
+ \\alpha_k^{(c)}(i, j) = \\frac{1}{\\sum\\limits_{i, j} \\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)}}
+ = \\frac{\\frac{\\partial^2 Y^{(c)}}{(\\partial A_k(i,j))^2}}{2 \\cdot
+ \\frac{\\partial^2 Y^{(c)}}{(\\partial A_k(i,j))^2} + \\sum\\limits_{a,b} A_k (a,b) \\cdot
+ \\frac{\\partial^3 Y^{(c)}}{(\\partial A_k(i,j))^3}}
+
+ if :math:`\\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)} = 1` else :math:`0`.
 
  Example::
  >>> from torchvision.models import resnet18
@@ -122,8 +163,41 @@ def _get_weights(self, class_idx, scores):
 
 
 class SmoothGradCAMpp(_GradCAM):
- """Implements a class activation map extractor as described in https://arxiv.org/pdf/1908.01224.pdf
- with a personal correction to the paper (alpha coefficient numerator)
+ """Implements a class activation map extractor as described in `"Smooth Grad-CAM++: An Enhanced Inference Level
+ Visualization Technique for Deep Convolutional Neural Network Models" <https://arxiv.org/pdf/1908.01224.pdf>`_
+ with a personal correction to the paper (alpha coefficient numerator).
+
+ The localization map is computed as follows:
+
+ .. math::
+ L^{(c)}_{Smooth Grad-CAM++}(x, y) = \\sum\\limits_k w_k^{(c)} A_k(x, y)
+
+ with the coefficient :math:`w_k^{(c)}` being defined as:
+
+ .. math::
+ w_k^{(c)} = \\sum\\limits_{i=1}^H \\sum\\limits_{j=1}^W \\alpha_k^{(c)}(i, j) \\cdot
+ ReLU\\Big(\\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)}\\Big)
+
+ where :math:`A_k(x, y)` is the activation of node :math:`k` in the last convolutional layer of the model at
+ position :math:`(x, y)`,
+ :math:`Y^{(c)}` is the model output score for class :math:`c` before softmax,
+ and :math:`\\alpha_k^{(c)}(i, j)` being defined as:
+
+ .. math::
+ \\alpha_k^{(c)}(i, j)
+ = \\frac{\\frac{\\partial^2 Y^{(c)}}{(\\partial A_k(i,j))^2}}{2 \\cdot
+ \\frac{\\partial^2 Y^{(c)}}{(\\partial A_k(i,j))^2} + \\sum\\limits_{a,b} A_k (a,b) \\cdot
+ \\frac{\\partial^3 Y^{(c)}}{(\\partial A_k(i,j))^3}}
+ = \\frac{\\frac{1}{n} \\sum\\limits_{m=1}^n D^{(c, 2)}_k(i, j)}{
+ \\frac{2}{n} \\sum\\limits_{m=1}^n D^{(c, 2)}_k(i, j) + \\sum\\limits_{a,b} A_k (a,b) \\cdot
+ \\frac{1}{n} \\sum\\limits_{m=1}^n D^{(c, 3)}_k(i, j)}
+
+ if :math:`\\frac{\\partial Y^{(c)}}{\\partial A_k(i, j)} = 1` else :math:`0`. Here :math:`D^{(c, p)}_k(i, j)`
+ refers to the p-th partial derivative of the class score of class :math:`c` relatively to the activation in layer
+ :math:`k` at position :math:`(i, j)`, and :math:`n` is the number of samples used to get the gradient estimate.
+
+ Please note the difference in the numerator of :math:`\\alpha_k^{(c)}(i, j)`,
+ which is actually :math:`\\frac{1}{n} \\sum\\limits_{k=1}^n D^{(c, 1)}_k(i,j)` in the paper.
 
  Example::
  >>> from torchvision.models import resnet18