Docs improvements (#1403)

* quickstart + core * + * + * + * docs
catalyst-team · Feb 13, 2022 · f390239 · f390239
1 parent 7b41850
commit f390239
Show file tree

Hide file tree

Showing 57 changed files with 180 additions and 229 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -23,6 +23,44 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 -
 
 
+## [22.02] - 2022-02-13
+
+### Tl;dr
+- Catalyst architecture simplification.
+- [#1395](https://github.com/catalyst-team/catalyst/issues/1395), [#1396](https://github.com/catalyst-team/catalyst/issues/1396), [#1397](https://github.com/catalyst-team/catalyst/issues/1397), [#1398](https://github.com/catalyst-team/catalyst/issues/1398), [#1399](https://github.com/catalyst-team/catalyst/issues/1399), [#1400](https://github.com/catalyst-team/catalyst/issues/1400), [#1401](https://github.com/catalyst-team/catalyst/issues/1401), [#1402](https://github.com/catalyst-team/catalyst/issues/1402), [#1403](https://github.com/catalyst-team/catalyst/issues/1403).
+
+### Added
+
+- Additional tests for different hardware accelerators setups. Please check out the `tests/pipelines` folder for more information.
+- `BackwardCallback` and `BackwardCallbackOrder` as an abstraction on top of `loss.backward`. Now you could easily log model gradients or transform them before `OptimizerCallback`.
+- `CheckpointCallbackOrder` for `ICheckpointCallback`.
+
+### Changed
+
+- Minimal python version moved to `3.7`, minimal pytorch version moved to `1.4.0`.
+- Engines rewritten on top of Accelerate. First, we found these two abstractions very close to each other. Second, Accelerate provides additional user-friendly API and more stable API for "Nvidia APEX" and "Facebook Fairscale" - it does not support them.
+- SelfSupervisedRunner moved to the `examples` folder from the Catalyst API. The only Runners API, that will be supported in the future: `IRunner`, `Runner`, `ISupervisedRunner`, `SupervisedRunner` due to their consistency. If you are interested in any other Runner API - feel free to write your own `CustomRunner` and use `SelfSupervisedRunner` as an example.
+- `Runner.{global/stage}_{batch/loader/epoch}_metrics` renamed to `Runner.{batch/loader/epoch}_metrics`
+- `CheckpointCallback` rewritten from scratch.
+- Catalyst registry moved to full-imports-paths only.
+- Logger API changed to receive `IRunner` for all `log_*` methods.
+- Metric API: `topk_args` renamed to `topk`
+- Contrib API: init imports from `catalyst.contrib` - removed, use `from catalyst.contrib.{smth} import {smth}`. Could be change to full-imports-only in future versions for stability.
+- All quickstarts, minimal examples, notebooks and pipelines moved to new version.
+- Codestyle moved to `89` right margin. Honestly speaking, it's much easier to maintain Catalyst with `89` right margin on MBP'16.
+
+### Removed
+
+- `ITrial` removed.
+- Stages support removed. While we embrace stages in deep learning experiments, current hardware accelerators are not prepared well for such setups. Additionally, ~95% of dl pipelines are single-stage. Multi-stage runner support is under review. For multi-stage support, please define a `CustomRunner` with rewritten API.
+- Config/Hydra API support removed. Config API is under review. For now, you could write your own Config API with [hydra-slayer](https://github.com/catalyst-team/hydra-slayer) if needed.
+- `catalyst-dl` scripts removed. Without Config API we don't need them anymore.
+- `Nvidia Apex`, `Fairscale`, `Albumentations`, `Nifti`, `Hydra` requiremets removed.
+- `OnnxCallback`, `PruningCallback`, `QuantizationCallback`, `TracingCallback` removed from callbacks API. Theese callbacks are under review now.
+
+If you have any questions on the Catalyst 22 edition updates, please join Catalyst slack for discussion.
+
+
 ## [21.12] - 2021-12-28
 
 ### Added

diff --git a/README.md b/README.md
@@ -75,12 +75,9 @@ from catalyst.contrib.datasets import MNIST
 model = nn.Sequential(nn.Flatten(), nn.Linear(28 * 28, 10))
 criterion = nn.CrossEntropyLoss()
 optimizer = optim.Adam(model.parameters(), lr=0.02)
-
-train_data = MNIST(os.getcwd(), train=True)
-valid_data = MNIST(os.getcwd(), train=False)
 loaders = {
- "train": DataLoader(train_data, batch_size=32),
- "valid": DataLoader(valid_data, batch_size=32),
+ "train": DataLoader(MNIST(os.getcwd(), train=True), batch_size=32),
+ "valid": DataLoader(MNIST(os.getcwd(), train=False), batch_size=32),
 }
 
 runner = dl.SupervisedRunner(
@@ -110,21 +107,18 @@ metrics = runner.evaluate_loader(
  loader=loaders["valid"],
  callbacks=[dl.AccuracyCallback(input_key="logits", target_key="targets", topk=(1, 3, 5))],
 )
-assert "accuracy01" in metrics.keys()
 
 # model inference
 for prediction in runner.predict_loader(loader=loaders["valid"]):
  assert prediction["logits"].detach().cpu().numpy().shape[-1] == 10
 
-features_batch = next(iter(loaders["valid"]))[0]
-# model tracing
-utils.trace_model(model=runner.model.cpu(), batch=features_batch)
-# model quantization
-utils.quantize_model(model=runner.model)
-# model pruning
-utils.prune_model(model=runner.model, pruning_fn="l1_unstructured", amount=0.8)
-# onnx export
-utils.onnx_export(model=runner.model.cpu(), batch=features_batch, file="./logs/mnist.onnx", verbose=True)
+# model post-processing
+model = runner.model.cpu()
+batch = next(iter(loaders["valid"]))[0]
+utils.trace_model(model=model, batch=batch)
+utils.quantize_model(model=model)
+utils.prune_model(model=model, pruning_fn="l1_unstructured", amount=0.8)
+utils.onnx_export(model=model, batch=batch, file="./logs/mnist.onnx", verbose=True)
 ```
 
 ### Step-by-step Guide
@@ -188,6 +182,7 @@ Tested on Ubuntu 16.04/18.04/20.04, macOS 10.15, Windows 10, and Windows Subsyst
 
 ### Documentation
 - [master](https://catalyst-team.github.io/catalyst/)
+- [22.02](https://catalyst-team.github.io/catalyst/v22.02/index.html)
 
 - <details>
  <summary>2021 edition</summary>

diff --git a/catalyst/__version__.py b/catalyst/__version__.py
@@ -1 +1 @@
-__version__ = "22.02rc0"
+__version__ = "22.02"
diff --git a/catalyst/callbacks/backward.py b/catalyst/callbacks/backward.py
@@ -20,7 +20,7 @@ class BackwardCallback(IBackwardCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/criterion.py b/catalyst/callbacks/criterion.py
@@ -17,7 +17,7 @@ class CriterionCallback(FunctionalMetricCallback, ICriterionCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/accuracy.py b/catalyst/callbacks/metrics/accuracy.py
@@ -88,7 +88,7 @@ class AccuracyCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -180,7 +180,7 @@ class MultilabelAccuracyCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/auc.py b/catalyst/callbacks/metrics/auc.py
@@ -76,7 +76,7 @@ class AUCCallback(LoaderMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/classification.py b/catalyst/callbacks/metrics/classification.py
@@ -80,7 +80,7 @@ class PrecisionRecallF1SupportCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -187,7 +187,7 @@ class MultilabelPrecisionRecallF1SupportCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/cmc_score.py b/catalyst/callbacks/metrics/cmc_score.py
@@ -145,7 +145,7 @@ def handle_batch(self, batch) -> None:
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/confusion_matrix.py b/catalyst/callbacks/metrics/confusion_matrix.py
@@ -89,7 +89,7 @@ class ConfusionMatrixCallback(Callback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/r2_squared.py b/catalyst/callbacks/metrics/r2_squared.py
@@ -54,7 +54,7 @@ class R2SquaredCallback(LoaderMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/recsys.py b/catalyst/callbacks/metrics/recsys.py
@@ -96,7 +96,7 @@ class HitrateCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -207,7 +207,7 @@ class MAPCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -318,7 +318,7 @@ class MRRCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -429,7 +429,7 @@ class NDCGCallback(BatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/scikit_learn.py b/catalyst/callbacks/metrics/scikit_learn.py
@@ -104,7 +104,7 @@ class SklearnBatchCallback(FunctionalBatchMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -233,7 +233,7 @@ class SklearnLoaderCallback(LoaderMetricCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/metrics/segmentation.py b/catalyst/callbacks/metrics/segmentation.py
@@ -90,7 +90,7 @@ def handle_batch(self, batch):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -208,7 +208,7 @@ def handle_batch(self, batch):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -328,7 +328,7 @@ def handle_batch(self, batch):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/mixup.py b/catalyst/callbacks/mixup.py
@@ -107,6 +107,7 @@ def handle_batch(self, batch):
  input_key="clf_logits",
  target_key="clf_targets_one_hot"
  ),
+ "backward": dl.BackwardCallback(metric_key="loss"),
  "optimizer": dl.OptimizerCallback(metric_key="loss"),
  "classification": dl.ControlFlowCallback(
  dl.PrecisionRecallF1SupportCallback(

diff --git a/catalyst/callbacks/optimizer.py b/catalyst/callbacks/optimizer.py
@@ -26,7 +26,7 @@ class OptimizerCallback(IOptimizerCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/callbacks/scheduler.py b/catalyst/callbacks/scheduler.py
@@ -32,7 +32,7 @@ class SchedulerCallback(ISchedulerCallback):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/core/runner.py b/catalyst/core/runner.py
@@ -62,7 +62,7 @@ class IRunner(ICallback, ILogger, ABC):
  .. note::
  Please follow the `minimal examples`_ sections for use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
 
  """
 
@@ -270,7 +270,7 @@ def on_experiment_start(self, runner: "IRunner"):
 
  # self.engine = self.get_engine()
  self.engine.setup(local_rank=self._local_rank, world_size=self._world_size)
- if self.engine.is_local_main_process:
+ if self.engine.is_main_process:
  self.loggers = self.get_loggers()
  self.log_hparams(hparams=self.hparams)
  with self.engine.local_main_process_first():

diff --git a/catalyst/data/dataset.py b/catalyst/data/dataset.py
@@ -45,7 +45,7 @@ class SelfSupervisedDatasetWrapper(Dataset):
  Args:
  dataset: original dataset for augmentation
  transforms: transforms which will be applied to original batch to get both
- left and right output batch.
+  left and right output batch.
  transform_left: transform only for left batch
  transform_right: transform only for right batch
  transform_original: transforms which will be applied to save original in batch
@@ -78,6 +78,9 @@ class SelfSupervisedDatasetWrapper(Dataset):
  transforms=transforms
  )
 
+ for transformed_sample, aug_1, aug_2 in cifar_contrastive:
+ pass
+
  .. _`A Simple Framework for Contrastive Learning of Visual Representations`:
  https://arxiv.org/abs/2002.05709
  """

diff --git a/catalyst/data/loader.py b/catalyst/data/loader.py
@@ -26,16 +26,13 @@ def __getattr__(self, key):
  Gets attribute by ``key``.
  Firstly, looks at the ``origin`` for the appropriate ``key``.
  If none founds - looks at the wrappers attributes.
- If could not found anything - raises ``NotImplementedError``.
+ If could not found anything - returns ``None``.
 
  Args:
  key: attribute key
 
  Returns:
  attribute value
-
- Raises:
- NotImplementedError: if could not find attribute in ``origin`` or ``wrapper``
  """
  some_default_value = "_no_attr_found_"
  value = self.origin.__dict__.get(key, some_default_value)
@@ -46,7 +43,7 @@ def __getattr__(self, key):
  # value = getattr(self, key, None)
  if value != some_default_value:
  return value
- raise NotImplementedError()
+ return None
 
  def __len__(self) -> int:
  """Returns length of the wrapper loader.

diff --git a/catalyst/metrics/_accuracy.py b/catalyst/metrics/_accuracy.py
@@ -130,7 +130,7 @@ class AccuracyMetric(TopKMetric):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(
@@ -253,7 +253,7 @@ class MultilabelAccuracyMetric(AdditiveMetric, ICallbackBatchMetric):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(

diff --git a/catalyst/metrics/_additive.py b/catalyst/metrics/_additive.py
@@ -130,7 +130,7 @@ def on_loader_end(self, runner):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(self, compute_on_call: bool = True, mode: str = "numpy"):

diff --git a/catalyst/metrics/_auc.py b/catalyst/metrics/_auc.py
@@ -131,7 +131,7 @@ class AUCMetric(ICallbackLoaderMetric):
  .. note::
  Please follow the `minimal examples`_ sections for more use cases.
 
- .. _`minimal examples`: http://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
+ .. _`minimal examples`: https://github.com/catalyst-team/catalyst#minimal-examples # noqa: E501, W505
  """
 
  def __init__(