Add early stopping with warmup. Remove mandatory background label in semantic segmentation task #3515
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Motivation:
Early stopping from lightning doesn't have a warmup parameter and it leads to underfitting when training on small Geti target datasets. I added two thresholds to have invariant for small datasets that can fit in one batch and for larger data
Inserting a background label actually can lead to class mismatches for the Segmentation head, sometimes datasets don't have background classes at all. I tried to divide the Geti case with polygons and ordinary training from file masks. When dm_dataset represents annotations with Polygons -> we insert a background label and create a head with num_classes + 1. For other cases, dataset_meta.json should explicitly reflect all classes for training. So, the number of indices in masks == num categories in dataset_meta.json.
Before these changes, if we add a background class to dataset_meta.json, the training will fail, because OTX deletes the background class during prefiltering. Now, the background class must be presented in a metadata file when needed.
How to test
Checklist
License
Feel free to contact the maintainers if that's a concern.