Finetune SemanticSegmentation with ImageEmbedder backbone

[ethanwharris]

Discussed in #1288

^{Originally posted by sirtris April 11, 2022}
Hi,
I have pre-trained my own network backbone using the ImageEmbedder. I would now use this model in a down-stream segmentation task. How could I do this?

For a classification task I would try:
model = ImageClassifier.load_from_checkpoint("image_classification_model.pt")
The problem is that the SemanticSegmentation() has no function load_from_checkpoint

Thanks

[sirtris]

For the meantime are there any suggestions for a workaround?
I usually use the Segmentation Models Pytorch library. Is there a way to use the backbone in one of their models?

[aniketmaurya]

hi @sirtris, the Segmentation task uses SMP for creating the backbone (encoder) and head (decoder) of the segmentation model. And in-order to create a head you need to pass the backbone to the head constructor.

To use a custom backbone you might have to make change to create_model(...) function of the SMP library instead.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.