How to Load a Close-set Fine-tuned Grounding Dino into Grounded SAM

[narchitect]

Hello. Following issue #241, I fine-tuned a grounding Dino via mmdetection for a close-set. However, when I try to load these fine-tuned weights into Grounded SAM, it says the model layers are different, and thus, loading fails. I'm curious if there's a way to successfully load a close-set fine-tuned Grounding Dino model into Grounded SAM. It seems it would be highly effective to be able to load the close-set model for downstream tasks. Thanks in advance for your response.

[rcao-hk]

Same issue, look at the network defination is slightly different.

[shz0519]

Hello, how to solve this problem?

Same issue, look at the network defination is slightly different.

[narchitect]

Hey @rcao-hk and @shz0519, thanks for your attention, and sorry for the late reply. I’ve resolved the issue by creating new code to perform transfer learning on both models using MMDetection's Grounding DINO and the SAM model. I referenced the Grounded SAM code and modified the bounding box extraction with MMDetection's DINO model.

[yoshi-show1]

Hey @rcao-hk and @shz0519, thanks for your attention, and sorry for the late reply. I’ve resolved the issue by creating new code to perform transfer learning on both models using MMDetection's Grounding DINO and the SAM model. I referenced the Grounded SAM code and modified the bounding box extraction with MMDetection's DINO model.

Thank you for the information. If possible, could you share the method? I am facing the same issue and am struggling with it