Selective Search Algorithm for Semantic and Instance Segmentation

[chenller]

Selective Search Algorithm for Semantic and Instance Segmentation

Description

The proposed tool aims to implement a feature for semantic and instance segmentation based on the Selective Search algorithm. This tool will enable users to merge segmented sub-regions of an image through interactive operations and create labels for them, thereby simplifying and accelerating the annotation process.

Functional Steps

1. Input Image: The user inputs an image for processing.

2. Apply Selective Search Algorithm: The Selective Search algorithm is applied to the input image to segment it and generate multiple sub-regions.

3. Merge Sub-regions

   • Based on Points: The user inputs one or more points, and the sub-regions containing these points are merged (must be adjacent). (A possible scheme for determining adjacency: First, generate the regions, then perform a connectivity domain analysis to determine if there are two connected domains, one being the background and the other the annotation box.)
   • Based on Bounding Box: The user inputs a bounding box, and the sub-regions within this box are merged. (A possible merging scheme: Determine the proportion of the overlap between the sub-region and the bounding box, set a threshold, and merge the sub-regions where the overlap exceeds this threshold.)
     

4. Smooth boundaries (optional)

5. Create Labels: After merging the sub-regions, the user creates labels for them.

Benefits

Simplified Annotation Process: The Selective Search algorithm allows users to quickly extract meaningful regions from an image, greatly simplifying the complexity of manual annotation.
Increased Efficiency: Automated image segmentation and intelligent merging strategies reduce the time consumption for users during the annotation process.
CPU-friendly: The tool does not require expensive GPU resources, allowing more users to run the tool on their regular hardware.
Wide Applicability: The tool is suitable for a variety of image segmentation tasks, including but not limited to semantic segmentation and instance segmentation.

References

Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T. et al. Selective Search for Object Recognition. Int J Comput Vis 104, 154–171 (2013). https://doi.org/10.1007/s11263-013-0620-5