ProMerge: Prompt and Merge for Unsupervised Instance Segmentation

Unsupervised instance segmentation aims to segment distinct object instances in an image without relying on human-labeled data. This field has recently seen significant advancements, partly due to the strong local correspondences afforded by rich visual feature representations from self-supervised models (e.g., DINO). Recent state-of-the-art approaches tackle this challenge by framing instance segmentation as a graph partitioning problem, solved via a generalized eigenvalue system (i.e., normalized-cut) using the self-supervised features. While effective, this strategy is limited by its computational demands, leading to slow inference speeds. In our work, we propose Prompt and Merge (ProMerge), a computationally efficient yet competitive method. We begin by leveraging self-supervised visual features to obtain initial groupings of patches and apply a strategic merging to these segments, aided by a sophisticated background-based mask pruning technique. ProMerge not only yields competitive results but also offers a significant reduction in inference time compared to state-of-the-art normalized-cut-based approaches. Furthermore, by training an object detector (i.e., Cascade Mask R-CNN) using our mask predictions as pseudo-labels, our results reveal that this detector surpasses current leading unsupervised methods. The code will be made publicly available.