Learning Accurate Segmentation Purely from Self-Supervision

Accurately segmenting objects without any manual anno-tations remains one of the core challenges in computer vision. In thiswork, we introduce Selfment, a fully self-supervised framework that seg-ments foreground objects directly from raw images without human labels,pretrained segmentation models, or any post-processing. Selfment firstconstructs patch-level affinity graphs from self-supervised features and ap-plies NCut to obtain an initial coarse foreground-background separation.We then introduce Iterative Patch Optimization (IPO), a feature-space refinement procedure that progressively enforces spatial coherenceand semantic consistency through iterative patch clustering. The refinedmasks are subsequently used as supervisory signals to train a lightweightsegmentation head with contrastive and region-consistency objectives,allowing the model to learn stable and transferable object representations.Despite its simplicity and complete absence of manual supervision, Self-ment sets new state-of-the-art (SoTA) results across multiple benchmarks.It achieves substantial improvements on Fmax over previous unsupervisedsaliency detection methods on ECSSD (+4.0\%), HKUIS (+4.6\%), andPASCAL-S (+5.7\%). Moreover, without any additional fine-tuning, Self-ment demonstrates remarkable zero-shot generalization to camouflagedobject detection tasks (e.g., .910 Sm on CHAMELEON and .792 \protect \mathcal {F}{x0008eta }^{\omega }on CAMO), outperforming all existing unsupervised approaches andeven rivaling the SoTA fully supervised methods. Codes and weights areavailable at: https://geshang777.github.io/Selfment/.