FoundYou: A Unified Model for Personalized Segmentation and Retrieval

Personalized segmentation and personalized retrieval bothaim to identify the same physical object across different images. Whilethe former localizes the object within a target image, the latter retrievesimages where it appears. Despite this shared instance-level objective, thetwo tasks have largely evolved separately and are addressed with distinctsolutions. In this work, we introduce FoundYou, a unified framework builton the observation that Segment Anything 2 (SAM 2), trained to pre-serve object identity across video frames, inherently captures instance-level cues. We leverage this property to match objects across independentimages, enabling segmentation and retrieval to emerge as two outcomesof the same instance alignment process. This unified view unlocks newcapabilities beyond traditional benchmarks, including few-shot person-alized retrieval and promptable personalized segmentation with flexibleprompts. Extensive experiments show consistent gains over unified andtask-specific methods, including +18.4 mIoU on PerMIS and +17.8 mAPon ILIAS. Performance scales with additional references and remainsrobust to weaker prompts. Beyond personalization, FoundYou achievesstate-of-the-art results on category-level retrieval benchmarks. Notably,our approach keeps the SAM 2-small model entirely frozen and addsonly 5.9 M trainable parameters, yielding a 52 M-parameter model thatis over 75× faster and 20× smaller than the only prior unified solution.