CURE: Contextual Debiasing and Unbiased Refinement for Training-Free Open-Vocabulary Semantic Segmentation

Recent advances in large-scale vision–language models (VLMs)have greatly advanced open-vocabulary semantic segmentation (OVSS),enabling segmentation beyond predefined category constraints. However,due to the image-level pre-training objectives of VLMs, existing training-free OVSS methods often suffer from global contextual artifacts, degrad-ing their ability to capture fine-grained and spatially localized semantics.In this work, we propose a novel framework, termed CURE (Contextualdebiasing and Unbiased REfinement), to address these limitations bymitigating both explicit and implicit global biases. Technically, CUREemploys a reject-and-reconstruction module to identify explicit outliertokens and reconstruct them using local neighborhood information. Aglobal contextual debiasing module is further introduced to suppressresidual implicit bias. Finally, a spatial correlation refinement moduleis designed to enhance spatial coherence by leveraging mid-level simi-larity patterns. Extensive experiments on five datasets demonstrate thatCURE consistently improves segmentation quality. When integrated intosix baseline methods, it yields average mIoU gains of 2.8% and 7.7% withViT-B/16 and ViT-L/14 backbones, respectively.