Skip to yearly menu bar Skip to main content


Poster

Cross-Domain Semantic Segmentation on Inconsistent Taxonomy using VLMs

Jeongkee Lim · Yusung Kim

Strong blind review: This paper was not made available on public preprint services during the review process Strong Double Blind
[ ]
Fri 4 Oct 1:30 a.m. PDT — 3:30 a.m. PDT

Abstract:

The challenge of semantic segmentation in Unsupervised Domain Adaptation (UDA) emerges not only from domain shifts between source and target images but also from discrepancies in class taxonomies across domains. Traditional UDA research assumes consistent taxonomy between the source and target domains, thereby limiting their ability to recognize and adapt to the taxonomy of the target domain. This paper introduces a novel approach, Cross-Doamin Semantic Segmentation on Inconsistent Taxonomy using Vision Language Models (CSI), which effectively performs domain-adaptive semantic segmentation even in situations of source-target class mismatches. CSI leverages the semantic generalization potential of Visual Language Models (VLMs) to create synergy with previous UDA methods. It utilizes segment reasoning obtained through traditional UDA methods, alongside the rich semantic knowledge embedded in VLMs, to perform relabeling to classes of the target domain. This approach allows for effective adaptation to changed taxonomies without requiring any ground truth label for the target domain. Our method has shown to be effective across various benchmarks in situations of inconsistent taxonomy settings, such as coarse-to-fine taxonomy and open taxonomy, and demonstrates consistent synergy effects when integrated with previous state-of-the-art UDA methods.

Live content is unavailable. Log in and register to view live content