PanoRec: Spatially-Structured Sequence Modeling for Multi-Granularity Panoramic Retrieval

Panoramic images capture holistic environments, yet retriev-ing them using fine-grained, localized textual descriptions remains a fun-damental challenge. Through empirical analysis, we reveal that modernVision-Language Models (VLMs) suffer from severe semantic dilutionwhen processing panoramic inputs. By compressing an information-densepanorama into a single global embedding, VLMs inevitably submerge lo-cal details within vast backgrounds, restricting retrieval to coarse scene-level matching. To overcome this bottleneck, we propose PanoRec, amulti-granularity panoramic retrieval framework built on a spatially-structured sequence modeling paradigm. Specifically, PanoRec serializesdistortion-free cubemap faces and a downsampled global panorama along-side spatial anchor tokens into a unified sequence. This enables the ef-ficient extraction of decoupled local and global representations within asingle forward pass. To effectively supervise this multi-granularity featurespace, we formulate a joint spatial InfoNCE objective. For local match-ing, we adopt a MaxSim routing strategy that dynamically aligns eachquery with its most relevant cubemap face. Crucially, this strategy notonly suppresses background noise during inference but also inherentlyintroduces a powerful hard spatial negative mining mechanism duringtraining. Extensive experiments demonstrate that PanoRec achieves im-pressive performance across multiple scenarios, effectively unifying holis-tic scene-level retrieval with fine-grained spatial discrimination.