Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

Vision-language models (VLMs) achieve strong performanceon spatial reasoning benchmarks, yet it remains unclear whether this re-flects structured 3D understanding or reliance on statistical shortcuts innatural images. We introduce a representation-level analysis frameworkthat constructs minimal contrastive pairs to measure how spatial axes areorganized and disentangled within VLM embeddings. Our analysis acrossmultiple model families reveals a consistent vertical-distance entangle-ment: models conflate vertical image position with distance, mirroringthe perspective bias of natural photographs. This bias produces a signif-icant accuracy gap between perspective-consistent and counter-heuristicexamples, and intensifies under data scaling even as overall benchmarkaccuracy improves. We further show that models with similar benchmarkscores can exhibit different internal representations, and that these dif-ferences predict accuracy and robustness across diverse spatial reasoningbenchmarks. To isolate this bias from evaluation-set skew, we introduceSpatialTunnel, a synthetic benchmark designed to expose spatial short-cut biases by removing common correlations present in natural images.Experiments suggest that the entanglement is model-intrinsic, and thatmodels with well-separated spatial axes exhibit greater robustness, indi-cating that well-structured spatial representations lead to more reliablespatial reasoning across diverse benchmarks. Code and benchmark areavailable on the project website.