Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidth

Vision-Language Models (VLMs) deployed as situated agentsin high-resolution visual environments require active perception — theability to dynamically decide where to look through operations like zoom-ing, cropping, and panning. However, current training paradigms pro-duce models that mimic the surface form of such operations withoutfunctionally depending on their outputs, a phenomenon we term lazyperception. We trace this to a fundamental learning asymmetry: whencoarse global views combined with language priors suffice for moderateaccuracy, the model has no incentive to learn harder multi-step visualsearch. If a model can succeed without actively looking, it will neverlearn to look. This motivates Starve to Perceive, a training paradigm thatconstrains visual bandwidth — restricting each observation to a tight to-ken budget so that no single view suffices for task completion, therebystrongly incentivizing strategic and efficient active perception. Despiterequiring no auxiliary losses, reward shaping, or architectural changes— serving as a minimal, plug-in modification to standard post-trainingpipelines — models trained under perceptual starvation achieve substan-tial gains of 5% average relative improvement across diverse benchmarks.Our codes and data will be publicly available at https://github.com/WhuanY/Starve2Perceive.