DeCoPatch: Revealing Causal Latent Subspaces in Vision-Language Models for GUI Grounding

Vision–Language Models (VLMs) have recently emerged asa powerful paradigm for autonomously perceiving and interacting withcomplex and dynamic Graphical User Interface (GUI) environments.However, their interaction capability remains fundamentally limited byinsufficient precision in spatial grounding. Intriguingly, simply overlay-ing visual markers on screenshots has been extensively shown to sub-stantially enhance the grounding performance of models. Despite its em-pirical effectiveness, the underlying mechanism—specifically, why theseexplicit visual cues can influence the model’s internal spatial understand-ing—remains largely unexplained. We find that the causal influence ofvisual markers on grounding performance is mediated by low-rank la-tent subspaces associated with specific neurons in the late decoder lay-ers. Based on this key insight, we propose DeCoPatch, a novel decode-time causal intervention method that precisely modulates the activa-tions of key neurons relevant to GUI grounding. DeCoPatch follows atwo-stage design: it first identifies top-k critical neurons during the pre-fill stage, effectively isolating a low-rank latent subspace, and then re-inforces these neurons during the decoding stage to causally steer themodel’s spatial grounding behavior. Extensive experiments on four chal-lenging benchmarks demonstrate that DeCoPatch consistently enhancesthe GUI grounding performance of diverse VLMs with almost no addi-tional computational overhead.