When Sinks Help or Hurt: Unified Framework for Attention Sink in MLLMs

Attention sinks are defined as tokens that attract dispro-portionate attention. While these have been studied in single modalitytransformers, their cross-modal impact in Large Vision-Language Mod-els (LVLM) remains largely unexplored: are they redundant artifactsor essential global priors? This paper first categorizes visual sinks intotwo distinct categories: ViT-emerged sinks (V-sinks), which propagatefrom the vision encoder, and LLM-emerged sinks (L-sinks), which arisewithin deep LLM layers. Based on the new definition, our analysis re-veals a fundamental performance trade-off: while sinks effectively encodeglobal scene-level priors, their dominance can suppress the fine-grainedvisual evidence required for local perception. Furthermore, we identifyspecific functional layers where modulating these sinks most significantlyimpacts downstream performance. To leverage these insights, we proposeLayer-wise Sink Gating (LSG), a lightweight, plug-and-play module thatdynamically scales the attention contributions of V-sink and the restvisual tokens. LSG is trained via standard next-token prediction, re-quiring no task-specific supervision while keeping the LVLM backbonefrozen. In most layers, LSG yields improvements on representative mul-timodal benchmarks, effectively balancing global reasoning and preciselocal evidence.1