ECC: Encoder-Centric Corruption for Fine-Grained Vision in VLMs

Modern Vision-Language Models (VLMs) often suffer from granularity mismatch, failing to encode high-density visual details not explicitly described in abstract text prompts, which results in low-frequency bias and object hallucination. While additive noise naturally introduces high-frequency variations, suggesting corruption-to-reconstruction (C2R) as a potential remedy, existing noise-based extensions of masked image modeling (MIM) surprisingly yield little improvement on recognition tasks. We identify the cause as a structural limitation: recent trends toward decoder-style designs prevent noise-induced signals from being deeply integrated into the encoder representations. To address this issue, we propose Encoder-Centric Corruption (ECC), a framework guided by three principles: (i) Encoder-Centricity, performing restoration within the encoder to shape transferable features; (ii) FeatureLevel Noising, injecting noise at intermediate layers to capture finegrained textures; and (iii) Task Disentanglement, separating mask reconstruction and denoising via a disruption loss. ECC surpasses prior MIM and noise-based methods by up to 8.1%. Moreover, integrating ECC into large-scale frameworks such as CLIP and AIM-v2 consistently improves performance, demonstrating its effectiveness as a scalable pretraining strategy for next-generation foundation models. Code is available at https://github.com/doihye/ecc.