FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models

Large Vision-Language Models (VLMs) typically process hun-dreds or even thousands of visual tokens per image or video frame, in-curring quadratic attention costs and significant information redundancy.Existing token-reduction methods either ignore the textual query or relyon deep attention maps, which become unstable under aggressive prun-ing and often lead to degraded semantic alignment. To address this, weintroduce FlashVLM, a text-guided visual token selection frameworkthat dynamically adapts visual inputs to the given query. Rather thanrelying on noisy attention weights, FlashVLM computes an explicit cross-modal similarity between projected image tokens and normalized textembeddings within the LLM space. It then fuses this extrinsic text-imagerelevance with intrinsic visual saliency using log-domain weighting andtemperature-controlled sharpening. Furthermore, a diversity-preservingpartitioning mechanism retains a minimal yet representative set of back-ground tokens to preserve global context. Evaluated under identical to-ken budgets and protocols, FlashVLM achieves beyond-lossless compres-sion—slightly surpassing the unpruned baseline while discarding up to77.8% of visual tokens on LLaVA-1.5. Remarkably, it maintains 92.8%performance even under an aggressive 94.4% compression rate. Extensiveexperiments across 14 image and video benchmarks demonstrate thatFlashVLM establishes a new state-of-the-art in efficiency-performancetrade-offs, exhibiting strong robustness and broad generalization acrossmainstream VLMs.