TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs

Multimodal large language models (MLLMs) often halluci-nate, producing fluent but visually ungrounded outputs, partly becausedirect preference optimization (DPO) overfits to superficial linguisticcues under static preference supervision. We propose TARS, a token-adaptive preference strategy that reformulates DPO as a min-max op-timization problem: the inner maximization perturbs visual-agnostic to-kens to induce worst-case distributional shifts, while the outer minimiza-tion enforces alignment with causal visual signals rather than surface-level patterns. A spectral alignment loss further regularizes hidden rep-resentations in the frequency domain via the Fast Fourier Transform(FFT). With only 4.8k preference samples and no expert feedback, TARShalves the hallucination rate (26.4% → 13.2%) and reduces the cognitionscore from 2.5 to 0.4, outperforming standard DPO. It also surpasses 5×larger LLM-based data augmentation (28.8k samples; Hal-Rate 16.0% vs.13.2%) and narrows the gap to GPT-4o, suggesting that token-adaptiveoptimization improves data efficiency beyond simply scaling augmentedpreference data.