Iterative Perceptual Alignment for VLMs via Deterministic Reconstruction Feedback

While high-quality preference data is vital for fine-grainedvision-language alignment, its creation remains bottlenecked by the highcost and subjective noise of human annotation. We propose a self-supervisedframework that derives solid, highly discriminative supervisory signalsthrough deterministic reconstruction feedback. By transforming abstractsemantic matching into concrete visual reconstruction, our approach pro-vides reliable consistency signals that overcome the input limits of tra-ditional models. A reward model trained on our generated data achievescompetitive accuracy across multiple zero-shot preference benchmarksusing only 20% of the typical training data volume. Furthermore, fine-tuning VLMs via Direct Preference Optimization (DPO) based on thisdata significantly improves performance in handling complex, high-densitydescriptions. Our results demonstrate the potential of reconstruction-driven self-correction for low-cost, high-precision alignment of multi-modal models. Our code of the scoring model is released at https://github.com/XC0053/iterative-perceptual-alignment.