Personalizing MLLMs via Reinforced Multimodal Reference Game

Personalizing Multimodal Large Language Models (MLLMs)aims to recognize users’ unique concepts from visual data and providepersonalized responses. Although prior work has shown the benefit ofconcept descriptions and reasoning for this task, MLLM descriptions ofteninclude information, such as state and context, that does not help andmay in fact hinder the unique identification of the target concept amongother visually similar items. Effective descriptions of personal conceptsshould instead be accurate, discriminative, and free of distracting details.To achieve such descriptions, we introduce Reinforced Reference Game(RRG), a learning framework that promotes discriminative descriptionsthrough a novel reinforced multimodal reference game. The MLLM playsboth the roles of speaker and listener in a contrastive game setting, whosegoal is to effectively communicate discriminative information about atarget concept. Our approach formulates a verifiable contrastive rewardover hard positives (dissimilar views of the same concept) and hardnegatives (visually similar but different concepts). Empirically, RRGachieves state-of-the-art across multiple tasks on three personalizationbenchmarks. RRG generalizes to unseen domains and outperforms existingmethods based on concept descriptions and personalization-specific RLframeworks. We will release code and models in the project page.