ViPer: Visual Personalization of Generative Models via Individual Preference Learning

Personalized image generation involves creating images aligned with an individual’s visual preference. Current generative models are, however, tuned to produce outputs that appeal to a broad audience, and personalization to individual users' visual preferences relies on iterative and manual prompt engineering by the user, which is neither time-efficient nor scalable. We propose to personalize the image generation process by first inviting users to comment on a small selection of images, explaining why they like or dislike each. Based on these comments, we infer a user’s liked and disliked visual attributes, i.e., their visual preference, using a large language model. These attributes are used to guide a text-to-image model toward producing images that are personalized towards the individual user's visual preference. Through a series of user tests and large language model guided evaluations, we demonstrate that our proposed method results in generations that are well aligned with individual users' visual preferences.