RubricRL: Simple Generalizable Rewards for Text-to-Image Generation

Reinforcement learning (RL) has recently emerged as a promis-ing approach for aligning text-to-image generative models with humanpreferences. A key challenge, however, lies in designing effective and in-terpretable rewards. Existing methods often rely on either compositemetrics (e.g., CLIP, OCR, and realism scores) with fixed weights or asingle scalar reward distilled from human preference models, which canlimit interpretability and flexibility. We propose RubricRL, a simple andgeneral framework for rubric-based reward design that offers greater in-terpretability, composability, and user control. Instead of using a black-box scalar signal, RubricRL dynamically constructs a structured rubricfor each prompt—a decomposable checklist of fine-grained visual cri-teria such as object correctness, attribute accuracy, OCR fidelity, andrealism—tailored to the input text. Each criterion is independently eval-uated by a multimodal judge (e.g., o4-mini), and a prompt-adaptiveweighting mechanism emphasizes the most relevant dimensions. This de-sign not only produces interpretable and modular supervision signalsfor policy optimization (e.g., GRPO or DiffusionNFT), but also enablesusers to directly adjust which aspects to reward or penalize. Experimentswith autoregressive and diffusion text-to-image models demonstrate thatRubricRL improves prompt faithfulness, visual detail, and generalizabil-ity, while offering a flexible and extensible foundation for interpretableRL alignment across text-to-image architectures.