Steering Diffusion Models via Class-Contrastive Influence for Few-Shot Classification

When labeled data are scarce, off-the-shelf diffusion modelscan augment training sets for few-shot medical image classification, butnot all generated samples are equally useful for the downstream task. Ex-isting approaches largely improve synthetic data by increasing realism,diversity, or domain adaptation, while overlooking a more fundamen-tal question: how should sample usefulness for classification be measuredand optimized? We address this with Class-Contrastive Influence (C2I),a criterion that quantifies a sample’s usefulness through its gradient-based influence on the classifier. We find that effective samples exhibita strong C2I gap: their loss gradients align with validation gradientsfrom the same class and oppose those from other classes. Our anal-ysis further suggests that such high-C2I samples are hard, boundary-proximal examples that help refine the decision boundary and improverobustness. Building on this insight, we fine-tune diffusion models withreinforcement learning using a C2I-based reward to steer generation to-ward class-informative samples. Across several few-shot medical imagingbenchmarks, C2I-guided generation improves downstream accuracy androbustness over diffusion-based augmentation baselines, showing thatsynthetic augmentation is most effective when guided by task usefulnessrather than image quality alone.