FD²: A Dedicated Framework for Fine-Grained Dataset Distillation

Dataset distillation (DD) compresses a large training set intoa small synthetic set, reducing storage and training cost, and has shownstrong results on general benchmarks. Decoupled DD further improvesefficiency by splitting the pipeline into pretraining, sample distillation,and soft-label generation. However, existing decoupled methods largelyrely on coarse class-label supervision and optimize samples within eachclass in a nearly identical manner. On fine-grained datasets, this oftenyields distilled samples that (i) retain large intra-class variation withsubtle inter-class differences and (ii) become overly similar within thesame class, limiting localized discriminative cues and hurting recognition.To solve the above-mentioned problems, we propose FD2 , a dedicatedframework for Fine-grained Dataset Distillation. FD2 localizes discrimi-native regions and constructs fine-grained representations for distillation.During pretraining, counterfactual attention learning aggregates discrim-inative representations to update class prototypes. During distillation,a fine-grained characteristic constraint aligns each sample with its classprototype while repelling others, and a similarity constraint diversifies at-tention across same-class samples. Experiments on multiple fine-grainedand general datasets show that FD2 integrates seamlessly with decou-pled DD and improves performance in most settings, indicating strongtransferability. Code is available at https://github.com/Guang000/FD2.