ColorFM: An Optimization-to-Learning Framework for Color Transfer via Flow Matching

Color transfer aims to align the color distribution of a sourceimage with that of a reference image while preserving structural andsemantic consistency. However, existing methods often suffer from inac-curate global mapping, semantic misalignment, and visual artifacts. Toaddress these issues, we propose ColorFM, an optimization-to-learningframework. ColorFM connects online optimization to offline inference byreformulating color transfer as the transport of pixel distributions alongvelocity fields via Flow Matching. Specifically, we introduce ColorFM-O, an instance-specific optimization scheme that fits the velocity fieldthrough hierarchical color coupling guided by semantic priors. By nu-merically integrating the induced flow trajectories, ColorFM-O producesprecise and semantically consistent color transfer results, while generat-ing high-quality paired data as pseudo-supervision. Building upon this,we design ColorFM-L, an efficient feed-forward model trained on thegenerated pairs. Through implicit state modeling, ColorFM-L extractsdeep semantic features to predict flow parameters for bidirectional lin-earized transport, ensuring accurate color transfer. Extensive experi-ments demonstrate that ColorFM-L outperforms state-of-the-art meth-ods in visual quality, structural fidelity, and semantic consistency, suc-cessfully combining the accuracy of optimization with the speed of feed-forward inference.