ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation

Existing stylized motion generation models have shown theirremarkable ability to understand specific style information from the stylemotion, and insert it into the content motion. However, capturing intra-style diversity, where a single style should correspond to diverse motionvariations, remains a significant challenge. In this paper, we propose aclustering-based framework, ClusterStyle, to address this limitation.Instead of learning an unstructured embedding from each style motion,we leverage a set of prototypes to effectively model diverse style pat-terns across motions belonging to the same style category. We considertwo types of style diversity: global-level diversity among style motionsof the same category, and local-level diversity within the temporal dy-namics of motion sequences. These components jointly shape two struc-tured style embedding spaces, i.e., global and local, optimized via align-ment with non-learnable prototype anchors. Furthermore, we augmentthe pretrained text-to-motion generation model with the Stylistic Mod-ulation Adapter (SMA) to integrate the style features. Extensive ex-periments demonstrate that our approach outperforms existing state-of-the-art models in stylized motion generation and motion style transfer.Project page: https://1233chen.github.io/ClusterStyle/.