Think2Sing: Orchestrating Structured Motion Subtitles for Singing-Driven 3D Head Animation

Singing-driven 3D head animation is a compelling yet underexplored task with broad applications in virtual avatars, entertainment, and education. Existing speech-driven approaches, which typically map audio directly to motion through implicit phoneme-to-viseme correspondences, often yield over-smoothed, emotionally flat, and semantically inconsistent results. These limitations render them inadequate for the unique demands of singing-driven animation. To address this challenge, we propose Think2Sing, a unified diffusion-based framework that integrates pretrained large language models to generate semantically consistent and temporally coherent 3D head animations conditioned on both lyrics and acoustics.

Central to our framework is the introduction of motion subtitles, a structured, time-aligned representation generated via a Singing Chain-of-Thought process with acoustic-guided retrieval. These subtitles provide region-specific expressive cues that serve as interpretable priors for animation synthesis. We further formulate head animation as motion intensity prediction over key facial regions, enabling fine-grained control and more faithful expressive modeling. To support this paradigm, we construct the first multimodal singing dataset with synchronized 3D motion, acoustic features, and aligned motion subtitles, enabling semantically grounded and expressive motion learning. Extensive experiments demonstrate that Think2Sing significantly outperforms state-of-the-art methods in realism, expressiveness, and emotional fidelity. Furthermore, our framework supports flexible subtitle-conditioned editing, enabling precise and user-controllable animation synthesis.