Sashuai Zhou, Qiang Zhou, Jijin Hu, Hanqing Yang, Yue Cao, Junpeng Ma, Yinchao Ma, Jun Song, Tiezheng Ge, Cheng Yu, Bo Zheng, Zhou Zhao. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Sketch animation, the transformation of static sketches into dynamic experiences, is an essential tool for visual expression. Current methods rely on global-level optimization and neglect instance-level priors, resulting in optimization mismatch and overfitting across multiple sketches. To solve this problem, information about the attention map, representing different instances is needed to guide the optimization. In this work, we propose AnimateSketches, a novel optimization-based framework that focuses on animating complex vectorized sketches. We introduce prompt-guided instance-aware mask generation (PGIM), which leverages attention maps from a pretrained diffusion model to guide the optimization of individual sketches. In addition, we use mask-based score distillation sampling (MSDS) to maintain the integrity of untargeted sketches. Quantitative and qualitative evaluations show the superiority of our approach over baseline methods in terms of visual quality and instance-level prompt-guided correspondence.

While diffusion models have revolutionized generative AI, their application to human sketch generation, especially in the creation of complex yet concise and recognizable sketches, remains largely unexplored. Existing efforts have primarily focused on vector-based sketches, limiting their ability to handle intricate sketch data. This paper introduces an innovative extension of diffusion models to pixellevel sketch generation, addressing the challenge of dynamically optimizing the guidance scale for classifier-guided diffusion. Our approach achieves a delicate balance between recognizability and complexity in generated sketches through scale-adaptive classifier-guided diffusion models, a scaling indicator, and the concept of a residual sketch. We also propose a three-phase sampling strategy to enhance sketch diversity and quality. Experiments on the QuickDraw dataset showcase the potential of diffusion models to push the boundaries of sketch generation, particularly in complex scenarios unattainable by vector-based methods.