Aligning text-to-image (T2I) diffusion models with human preferences has emerged as a critical research challenge. While Direct Preference Optimization (DPO) has established a foundation for preference learning in large language models (LLMs), its extension to diffusion models remains limited in alignment performance. In this work, we propose an enhanced version of Diffusion-DPO by introducing a stable reference model update strategy. This strategy facilitates the exploration of better alignment solutions while maintaining training stability. Moreover, we design a timestep-aware optimization strategy that further boosts performance by addressing preference learning imbalance across timesteps. Through the synergistic combination of our exploration and timestep-aware optimization, our method significantly improves the alignment performance of Diffusion-DPO on human preference evaluation benchmarks, achieving state-of-the-art results.

Recent advancements in text-to-image diffusion models have shown impressive results but often fail to fully capture user’s intent. Existing methods combining textual inputs with bounding boxes or region masks lack precise spatial guidance, leading to misaligned or unintended object orientations. To address these issues, we propose Scribble-Guided Diffusion (ScribbleDiff), a training-free approach that employs user-provided scribbles as visual prompts for image generation. However, incorporating scribbles poses challenges due to their sparse and thin nature, which complicates accurate alignment. To resolve this, we introduce moment alignment and scribble propagation, enabling effective and flexible alignment between generated images and scribble inputs. Experiments on the PASCAL-Scribble dataset demonstrate notable improvements in spatial control and consistency, validating the effectiveness of our method in scribble-guidance.