Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Shaokai Chen, Mengshu Sun, Binbin Hu, Zhiqiang Zhang, Lei Liang, Wen Zhang, Huajun Chen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Current Text-to-Image (T2I) generation methods struggle to accurately create images with complex object relationships and scene compositions. To overcome these challenges, we propose KAIG, a novel text-to-image generative model that integrates a knowledge graph into the image generation process. Unlike traditional models, KAIG uses structured knowledge to enhance the retrieval of relevant information, enabling the generation of high-quality, contextually rich, and semantically consistent images from multi-modal inputs. We introduce a two-stage training strategy: first, condition adapters are trained to align multi-modal inputs, followed by fine-tuning the entire diffusion model. This approach ensures precise alignment between retrieved conditions and the image generation process, leading to an efficient and scalable pipeline. Our experiments on two popular datasets, MS-COCO and CUB-200-2011, show that KAIG consistently outperforms existing methods in both image quality and consistency. Notably, KAIG can seamlessly integrate with any pre-trained diffusion model, requiring minimal additional training while achieving superior results. Ultimately, KAIG demonstrates strong potential for addressing key limitations in current T2I models and advancing the field of image synthesis.

Facial expressions contain massive affective information. Previous methods have focused on using diverse models based on CNN or Transformer to handle the facial expression recognition(FER) task. However, most of them treat the FER task as a general image classification task and neglect the impact of regions of interest (ROIs) for the performance of FER. To verify the influence of different ROIs, in this paper, we propose a vision Transformer based onFusedShiftedwindows (FSwin), called FSwin Transformer. The semantic information of the face, obtained by ROIs, guides the FSwin Transformer to focus more on the key regions. The fused shifted windows enable the model to perform global semantic interactions, allowing it to concentrate on key regions without losing the topological structural information of the entire face. Additionally, learnable parameters are introduced to learn the feature weights expressed by each ROI, helping the model dynamically adjust the attention distribution. We have conducted controlled experiments to quantitatively verify the impact of ROIs. And the experimental results show that with the increase of the number of ROIs, the accuracy of FER is significantly improved, demonstrating that the key ROIs play an important role in feature extraction. The results on Jaffe, CK+, FER2013, AffectNet have reached 99.8%, 99.0%, 74.8%, and 68.9%, respectively, which all set new state-of-the-art. Extensive cross-dataset experiments also show that the FSwin Transformer has good generalization ability, proving that our proposed model has a beneficial effect on FER tasks.