Most visual-guided image inpainting methods based on generative adversarial networks (GANs) struggle when the missing region has weak correlations with the surrounding visual context. Recently, diffusion-based methods guided by textual context have been proposed to address this limitation by leveraging additional semantic information to restore corrupted objects. However, these models typically involve more parameters and exhibit slower generation speeds compared to GAN-based approaches. To address this problem, we propose a novel text-guided image inpainting model, the cross-modal semantic fusion generative adversarial network (CSF-GAN). CSF-GAN is designed as a one-stage GAN with the following key contributions. First, a novel semantic fusion module (SFM) is introduced to integrate sentence- and word-level textual context into the inpainting process, enabling more effective guidance from multi-granularity semantic information. Second, a newly designed word-level local discriminator provides detailed feedback to the generator, enhancing the accuracy of generated content in alignment with word-level semantics. Third, two loss functions, the inpainting loss and edge loss, are employed to enhance both structural coherence and textural realism in the generated results. Extensive experiments on two benchmark datasets demonstrate that CSF-GAN outperforms state-of-the-art methods.

Generating realistic images based on text descriptions remains challenging in computer vision. Existing multi-stage generation methods are sufficient to generate high-resolution images. However, these methods mainly use one sentence to synthesize images, which are difficult to extract adequate semantic features, resulting in the generated images being far apart from ground-truth images. In this paper, we propose a Multi-Sentence Complementary Generative Adversarial Network, MSCGAN, which assists in generating accurate images by fusing the same semantics from different sentences and preserving their unique semantics. More specifically, the latest BERT model is employed to identify semantic features and a multi-semantic fusion module (MSFM) is designed to fuse the semantic features of different sentences. Besides, a pre-trained cross-modal contrast similarity model (CCSM) is developed to explore fine-grained loss on generated images. Moreover, a multi-sentence joint discriminator is designed to ensure that the generated images match all sentences. Experiments and ablation studies on CUB and MS-COCO datasets demonstrate the significant superiority of the proposed method compared to state-of-the-art methods.

A new stopping criterion for active learning SVM is proposed. It takes advantage of the low density separation idea, which is extensively used in semi-supervised learning and unsupervised learning methods. When the separating hyperplane of an active learning SVM (ALSVM) lies in a sparse region in t he feature space, its generalization error is expected to have reached a local minimum, and active learning can stop in such circumstances. An improved Normalized Cut algorithm, Stochastic Search Normalized Cut (SSNcut), is employed for measuring whether SVM's separating hyperplane lies in a low density region. In each active learning iteration, the label vector predicted by SVM is compared with that obtained from SSNcut. If these two label vectors are very similar to each other, the SVM's separating hyperplane can be considered as lying in a sparse region. When calculating these label vectors, only points that are nearest to the current separating hyperplane of ALSVM are considered. This makes Normalized Cut less likely to be influenced by outliers. Experiments show that the proposed stopping criterion strikes a good balance between generalization performance and label cost.