Balancing the consistency of style and the integrity of content is the main challenge in arbitrary style transfer domain. Currently, local style details can be effectively captured by attention mechanism but easily produce distorted style patterns and inconsistent content structure. In this paper, we propose a Content Affinity Preserving Arbitrary Style Transfer (CAPAST) framework to ensure style features can be stably integrated into the content structure. Considering the local feature learning ability of CNN and the global feature representation advantage of transformer, a dual encoder is proposed to capture local and global features of images with the combination between transformer and CNN. In addition, a channel and spatially aligned attention (CSAA) is introduced to generate high-quality results by stably fusing style features and content features. In experiments, we demonstrated the superior performance of our method in preventing content structure distortion and maintaining consistency between style and content. Codes are available at https://github.com/miaopashi-zxy/CAPAST.

Abstract The Coronavirus Disease 2019 (COVID‐19) epidemic has constituted a Public Health Emergency of International Concern. Chest computed tomography (CT) can help early reveal abnormalities indicative of lung disease. Thus, accurate and automatic localisation of lung lesions is particularly important to assist physicians in rapid diagnosis of COVID‐19 patients. The authors propose a classifier‐augmented generative adversarial network framework for weakly supervised COVID‐19 lung lesion localisation. It consists of an abnormality map generator, discriminator and classifier. The generator aims to produce the abnormality feature map M to locate lesion regions and then constructs images of the pseudo‐healthy subjects by adding M to the input patient images. Besides constraining the generated images of healthy subjects with real distribution by the discriminator, a pre‐trained classifier is introduced to enhance the generated images of healthy subjects to possess similar feature representations with real healthy people in terms of high‐level semantic features. Moreover, an attention gate is employed in the generator to reduce the noise effect in the irrelevant regions of M . Experimental results on the COVID‐19 CT dataset show that the method is effective in capturing more lesion areas and generating less noise in unrelated areas, and it has significant advantages in terms of quantitative and qualitative results over existing methods.

Most face-inpainting methods perform well in face repair. However, these methods can only complete a single face image per input. Although existing various image-inpainting methods can achieve pluralistic image inpainting, they typically produce faces with distorted structures or the same texture. To resolve these shortcomings and achieve high-quality diverse face inpainting, we propose PFTANet, a two-stage pluralistic face-inpainting network that transforms attribute information. In the first stage, the face-parsing network is fine-tuned to obtain semantic facial region information. In the second stage, a generator consisting of SNBlock, CF_ShiftBlocks, and CF_MergeBlock, which ensures that high-quality pluralistic face results are generated, is used. Specifically, CF_ShiftBlocks completes pluralistic face generation by transforming the attribute information from the conditional face extracted by the attribute extractor and ensuring the consistency of the attribute information between the conditional and generated faces. CF_MergeBlock ensures structural consistency between the masked and background regions of the generated face using facial region semantic information. A multi-patch discriminator is used to enhance facial detail generation. Experimental results for the CelebA and CelebA-HQ datasets indicated that PFTANet achieved pluralistic and visually realistic face inpainting.

Deep learning has been widely applied into image inpainting. However, traditional image processing methods (i.e., patch-based and diffusion-based methods) generally fail to produce visually natural contents and semantically reasonable structures due to ineffectively processing the high-level semantic information of images. To solve the problem, we propose a stacked generator networks assisted by patch discriminator for image inpainting by multistage. In the proposed method, our generator network mainly consists of three-layer stacked encoder-decoder architecture, which could fuse different level feature information and achieve image inpainting via a coarse-to-fine hierarchical representation. Meanwhile, we split the masked image into different patches in each layer, which could effectively enlarge the receptive field and extract more useful features of images. Moreover, the patch discriminator is introduced to judge the patches of inpainting image are real or fake. In this way, our network can effectively utilize the semantic information to complete a fine result. Furthermore, both perceptual loss and style loss are used to improve the inpainting results in verse. Experimental results on Places2 and Paris StreetView illustrate that our approach could generate high-quality inpainting results, and our method is more effective than the existing image inpainting methods.

For patients with dilated cardiomyopathy (DCM), fast and accurate diagnosis is important to save lives. MRI is a non-invasive, effective medical imaging method that allows doctors to diagnose DCM. However, manual and semi-automatic segmentation is subjective, non-reproducible and time-consuming task. In this paper, a new attention-based convolutional encoder-decoder network is proposed to automatically segment my-ocardium in DCM, which assisting the doctor to quickly diagnose. In the proposed method, the attention mechanism module is used, which is able to fully highlight useful features that facilitate segmentation while suppress useless features that are not conducive to segmentation. Combining with the multi-scale convolution, our encoder-decoder network can accurately segment the my-ocardium in DCM. We verified our approach on 1155 myocardial MRI. Our network achieves the most advanced segmentation performance on the cardiac DCM dataset. Experiment results demonstrate the effectiveness of the proposed method.