Balancing the consistency of style and the integrity of content is the main challenge in arbitrary style transfer domain. Currently, local style details can be effectively captured by attention mechanism but easily produce distorted style patterns and inconsistent content structure. In this paper, we propose a Content Affinity Preserving Arbitrary Style Transfer (CAPAST) framework to ensure style features can be stably integrated into the content structure. Considering the local feature learning ability of CNN and the global feature representation advantage of transformer, a dual encoder is proposed to capture local and global features of images with the combination between transformer and CNN. In addition, a channel and spatially aligned attention (CSAA) is introduced to generate high-quality results by stably fusing style features and content features. In experiments, we demonstrated the superior performance of our method in preventing content structure distortion and maintaining consistency between style and content. Codes are available at https://github.com/miaopashi-zxy/CAPAST.

Accurate wave height prediction is essential for various marine operations. However, the complexity of the marine environment, influenced by numerous factors, underscores the importance of effectively leveraging available data. This paper introduces a novel spatiotemporal model, FourCastNet, which serves as a baseline for capturing wave height trends through spectral analysis. To refine spatiotemporal representations, we incorporate 3D convolution to extract local features from the data via nonlinear transformations. Employing Fourier transform, we convert the features into the frequency domain, filtering out high-frequency noise while enhancing the visibility of spatiotemporal patterns. Finally, we fuse various features and capture complex patterns via channel mixing, thereby enhancing the model’s predictive accuracy. Iterative forecasting techniques are applied to minimize prediction errors. Experiments using French wave reanalysis data, projected up to 24 hours at three-hour intervals, demonstrate that our method achieves approximately a 10% improvement in accuracy over existing state-of-the-art techniques, particularly for short-term forecasts within the first 6 hours.

In the Internet of Audio Things, communication security of the audio control terminal is vulnerable to copy-move threats, and detecting and locating audio copy-move forgery remains challenging nowadays. The forgery detection method based on deep learning achieves higher detection accuracy but fails to localize forged regions. To address this issue, this article proposes an S-Faster R-CNN model for audio copy-move forgery detection and localization. We integrate a novel Similarity Computation Module (SCM) into the Faster R-CNN framework, forming the S-Faster R-CNN model. Obtaining the integration of the SCM, which allows the S-Faster R-CNN to precisely localize forgery regions within the spectrogram. Finally, the image coordinate transformation algorithm is used to map these forged regions to the corresponding locations of the original audio waveform, thus completing the audio copy-move forgery detection and localization. Evaluated on three datasets, our method achieves an average recall of 90%, an average precision of 84%, and an average F1-score of 87%, respectively. Experimental results indicate that the S-Faster R-CNN outperforms state-of-the-art methods in both forgery detection accuracy and especially in localization. Moreover, the proposed method shows good robustness under multiple post-processing.

For the interpretability of deep neural networks (DNNs) in visual-related tasks, existing explanation methods commonly generate a saliency map based on the linear relation between output results and input features. However, when the explanation conflicts with a human visual examination, these methods do not provide further evidence to analyze the saliency explanation. Most may fail to provide feature attribution with identifiable semantics or produce misleading explanations due to their insufficient robustness. In this paper, we first propose four key characteristics (richness, adaptivity, exclusiveness, and fairness) to evaluate the existing linear relation-based explanation method, and then construct an interpretable linear model to satisfy them. We formalize the characteristics and develop a novel explanation method based on this. We extract and reconstruct key exclusive semantic features from the feature map using the Nonnegative Matrix Factorization (NMF) algorithm, utilize the information entropy model to determine the number of features adaptively and their richness, and then linearly combine each feature with fairly assigned weights using an approximate Shapley algorithm to generate the saliency map. Compared with the state-of-the-art methods, our explanations of different datasets and DNNs are more convincing and robust in terms of Average drop (AD), Average increase (AI), Deletions (Del), and Insertions (Ins). Our supplementary experiments provide sufficient evidence that the four characteristics guarantee the feasibility of feature attribution analysis and enhance the quality of the resulting explanations.

Motion blur estimation is a critical and fundamental task in scene analysis and image restoration. While most state-of-the-art deep learning-based methods for single-image motion image deblurring focus on constructing deep networks or developing training strategies, the characterization of motion blur has received less attention. In this paper, we innovatively propose a non-parametric Variational Bayesian Kernel Generation Network (VB-KGN) for characterizing motion blur in a single image. To solve this model, we employ the variational inference framework to approximate the expected statistical distribution of motion blur images in a data-driven manner. The qualitative and quantitative evaluations of our experimental results demonstrate that our proposed model can generate highly accurate motion blur kernels, significantly improving motion image deblurring performance and substantially reducing the need for extensive training sample preprocessing for deblurring tasks.

Abstract The Coronavirus Disease 2019 (COVID‐19) epidemic has constituted a Public Health Emergency of International Concern. Chest computed tomography (CT) can help early reveal abnormalities indicative of lung disease. Thus, accurate and automatic localisation of lung lesions is particularly important to assist physicians in rapid diagnosis of COVID‐19 patients. The authors propose a classifier‐augmented generative adversarial network framework for weakly supervised COVID‐19 lung lesion localisation. It consists of an abnormality map generator, discriminator and classifier. The generator aims to produce the abnormality feature map M to locate lesion regions and then constructs images of the pseudo‐healthy subjects by adding M to the input patient images. Besides constraining the generated images of healthy subjects with real distribution by the discriminator, a pre‐trained classifier is introduced to enhance the generated images of healthy subjects to possess similar feature representations with real healthy people in terms of high‐level semantic features. Moreover, an attention gate is employed in the generator to reduce the noise effect in the irrelevant regions of M . Experimental results on the COVID‐19 CT dataset show that the method is effective in capturing more lesion areas and generating less noise in unrelated areas, and it has significant advantages in terms of quantitative and qualitative results over existing methods.

Most face-inpainting methods perform well in face repair. However, these methods can only complete a single face image per input. Although existing various image-inpainting methods can achieve pluralistic image inpainting, they typically produce faces with distorted structures or the same texture. To resolve these shortcomings and achieve high-quality diverse face inpainting, we propose PFTANet, a two-stage pluralistic face-inpainting network that transforms attribute information. In the first stage, the face-parsing network is fine-tuned to obtain semantic facial region information. In the second stage, a generator consisting of SNBlock, CF_ShiftBlocks, and CF_MergeBlock, which ensures that high-quality pluralistic face results are generated, is used. Specifically, CF_ShiftBlocks completes pluralistic face generation by transforming the attribute information from the conditional face extracted by the attribute extractor and ensuring the consistency of the attribute information between the conditional and generated faces. CF_MergeBlock ensures structural consistency between the masked and background regions of the generated face using facial region semantic information. A multi-patch discriminator is used to enhance facial detail generation. Experimental results for the CelebA and CelebA-HQ datasets indicated that PFTANet achieved pluralistic and visually realistic face inpainting.

Abstract Radiation therapy is the standard treatment for early stage Nasopharyngeal cancer (NPC). Thus, accurate delineation of target volumes at risk in NPC is important. While manual delineation is time‐consuming and labour‐intensive process and also leads to significant inter‐ and intra‐practitioner variability. Thus, computer‐aided segmentation algorithm is required. However, segmentation task is not trivial due to large variations (e.g., shape and size) of nasopharynx structure across subjects. Moreover, extreme foreground and background class imbalance in NPC segmentation remains challenge. In this paper, we propose a threedimensional densely connected convolutional neural network with multi‐scale feature pyramids for NPC segmentation. We adapt the densely connected convolutional block into a new structure via adding feature pyramids. The concatenated pyramid feature carries multi‐scale and hierarchical semantic information which is effective for segmenting different size of tumors and perceiving hierarchical context information. To address the foreground and background imbalance problem, we propose an enhanced version of focal loss. It prevents the large number of negative voxels far from boundaries from overwhelming the segmentation algorithm. We validated the proposed method on 120 clinical subjects. Experimental results demonstrate that our approach out‐performed state‐of‐the‐art methods and human experts.

Deep learning has been widely applied into image inpainting. However, traditional image processing methods (i.e., patch-based and diffusion-based methods) generally fail to produce visually natural contents and semantically reasonable structures due to ineffectively processing the high-level semantic information of images. To solve the problem, we propose a stacked generator networks assisted by patch discriminator for image inpainting by multistage. In the proposed method, our generator network mainly consists of three-layer stacked encoder-decoder architecture, which could fuse different level feature information and achieve image inpainting via a coarse-to-fine hierarchical representation. Meanwhile, we split the masked image into different patches in each layer, which could effectively enlarge the receptive field and extract more useful features of images. Moreover, the patch discriminator is introduced to judge the patches of inpainting image are real or fake. In this way, our network can effectively utilize the semantic information to complete a fine result. Furthermore, both perceptual loss and style loss are used to improve the inpainting results in verse. Experimental results on Places2 and Paris StreetView illustrate that our approach could generate high-quality inpainting results, and our method is more effective than the existing image inpainting methods.

For iris recognition in non-cooperative environments, iris segmentation has been regarded as the first most important challenge still open to the biometric community, affecting all downstream tasks from normalization to recognition. In recent years, deep learning technologies have gained significant popularity among various computer vision tasks and also been introduced in iris biometrics, especially iris segmentation. To investigate recent developments and attract more interest of researchers in the iris segmentation method, we organized the 2021 NIR Iris Challenge Evaluation in Non-cooperative Environments: Segmentation and Localization (NIR-ISL 2021) at the 2021 International Joint Conference on Biometrics (IJCB 2021). The challenge was used as a public platform to assess the performance of iris segmentation and localization methods on Asian and African NIR iris images captured in non-cooperative environments. The three best-performing entries achieved solid and satisfactory iris segmentation and localization results in most cases, and their code and models have been made publicly available for reproducibility research.

Outlier detection has drawn significant interest from both academia and industry, such as network intrusion detection. Most existing methods implicitly or explicitly rely on distances in Euclidean space. However, the Euclidean distance may be incapable of measuring the similarity among high-dimensional data due to the curse of dimensionality, thus leading to inferior performance in practice. This paper presents an innovative approach for outlier detection from the view of meaningful structure scores. If two points have similar features, the difference between their structural scores is small and vice versa. The scores are calculated by measuring the variance of angles weighted by data representation, which takes the global data structure into the measurement. Thus, it could consistently rank more similar points. Compared with existing methods, our structural scores could be better to reflect the characteristics of data in a high-dimensional space. The proposed method consistently ranks more similar points. Experiments on synthetic and several real-world datasets have demonstrated the effectiveness and efficiency of our proposed methods.

For patients with dilated cardiomyopathy (DCM), fast and accurate diagnosis is important to save lives. MRI is a non-invasive, effective medical imaging method that allows doctors to diagnose DCM. However, manual and semi-automatic segmentation is subjective, non-reproducible and time-consuming task. In this paper, a new attention-based convolutional encoder-decoder network is proposed to automatically segment my-ocardium in DCM, which assisting the doctor to quickly diagnose. In the proposed method, the attention mechanism module is used, which is able to fully highlight useful features that facilitate segmentation while suppress useless features that are not conducive to segmentation. Combining with the multi-scale convolution, our encoder-decoder network can accurately segment the my-ocardium in DCM. We verified our approach on 1155 myocardial MRI. Our network achieves the most advanced segmentation performance on the cardiac DCM dataset. Experiment results demonstrate the effectiveness of the proposed method.

For dilated cardiomyopathy (DCM) patients, immediate emergency diagnosis and treatment are critical for life saving and later recovery.T1 mapping is a non-invasive and effective diagnostic imaging approach to detect DCM.However, it is a demanding and time-consuming approach.In this paper, we propose an attention-based network structure, which can automatically identify DCM patients in a speedy manner to prioritize their treatment.In the proposed method, we adopt attention modules to generate attention-aware features.Inside each attention module, a bottom-up top-down feed-forward structure is used to unfold the feed-forward and feed-back attention processes into a single feed-forward process.It allows the network to focus more on determining useful information about the current output that is significant in the input data.Moreover, inspired by the residual network idea, we make full use of the characteristics of the original data.Combined residual block, we design down-residual modules for classification tasks.It consists of seven convolution layers and three layers of residual blocks.Our network achieves the most advanced recognition performance on cardiac datasets.We evaluated our approach on CMR(cardiac magnetic resonance) T1 mapping images with lower PSNR(peak signal to noise ratio), and the results demonstrate that our architecture outperforms previous approaches.

Image generation based on the generative adversarial network (GAN) has been widely used in the field of computer vision.It helps generate images similar to the given data by learning their distribution.However, in many tasks, training on small datasets of scenes may lead to mode collapse, such that the generated images are often blurred and almost the same.To solve this problem, we propose a generative adversarial network with transfer learning for mode collapse called TL-GAN.Owing to the size of the training dataset, we introduce transfer learning (VGG pre-training network) to extract more useful features from the underlying pixels and add them to the discriminator, which can be used to calculate the distance between samples, and to provide the discriminator with a new training target.The discriminator thus learns the best features that can distinguish between real data and generated data using the proposed model.This also enhances the learning capability of the generator, which learn further about the distribution of real data.Meanwhile, generator can produce new images more realistic.The results of experiments show that the TL-GAN can guarantee the diversity of samples.A qualitative comparison with several prevalent methods confirmed its effectiveness.

Outlier detection is one of the most frequently demanded task for optimizing results. Distance-based methods are a popular approach. They require no prior assumptions about the data generating distribution and are uncomplicated to implement. However, related methods have different parameters that are difficult to determine such that the identification results are generally unstable. Presenting related techniques without sacrificing stability is a challenging task. In this paper, we propose a new distance-based method that depends on the data structure to detect such points. In the proposed method, a global binary tree is constructed and the local distance score of a point is calculated to evaluate to what degree the observation is an outlier. The greater the value of the distance score, the more likely the point is an outlier point. Unlike typical distance-based methods, our algorithm has good scalability. Even when the dimension of the data points increases, the performance of our algorithm does not diminish. To reduce extra parameters, the top-p ranked points can be identified as outliers. Experimental results on synthetic and real-world datasets demonstrate the effectiveness and stability of our method.

Detecting boundary points (including outliers) is often more interesting than detecting normal observations, since they represent valid, interesting, and potentially valuable patterns. Since data representation can uncover the intrinsic data structure, we present an efficient representation-based method for detecting such points, which are generally located around the margin of densely distributed data, such as a cluster. For each point, the negative components in its representation generally correspond to the boundary points among its affine combination of points. In the presented method, the reverse unreachability of a point is proposed to evaluate to what degree this observation is a boundary point. The reverse unreachability can be calculated by counting the number of zero and negative components in the representation. The reverse unreachability explicitly takes into account the global data structure and reveals the disconnectivity between a data point and other points. This paper reveals that the reverse unreachability of points with lower density has a higher score. Note that the score of reverse unreachability of an outlier is greater than that of a boundary point. The top- ranked points can thus be identified as outliers. The greater the value of the reverse unreachability, the more likely the point is a boundary point. Compared with related methods, our method better reflects the characteristics of the data, and simultaneously detects outliers and boundary points regardless of their distribution and the dimensionality of the space. Experimental results obtained for a number of synthetic and real-world data sets demonstrate the effectiveness and efficiency of our method.

This paper presents an angle and density-based data preprocessing method. It can be used to simultaneously identify outliers, boundary points and center points of clusters. Boundary points and outliers are generally located around the margin of densely distributed data such as a cluster. Detecting boundary points and outliers is often more interesting than detecting normal observations since they represent valid, interesting, and potentially valuable patterns. We propose an approach based on the idea that boundary points are characterized by a lower local density and by a smaller angle variance than that of their neighbors. Outliers, boundary points and inner points can be identified by both angle and density measurements. Experimental results obtained for several test cases demonstrate the effectiveness and efficiency of our method.