Infrared and visible image fusion aims to generate one image with comprehensive information. It can maintain rich texture characteristics and thermal information. However, for existing image fusion methods, the fused images either sacrifice the salience of thermal targets and the richness of textures or introduce the interference of useless information like artifacts. To alleviate these problems, an effective cross-modal coordinate attention network for infrared and visible image fusion called CCAFusion is proposed in this paper. To fully integrate complementary features, the cross-modal image fusion strategy based on coordinate attention is designed, which consists of the feature-awareness fusion module and the feature-enhancement fusion module. Moreover, a multiscale skip connection-based network is employed to obtain multiscale features in the infrared image and the visible image, which can fully utilize the multi-level information in the fusion process. To reduce the discrepancy between the fused image and the input images, a multiple constrained loss function including the base loss and the auxiliary loss is developed to adjust the gray-level distribution and ensure the harmonious coexistence of structure and intensity in fused images, thereby preventing the pollution of useless information like artifacts. Extensive experiments conducted on widely used datasets demonstrate that our CCAFusion achieves superior performance over state-of-the-art image fusion methods in both qualitative evaluation and quantitative measurement. Furthermore, the application to salient object detection reveals the potential of our CCAFusion for high-level vision tasks, which can effectively boost the detection performance.

Multimodal magnetic resonance imaging (MRI) contains complementary information in anatomical and functional images that help the accurate diagnosis and treatment evaluation of lung cancers. However, effectively exploiting the complementary information in chest MRI images remains challenging due to the lack of rigorous registration. In this paper, a novel method is proposed that can effectively exploit the complementary information in weakly paired images for accurate tumor segmentation, namely coco-attention mechanism. Coco-attention module consists of two parts: the multi-modal co-attention (MultiCo-attn) and the multi-level coordinate attention (MultiCord-attn). The former aims to obtain tumor-aware deep features for accurate tumor localization, and the latter aims to highlight tumor area for more precise segmentation. Specifically, the MultiCo-attn extracts complementary information from multimodal high-dimensional semantic features using a bidirectional algorithm to generate attention maps focused on tumor region, and then uses the attention maps to enhance the feature representations. The MultiCord-attn leverages multi-level feature information to highlight tumor regions by adjusting the weight of each point in the feature. We evaluate the proposed method on lung tumor segmentation with a clinical dataset of 90 chest MRI scans of non-small cell lung cancer (NSCLC). The results show that the proposed method is effective for tumor segmentation in weakly paired images and achieves significant improvement (p < 0.005) over several commonly used multimodal segmentation methods. Furthermore, the ablation experiment results confirm the effectiveness and interpretability of the proposed coco-attention module.

Person re-identification (re-ID) aims to match images of the same person in different camera views. The general convolutional neural network has the problem of insufficient ability to discriminate specific targets, resulting in limited feature representation when directly applied in person re-ID task. In this paper, a person re-ID baseline model based on attention block neural architecture search is proposed. Since the attention mechanism is helpful to improve the feature expression ability of the network, attention blocks are automatically searched and added to ResNet-50 model, which aims to find the most suitable model structure for person re-ID task. Besides, in order to integrate information between the samples of same identity, an intra-class self-distillation loss is introduced according to the idea of knowledge integration. Experiments on two popular datasets confirm the effectiveness of our baseline. The code has been released in https://github.com/Nicholasxin/Attention-NASReID.

Unsupervised cross domain (UCD) person re-identification (re-ID) aims to apply a model trained on a labeled source domain to an unlabeled target domain. It faces huge challenges as the identities have no overlap between these two domains. At present, most UCD person re-ID methods perform "supervised learning" by assigning pseudo labels to the target domain, which leads to poor re-ID performance due to the pseudo label noise. To address this problem, a multi-loss optimization learning (MLOL) model is proposed for UCD person re-ID. In addition to using the information of clustering pseudo labels from the perspective of supervised learning, two losses are designed from the view of similarity exploration and adversarial learning to optimize the model. Specifically, in order to alleviate the erroneous guidance brought by the clustering error to the model, a ranking-average-based triplet loss learning and a neighbor-consistency-based loss learning are developed. Combining these losses to optimize the model results in a deep exploration of the intra-domain relation within the target domain. The proposed model is evaluated on three popular person re-ID datasets, Market-1501, DukeMTMC-reID, and MSMT17. Experimental results show that our model outperforms the state-of-the-art UCD re-ID methods with a clear advantage.

Image registration of lung dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) is challenging because the rapid changes in intensity lead to non-realistic deformations of intensity-based registration methods. To address this problem, we propose a novel landmark-based registration framework by incorporating landmark information into a group-wise registration. Robust principal component analysis is used to separate motion from intensity changes caused by a contrast agent. Landmark pairs are detected on the resulting motion components and then incorporated into an intensity-based registration through a constraint term. To reduce the negative effect of inaccurate landmark pairs on registration, an adaptive weighting landmark constraint is proposed. The method for calculating landmark weights is based on an assumption that the displacement of a good matched landmark is consistent with those of its neighbors. The proposed method was tested on 20 clinical lung DCE-MRI image series. Both visual inspection and quantitative assessment are used for the evaluation. Experimental results show that the proposed method effectively reduces the non-realistic deformations in registration and improves the registration performance compared with several state-of-the-art registration methods.

Accurate breast mass segmentation of automated breast ultrasound (ABUS) images plays a crucial role in 3D breast reconstruction which can assist radiologists in surgery planning. Although the convolutional neural network has great potential for breast mass segmentation due to the remarkable progress of deep learning, the lack of annotated data limits the performance of deep CNNs. In this article, we present an uncertainty aware temporal ensembling (UATE) model for semi-supervised ABUS mass segmentation. Specifically, a temporal ensembling segmentation (TEs) model is designed to segment breast mass using a few labeled images and a large number of unlabeled images. Considering the network output contains correct predictions and unreliable predictions, equally treating each prediction in pseudo label update and loss calculation may degrade the network performance. To alleviate this problem, the uncertainty map is estimated for each image. Then an adaptive ensembling momentum map and an uncertainty aware unsupervised loss are designed and integrated with TEs model. The effectiveness of the proposed UATE model is mainly verified on an ABUS dataset of 107 patients with 170 volumes, including 13382 2D labeled slices. The Jaccard index (JI), Dice similarity coefficient (DSC), pixel-wise accuracy (AC) and Hausdorff distance (HD) of the proposed method on testing set are 63.65%, 74.25%, 99.21% and 3.81mm respectively. Experimental results demonstrate that our semi-supervised method outperforms the fully supervised method, and get a promising result compared with existing semi-supervised methods.

Surface nodularity is an important image biomarker for cirrhosis. In this study, we explore how the location of the liver boundary curve may influence the liver surface nodularity (LSN). Based on computed tomography images of 7 patients with a normal liver and 9 patients with cirrhosis, we quantitatively estimate the LSN of the boundary curves selected from different locations of the liver. By repeating the estimation for different boundary curves, the difference in the variation of the LSN between the normal liver and cirrhosis is investigated. Receiver operating characteristic (ROC) analysis is used to assess the classification performance between the normal and cirrhotic livers. Results show that the LSN of the normal liver is significantly different from that of the cirrhosis, that the variation pattern of the LSN is different if the boundary curve is selected from different locations of the liver, and that the LSN leads to accurate classification between the normal and cirrhotic livers with the area under the ROC curve of 0.97- 0.98. We conclude that the LSN can be used to classify cirrhosis from the normal liver and studies are on-going to validate the conclusion.

Prostate cancers were segmented directly in T2-weighted images in some studies of computer-aided detection (CAD). These methods don't consider the differences between lesion and non-lesion region in T2-weighted images, so some lesions are not easy to be detected. In this paper, to consider the differences between lesion and non-lesion region, some features extraction is proposed by using graph spectral method. First, whole prostate is extracted. And then, some statistics are computed in the region of prostate to find differences between cancer and noncancer in prostate. The statistics are mapped into m dimensional Euclidean space by using graph spectral method to detect prostate cancers. Experiments show features with m dimensional Euclidean space by using graph spectral method can find some differences between lesion and non-lesion region.