Radiologists possess diverse training and clinical experiences, leading to variations in the segmentation annotations of lung nodules and resulting in segmentation uncertainty. Conventional methods typically select a single annotation as the learning target or attempt to learn a latent space comprising multiple annotations. However, these approaches fail to leverage the valuable information inherent in the consensus and disagreements among the multiple annotations. In this paper, we propose an Uncertainty-Aware Attention Mechanism (UAAM) that utilizes consensus and disagreements among multiple annotations to facilitate better segmentation. To this end, we introduce the Multi-Confidence Mask (MCM), which combines a Low-Confidence (LC) Mask and a High-Confidence (HC) Mask. The LC mask indicates regions with low segmentation confidence, where radiologists may have different segmentation choices. Following UAAM, we further design an Uncertainty-Guide Multi-Confidence Segmentation Network (UGMCS-Net), which contains three modules: a Feature Extracting Module that captures a general feature of a lung nodule, an Uncertainty-Aware Module that produces three features for the annotations' union, intersection, and annotation set, and an Intersection-Union Constraining Module that uses distances between the three features to balance the predictions of final segmentation and MCM. To comprehensively demonstrate the performance of our method, we propose a Complex-Nodule Validation on LIDC-IDRI, which tests UGMCS-Net's segmentation performance on lung nodules that are difficult to segment using common methods. Experimental results demonstrate that our method can significantly improve the segmentation performance on nodules that are difficult to segment using conventional methods.

Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event and the clinical decision of treatment planning. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features. In this paper, we propose a novel domain adaptation (DA) method with two components to address these problems. The first component is a stochastic class-balanced boosting sampling strategy that overcomes the imbalanced learning problem and improves the classification performance on poorly-predicted classes. The second component is a representation learning that guarantees three properties: 1) domain-transferability by prototype triplet loss, 2) discriminant by conditional maximum mean discrepancy loss, and 3) completeness by multi-view reconstruction loss. Particularly, we propose a domain translator and align the heterogeneous data to the estimated class prototypes (i.e., class centers) in a hyper-sphere manifold. Experiments on cross-site severity assessment of COVID-19 from CT images show that the proposed method can effectively tackle the imbalanced learning problem and outperform recent DA approaches.

In this paper, we propose a novel feature decoupling method to tackle two critical problems in the lung nodule segmentation task: (i) ambiguity of nodule boundary leads to the imprecise segmentation boundary and (ii) the high false positive rate of segmentation result. Our motivation is that an accurate segmentation network needs explicitly modeling the nodule boundary and texture information, and suppressing the noise information. To do so, a novel Deep Feature Decoupling Module (DFDM) is proposed to decouple the nodule boundary, noise, and texture information from the original feature maps. The decoupled boundary and texture information is used to benefit the segmentation, and the noise information is removed from the input features to reduce the false positive rate. The proposed DFDM consists of three parallel branches, including Boundary Sensitive Branch (BSB), Noise Removal Branch (NRB), and Texture Preserving Branch (TPB) to decouple the mentioned three information, respectively. In particular, we design our BSB with a novel architecture to effectively capture the boundary information of lung nodules. We apply the proposed DFDM to the U-Net architecture and achieve convincing segmentation results on the LIDC–IDRI dataset. Code and models are available at https://github.com/chinichenw/DFDM.

Pulmonary nodule detection and segmentation are the necessary successively steps in lung cancer screening with low-dose computed tomography (CT) scans. However, the state-of-the-art models focus on solving tasks separately, thereby ignore the correlation between each task. Besides, most nodule detectors adopt anchor-based method falling to achieve good performance in low FPs per scan. To overcome those barriers, we present a novel multi-task 3D convolutional network (DeepNodule) for simultaneous nodule detection and segmentation in a shared-and-fined manner. Meanwhile, we utilize the center-point of the predicted segmentation masks to refine the bounding box coordinate and get a more precise nodule location. Furthermore, we design a 3D Gated Channel Transformation convolutional attention block for learning nodule features better. Experiments conducted on LUNA16 dataset demonstrates that DeepNodule obtains competitive performance, with the sensitivity of nodule candidate detection achieving 92.0%, and the accuracy of nodule segmentation reaching 80.04%.

Many medical images domains suffer from inherent ambiguities. A feasible approach to resolve the ambiguity of lung nodule in the segmentation task is to learn a distribution over segmentations based on a given 2D lung nodule image. Whereas lung nodule with 3D structure contains dense 3D spatial information, which is obviously helpful for resolving the ambiguity of lung nodule, but so far no one has studied it. To this end we propose a probabilistic generative segmentation model consisting of a V-Net and a conditional variational autoencoder. The proposed model obtains the 3D spatial information of lung nodule with V-Net to learn a density model over segmentations. It is capable of efficiently producing multiple plausible semantic lung nodule segmentation hypotheses to assist radiologists in making further diagnosis to resolve the present ambiguity. We evaluate our method on publicly available LIDC-IDRI dataset and achieves a new state-of-the-art result with 0.231±0.005 in $D_{GED}^2$. This result demonstrates the effectiveness and importance of leveraging the 3D spatial information of lung nodule for such problems. Code is available at: https://github.com/jiangjiangxiaolong/PV-Net.

Many deep learning methods have been proposed for the diagnosis of COVID-19 since the global pandemic. However, few studies have focused on the disease course classification of COVID-19, which is crucial for radiologists to determine treatment plans. This paper proposes a Multi-Modal Fusion Cascade (MMFC) framework for this task, which can make the most of multi-modal information, including CT image and bio-information (laboratory examination, clinical characterization, etc.). The proposed framework consists of two parts: Bio-Visual Feature Learning Module (BFL) and Joint Decision Module (JD). Firstly, BFL learns the discriminative visual features from the mediastinal window with the assistance of bio-information. According to the official Treatment Protocol of China, the bio-information is chosen and helps the BFL better extract the images’ bio-visual features and then obtained a disease course classification result based on CT images. Secondly, JD uses bio-information again and fuses the confidence of BFL’s result to make the joint decision. Experimental results show that our framework significantly improves accuracy and sensitivity compared to the baseline.

The important cues for a realistic lung nodule synthesis include the diversity in shape and background, controllability of semantic feature levels, and overall CT image quality. To incorporate these cues as the multiple learning targets, we introduce the Multi-Target Co-Guided Adversarial Mechanism, which utilizes the foreground and background mask to guide nodule shape and lung tissues, takes advantage of the CT lung and mediastinal window as the guidance of spiculation and texture control, respectively. Further, we propose a Multi-Target Co-Guided Synthesizing Network with a joint loss function to realize the co-guidance of image generation and semantic feature learning. The proposed network contains a Mask-Guided Generative Adversarial Sub-Network (MGGAN) and a Window-Guided Semantic Learning Sub-Network (WGSLN). The MGGAN generates the initial synthesis using the mask combined with the foreground and background masks, guiding the generation of nodule shape and background tissues. Meanwhile, the WGSLN controls the semantic features and refines the synthesis quality by transforming the initial synthesis into the CT lung and mediastinal window, and performing the spiculation and texture learning simultaneously. We validated our method using the quantitative analysis of authenticity under the Fréchet Inception Score, and the results show its state-of-the-art performance. We also evaluated our method as a data augmentation method to predict malignancy level on the LIDC-IDRI database, and the results show that the accuracy of VGG-16 is improved by 5.6%. The experimental results confirm the effectiveness of the proposed method.

Accurate segmentation for lung nodules in lung computed tomography (CT) scans plays a key role in the early diagnosis of lung cancer. Many existing methods, especially U-Net, have made significant progress in lung nodule segmentation. However, due to the complex shapes of lung nodules and the similarity of visual characteristics between nodules and lung tissues, an accurate segmentation with low false positives of lung nodules is still a challenging problem. Considering the fact that both boundary and texture information of lung nodules are important for obtaining an accurate segmentation result, we propose a novel Mask and Texture-driven Generative Adversarial Network (MTGAN) with a joint multi-scale L1 loss for lung nodule segmentation, which takes full advantages of U-Net and adversarial training. The proposed MTGAN leverages adversarial learning strategy guided by the boundary and texture information of lung nodules to generate more accurate segmentation results with lesser false positives. We validate our model with the LIDC-IDRI dataset, and experimental results show that our method achieves excellent segmentation results for a variety of lung nodules, especially for juxtapleural nodules and low-dense nodules. Without any bells and whistles, the proposed MTGAN achieves significant segmentation performance with the Dice similarity coefficient (DSC) of 85.24% on the LIDC-IDRI dataset.

Computer-Aided Diagnosis (CAD) systems for lung nodule diagnosis based on deep learning have attracted much attention in recent years. However, most existing methods ignore the relationships between the segmentation and classification tasks, which leads to unstable performances. To address this problem, we propose a novel multi-task framework, which can provide lung nodule segmentation mask, malignancy prediction, and medical features for interpretable diagnosis at the same time. Our framework mainly contains two sub-network: (1) Multi-Channel Segmentation Sub-network (MSN) for lung nodule segmentation, and (2) Joint Classification Sub-network (JCN) for interpretable lung nodule diagnosis. In the proposed framework, we use U-Net down-sampling processes for extracting low-level deep learning features, which are shared by two sub-networks. The JCN forces the down-sampling processes to learn better low-level deep features, which lead to a better construct of segmentation masks. Meanwhile, two additional channels constructed by OTSU and super-pixel (SLIC) methods, are utilized as the guideline of the feature extraction. The proposed framework takes advantages of deep learning methods and classical methods, which can significantly improve the performances of all tasks. We evaluate the proposed framework on public dataset LIDC-IDRI. Our framework achieves a promising Dice score of 86.43% in segmentation, 87.07% in malignancy level prediction, and convincing results in interpretable medical feature predictions.

We propose a conceptually simple framework for fast COVID-19 screening in 3D chest CT images. The framework can efficiently predict whether or not a CT scan contains pneumonia while simultaneously identifying pneumonia types between COVID-19 and Interstitial Lung Disease (ILD) caused by other viruses. In the proposed method, two 3D-ResNets are coupled together into a single model for the two above-mentioned tasks via a novel prior-attention strategy. We extend residual learning with the proposed prior-attention mechanism and design a new so-called prior-attention residual learning (PARL) block. The model can be easily built by stacking the PARL blocks and trained end-to-end using multi-task losses. More specifically, one 3D-ResNet branch is trained as a binary classifier using lung images with and without pneumonia so that it can highlight the lesion areas within the lungs. Simultaneously, inside the PARL blocks, prior-attention maps are generated from this branch and used to guide another branch to learn more discriminative representations for the pneumonia-type classification. Experimental results demonstrate that the proposed framework can significantly improve the performance of COVID-19 screening. Compared to other methods, it achieves a state-of-the-art result. Moreover, the proposed method can be easily extended to other similar clinical applications such as computer-aided detection and diagnosis of pulmonary nodules in CT images, glaucoma lesions in Retina fundus images, etc.