Low-dose computed tomography (LDCT) reduces health risks from radiation exposure but introduces imaging noise and artifacts. While numerous studies have employed deep learning for LDCT image denoising, the field continues to face significant challenges. Recent advancements have seen diffusion models applied to overcome issues of over-smoothness and unstable training inherent in prior deep learning approaches. However, the diffusion models face challenges in direct practical applications due to the extensive sampling steps, significant inference time required, and the need for hard-to-obtain paired data during training. To address these difficulties, this paper introduces a self-supervised diffusion model with edge prior for unpaired LDCT denoising. This method enables denoising within a lower-dimensional space, reducing computational complexity. Our proposed approach enhances denoised image clarity by applying prior edge constraints to compressed encodings; it employs a noise-conditioned encoding strategy to facilitate self-supervised image training, enabling the method to be applicable to unpaired CT data; and it utilizes compressed LDCT encoding as intermediate sampling results during the inference process, thereby accelerating sampling and reducing the time required for inference, making the method more real-time capable. Extensive validation across multiple datasets demonstrates that our method achieves competitive performance against state-of-the-art approaches in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual quality (LPIPS), while maintaining a practically acceptable inference time.

Transformers have been widely used in image dehazing tasks due to their powerful self-attention mechanism for capturing long-range dependencies. However, directly applying Transformers often leads to coarse details during image reconstruction, especially in complex real-world hazy scenarios. To address this problem, we propose a novel Hybrid Attention Encoder (HAE). Specifically, a channel-attention-based convolution block is integrated into the Swin-Transformer architecture. This design enhances the local features at each position through an overlapping block-wise spatial attention mechanism while leveraging the advantages of channel attention in global information processing to strengthen the network’s representation capability. Moreover, to adapt to various complex hazy environments, a high-quality codebook prior encapsulating the color and texture knowledge of high-resolution clear scenes is introduced. We also propose a more flexible Binary Matching Mechanism (BMM) to better align the codebook prior with the network, further unlocking the potential of the model. Extensive experiments demonstrate that our method consistently outperforms the second-best methods by a margin of 8% to 19% across multiple metrics on the RTTS and URHI datasets. The source code has been released at https://github.com/HanyuZheng25/HADehzeNet .

Automated ultrasound (US) image analysis is hindered by challenges stemming from low resolution, noise, and non-uniform grayscale distribution, which compromise image quality. While many existing studies address these issues using super-resolution (SR) techniques, they often focus exclusively on SR without considering downstream tasks or tailoring to the unique characteristics of US images. In this work, we propose ZSG-Net, a zero-shot super-resolution-guided network, designed to bridge the gap between US image quality enhancement and its benefits in segmentation and classification. First, we introduce a zero-shot self-supervised cycle generative adversarial network (ZSCycle-GAN), tailored to the unique characteristics of US images, to perform SR while preserving critical structural details. Unlike conventional SR methods that focus solely on image enhancement, ZSCycle-GAN is designed to optimize downstream tasks. Second, we adopt a zero-shot self-supervised learning strategy, eliminating the reliance on labeled data and addressing the scarcity of annotated medical imaging datasets. Third, we incorporate a random image degradation (RID) strategy to expand the degradation space for clinical US images, enabling robust learning of diverse quality variations. Extensive experiments on three US image datasets validate the effectiveness of the proposed model. Results demonstrate superior performance in segmentation and classification tasks compared to existing approaches, underscoring the potential of our method to improve US image analysis in clinical settings.

Myocardial infarction (MI) is a leading cause of death worldwide. Late gadolinium enhancement (LGE) and T2-weighted cardiac magnetic resonance (CMR) imaging can respectively identify scarring and edema areas, both of which are essential for MI risk stratification and prognosis assessment. Although combining complementary information from multi-sequence CMR is useful, acquiring these sequences can be time-consuming and prohibitive, e.g., due to the administration of contrast agents. Cine CMR is a rapid and contrast-free imaging technique that can visualize both motion and structural abnormalities of the myocardium induced by acute MI. Therefore, we present a new end-to-end deep neural network, referred to as CineMyoPS, to segment myocardial pathologies, i.e., scars and edema, solely from cine CMR images. Specifically, CineMyoPS extracts both motion and anatomy features associated with MI. Given the interdependence between these features, we design a consistency loss (resembling the co-training strategy) to facilitate their joint learning. Furthermore, we propose a time-series aggregation strategy to integrate MI-related features across the cardiac cycle, thereby enhancing segmentation accuracy for myocardial pathologies. Experimental results on a multi-center dataset demonstrate that CineMyoPS achieves promising performance in myocardial pathology segmentation, motion estimation, and anatomy segmentation.

Neural networks have found widespread application in medical image registration, although they typically assume access to the entire training dataset during training. In clinical scenarios, medical images of various anatomical targets, such as the heart, brain, and liver, may be obtained successively with advancements in imaging technologies and diagnostic procedures. The accuracy of registration on a new target may degrade over time, as the registration models become outdated due to domain shifts occurring at unpredictable intervals. In this study, we introduce a deep registration model based on continual learning to mitigate the issue of catastrophic forgetting during training with continuous data streams. To enable continuous network training, we propose a dynamic memory system based on a density-based clustering algorithm to retain representative samples from the data stream. Training the registration network on these representative samples enhances its generalization capabilities to accommodate new targets within the data stream. We evaluated our approach using the CHAOS dataset, which comprises multiple targets, such as the liver, left kidney, and spleen, to simulate a data stream. The experimental findings illustrate that the proposed continual registration network achieves comparable performance to a model trained with full data visibility.

Unsupervised domain adaptation(UDA) aims to mitigate the performance drop of models tested on the target domain, due to the domain shift from the target to sources. Most UDA segmentation methods focus on the scenario of solely single source domain. However, in practical situations data with gold standard could be available from multiple sources (domains), and the multi-source training data could provide more information for knowledge transfer. How to utilize them to achieve better domain adaptation yet remains to be further explored. This work investigates multi-source UDA and proposes a new framework for medical image segmentation. Firstly, we employ a multi-level adversarial learning scheme to adapt features at different levels between each of the source domains and the target, to improve the segmentation performance. Then, we propose a multi-model consistency loss to transfer the learned multi-source knowledge to the target domain simultaneously. Finally, we validated the proposed framework on two applications, i.e., multi-modality cardiac segmentation and cross-modality liver segmentation. The results showed our method delivered promising performance and compared favorably to state-of-the-art approaches.

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

Myocardial pathology segmentation (MyoPS) is critical for the risk stratification and treatment planning of myocardial infarction (MI). Multi-sequence cardiac magnetic resonance (MS-CMR) images can provide valuable information. For instance, balanced steady-state free precession cine sequences present clear anatomical boundaries, while late gadolinium enhancement and T2-weighted CMR sequences visualize myocardial scar and edema of MI, respectively. Existing methods usually fuse anatomical and pathological information from different CMR sequences for MyoPS, but assume that these images have been spatially aligned. However, MS-CMR images are usually unaligned due to the respiratory motions in clinical practices, which poses additional challenges for MyoPS. This work presents an automatic MyoPS framework for unaligned MS-CMR images. Specifically, we design a combined computing model for simultaneous image registration and information fusion, which aggregates multi-sequence features into a common space to extract anatomical structures (i.e., myocardium). Consequently, we can highlight the informative regions in the common space via the extracted myocardium to improve MyoPS performance, considering the spatial relationship between myocardial pathologies and myocardium. Experiments on a private MS-CMR dataset and a public dataset from the MYOPS2020 challenge show that our framework could achieve promising performance for fully automatic MyoPS.

Face super resolution (FSR) is a sub-field of super resolution (SR), which is to reconstruct low resolution (LR) face image into high resolution (HR) face image. Recently, the FSR methods based on face prior have been proved to be effective in FSR on higher upscaling factors. However, existing prior guided methods mostly adopt supervised prior extraction models trained with labels. The performance of supervised prior extraction method mainly depends on the accuracy of label so that the implicit informations of data are not fully utilized. And in practical application, the label acquisition work is routine and laborious. Therefore, to solve these problems, this paper proposes a novel contrastive learning (CL) based FSR method, which is based on the iterative collaboration of image reconstruction network and contrastive learning network. In each iteration, the reconstruction network uses the priors generated by the contrastive learning network to assist the image reconstruction and generates higher-quality SR images. Then, the SR image will feed into contrastive learning network to obtain more accurate prior. In addition, a new contrastive learning constraint function is designed to extract the representation of the augmented facial image as a prior by analysing the principal component information of the image. Quantitative and qualitative experimental results show that the proposed method is superior to the most advanced FSR method in high-quality face images super resolution reconstruction.

Road digitizing is a labor-intensive task. For reducing labor cost, this letter presents an effective semiautomatic road delineation method based on geodesic distance field (GDF) and piecewise polygon fitting. The main components of the approach are optimal circle calculation, soft road center kernel density estimation (KDE), fast GDF generation, and piecewise polygon fitting model. First, the adaptive circular template is proposed to automatically measure the road width. Next, the soft road center kernel density is estimated for fast GDF generation, which supports the extraction of road centerline between two adjacent seeds. Finally, piecewise polygon fitting is used to refine the road centerline. Extensive experiments demonstrate that the proposed algorithm is efficient and robust in road delineation. The proposed approach takes almost the same time to extract any length of road segment given fixed image size, and no hyperparameters needs to be set.

Road segmentation methods based on deep neural networks have achieved great success in recent years, but creating accurate pixel-wise training labels is still a boring and expensive task, especially for large-scale high-resolution remote sensing images (HRSIs). Inspired by the stacked hourglass model for human joints detection, we propose a weakly supervised road segmentation method using point annotations in this article. First, we design a patch-based deep convolutional neural network (DCNN) model for road seeds and background points detection and train the model using point annotations. Then, in the process of road segmentation, the DCNN model detects a series of road and background points that are used to train a Support Vector Machine Classifier (SVC) for classifying each pixel into road or nonroad. According to the local geometry of road and the inaccurate classification of SVC, a multiscale and multidirection Gabor filter (MMGF) is put forward to estimate the road potential. Finally, the active contour model based on local binary fitting energy (LBF-Snake) is introduced to extract the road regions from the inhomogeneous road potential. Qualitative and quantitative comparisons show that our method achieves results close to the fully supervised semantic methods without considering the annotation cost and outperforms them given a fixed budget.

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image; and the transformed atlas labels can be combined to generate target segmentation via label fusion schemes. Many conventional MAS methods employed the atlases from the same modality as the target image. However, the number of atlases with the same modality may be limited or even missing in many clinical applications. Besides, conventional MAS methods suffer from the computational burden of registration or label fusion procedures. In this work, we design a novel cross-modality MAS framework, which uses available atlases from a certain modality to segment a target image from another modality. To boost the computational efficiency of the framework, both the image registration and label fusion are achieved by well-designed deep neural networks. For the atlas-to-target image registration, we propose a bi-directional registration network (BiRegNet), which can efficiently align images from different modalities. For the label fusion, we design a similarity estimation network (SimNet), which estimates the fusion weight of each atlas by measuring its similarity to the target image. SimNet can learn multi-scale information for similarity estimation to improve the performance of label fusion. The proposed framework was evaluated by the left ventricle and liver segmentation tasks on the MM-WHS and CHAOS datasets, respectively. Results have shown that the framework is effective for cross-modality MAS in both registration and label fusion https://github.com/NanYoMy/cmmas.

Abstract The reliable and automatic segmentation of pulmonary lobes in computed tomography scans is an important pre‐condition for the diagnosis, assessment, and treatment of lung diseases. However, due to the incomplete lobar structures and morphological changes caused by diseases, the lobe segmentation still encounters great challenges. Recently, convolution neural network has exerted a tremendous impact on medical image analysis. Nevertheless, the basic convolution operations mainly obtain local features that are insufficient for accurate lobe segmentation. The idea that the global features are equally crucial especially when lesions appear is considered. Here, a dual‐attention V‐network named DAV‐Net for pulmonary lobe segmentation is proposed. First, a novel dual‐attention module to capture global contextual information and model the semantic dependencies in spatial and channel dimensions is introduced. Second, a progressive output scheme is used to avoid the vanishing gradient phenomenon and obtain relatively effective features in hidden layers. Finally, an improved combo loss is devised to address input and output lobe imbalance problem during training and inference. In the evaluation using the LUNA16 dataset and our in‐house dataset, the proposed DAV‐Net obtains Dice similarity coefficients of 0.947 and 0.934, respectively; these values are superior to those obtained by existing methods.

The task of person re-identification (re-ID) is to find the same pedestrian across non-overlapping camera views. Generally, the performance of person re-ID can be affected by background clutter. However, existing segmentation algorithms cannot obtain perfect foreground masks to cover the background information clearly. In addition, if the background is completely removed, some discriminative ID-related cues (i.e., backpack or companion) may be lost. In this article, we design a dual-stream network consisting of a Provider Stream (P-Stream) and a Receiver Stream (R-Stream). The R-Stream performs an a priori optimization operation on foreground information. The P-Stream acts as a pusher to guide the R-Stream to concentrate on foreground information and some useful ID-related cues in the background. The proposed dual-stream network can make full use of the a priori optimization and guided-learning strategy to learn encouraging foreground information and some useful ID-related information in the background. Our method achieves Rank-1 accuracy of 95.4% on Market-1501, 89.0% on DukeMTMC-reID, 78.9% on CUHK03 (labeled), and 75.4% on CUHK03 (detected), outperforming state-of-the-art methods.

Recently, Generative Adversarial Network (GAN) has been adopted to improve person re-identification (person re-ID) performance through data augmentation. However, directly leveraging generated data to train a re-ID model may easily lead to over-fitting issue on these extra data and decrease the generalisability of model to learn true ID-related features from real data. Inspired by the previous approach which assigns multi-pseudo labels on the generated data to reduce the risk of over-fitting, we propose to take sparse regularization into consideration. We attempt to further improve the performance of current re-ID models by using the unlabeled generated data. The proposed Sparse Regularized Multi-Pseudo Label (SRMpL) can effectively prevent the over-fitting issue when some larger weights are assigned to the generated data. Our experiments are carried out on two publicly available person re-ID datasets (e.g., Market-1501 and DukeMTMC-reID). Compared with existing unlabeled generated data re-ID solutions, our approach achieves competitive performance. Two classical re-ID models are used to verify our sparse regularization label on generated data, i.e., an ID-embedding network and a two-stream network.

The depth maps obtained by the consumer-level sensors are always noisy in the low-resolution (LR) domain. Existing methods for the guided depth super-resolution, which are based on the pre-defined local and global models, perform well in general cases (e.g., joint bilateral filter and Markov random field). However, such model-based methods may fail to describe the potential relationship between RGB-D image pairs. To solve this problem, this paper proposes a data-driven approach based on the deep convolutional neural network with global and local residual learning. It progressively upsamples the LR depth map guided by the high-resolution intensity image in multiple scales. A global residual learning is adopted to learn the difference between the ground truth and the coarsely upsampled depth map, and the local residual learning is introduced in each scale-dependent reconstruction sub-network. This scheme can restore the depth structure from coarse to fine via multi-scale frequency synthesis. In addition, batch normalization layers are used to improve the performance of depth map denoising. Our method is evaluated in noise-free and noisy cases. A comprehensive comparison against 17 state-of-the-art methods is carried out. The experimental results show that the proposed method has faster convergence speed as well as improved performances based on the qualitative and quantitative evaluations.

The task of person re-identification (re-id) is to find the same pedestrian across non-overlapping cameras. Normally, the performance of person re-id can be affected by background clutters. However, existing segmentation algorithms are hard to obtain perfect foreground person images. To effectively leverage the body (foreground) cue, and in the meantime pay attention to discriminative information in the background (e.g., companion or vehicle), we propose to use a cross-learning strategy to take both foreground and other discriminative information into account. In addition, since currently existing foreground segmentation result always involves noise, we use Label Smoothing Regularization (LSR) to strengthen the generalization capability during our learning process. In experiments, we pick up two state-of-the-art person re-id methods to verify the effectiveness of our proposed cross-learning strategy. Our experiments are carried out on two publicly available person re-id datasets. Obvious performance improvements can be observed on both datasets.

Although deep convolutional neural networks (DCNN) show significant improvement for single depth map (SD) super-resolution (SR) over the traditional counterparts, most SDSR DCNNs do not reuse the hierarchical features for depth map SR resulting in blurred high-resolution (HR) depth maps. They always stack convolutional layers to make network deeper and wider. In addition, most SDSR networks generate HR depth maps at a single level, which is not suitable for large up-sampling factors. To solve these problems, we present pyramid-structured depth map super-resolution based on deep dense-residual network. Specially, our networks are made up of dense residual blocks that use densely connected layers and residual learning to model the mapping between high-frequency residuals and low-resolution (LR) depth map. Furthermore, based on the pyramid structure, our network can progressively generate depth maps of various levels by taking advantages of features from different levels. The proposed network adopts a deep supervision scheme to reduce the difficulty of model training and further improve the performance. The proposed method is evaluated on Middlebury datasets which shows improved performance compared with 6 state-of-the-art methods.