Coronary artery extraction is a crucial prerequisite for computer-aided diagnosis of coronary artery disease. Accurately extracting the complete coronary tree remains challenging due to several factors, including presence of thin distal vessels, tortuous topological structures, and insufficient contrast. These issues often result in over-segmentation and under-segmentation in current segmentation methods. To address these challenges, we propose a topology-preserving three-stage framework for fully-connected coronary artery extraction. This framework includes vessel segmentation, centerline reconnection, and missing vessel reconstruction. First, we introduce a new centerline enhanced loss in the segmentation process. Second, for the broken vessel segments, we further propose a regularized walk algorithm to integrate distance, probabilities predicted by a centerline classifier, and directional cosine similarity, for reconnecting the centerlines. Third, we apply implicit neural representation and implicit modeling, to reconstruct the geometric model of the missing vessels. Experimental results show that our proposed framework outperforms existing methods, achieving Dice scores of 88.53% and 85.07%, with Hausdorff Distances (HD) of 1.07 mm and 1.63 mm on ASOCA and PDSCA datasets, respectively. Code will be available at https://github.com/YH-Qiu/CorSegRec.

For surgical planning and intra-operation imaging, CT reconstruction using X-ray images can potentially be an important alternative when CT imaging is not available or not feasible. In this paper, we aim to use biplanar X-rays to reconstruct a 3D CT image, because biplanar X-rays convey richer information than single-view X-rays and are more commonly used by surgeons. Different from previous studies in which the two X-ray views were treated indifferently when fusing the cross-view data, we propose a novel attention-informed coarse-to-fine cross-view fusion method to combine the features extracted from the orthogonal biplanar views. This method consists of a view attention alignment sub-module and a fine-distillation sub-module that are designed to work together to highlight the unique or complementary information from each of the views. Experiments have demonstrated the superiority of our proposed method over the SOTA methods.

The large-scale whole-slide images (WSIs) facilitate the learning-based computational pathology methods. However, the gigapixel size of WSIs makes it hard to train a conventional model directly. Current approaches typically adopt multiple-instance learning (MIL) to tackle this problem. Among them, MIL combined with graph convolutional network (GCN) is a significant branch, where the sampled patches are regarded as the graph nodes to further discover their correlations. However, it is difficult to build correspondence across patches from different WSIs. Therefore, most methods have to perform non-ordered node pooling to generate the bag-level representation. Direct non-ordered pooling will lose much structural and contextual information, such as patch distribution and heterogeneous patterns, which is critical for WSI representation. In this paper, we propose a hierarchical global-to-local clustering strategy to build a Node-Aligned GCN (NAGCN) to represent WSI with rich local structural information as well as global distribution. We first deploy a global clustering operation based on the instance features in the dataset to build the correspondence across different WSIs. Then, we perform a local clustering-based sampling strategy to select typical instances belonging to each cluster within the WSI. Finally, we employ the graph convolution to obtain the representation. Since our graph construction strategy ensures the alignment among different WSIs, WSI-level representation can be easily generated and used for the subsequent classification. The experiment results on two cancer subtype classification datasets demonstrate our method achieves better performance compared with the state-of-the-art methods.

In this paper, we focus on generating photo-realistic images from given text descriptions. Current methods first generate an initial image and then progressively refine it to a high-resolution one. These methods typically indiscriminately refine all granularity features output from the previous stage. However, the ability to express different granularity features in each stage is not consistent, and it is difficult to express precise semantics by further refining the features with poor quality generated in the previous stage. Current methods cannot refine different granularity features independently, resulting in that it is challenging to clearly express all factors of semantics in generated image, and some features even become worse. To address this issue, we propose a Hierarchical Disentangled Representations Generative Adversarial Networks (HDR-GAN) to generate photo-realistic images by explicitly disentangling and individually modeling the factors of semantics in the image. HDR-GAN introduces a novel component called multi-granularity feature disentangled encoder to represent image information comprehensively through explicitly disentangling multi-granularity features including pose, shape and texture. Moreover, we develop a novel Multi-granularity Feature Refinement (MFR) containing a Coarse-grained Feature Refinement (CFR) model and a Fine-grained Feature Refinement (FFR) model. CFR utilizes coarse-grained disentangled representations (e.g., pose and shape) to clarify category information, while FFR employs fine-grained disentangled representations (e.g., texture) to reflect instance-level details. Extensive experiments on two well-studied and publicly available datasets (i.e., CUB-200 and CLEVR-SV) demonstrate the rationality and superiority of our method.

Assessing the location and extent of lesions caused by chronic stroke is critical for medical diagnosis, surgical planning, and prognosis. In recent years, with the rapid development of 2D and 3D convolutional neural networks (CNN), the encoder-decoder structure has shown great potential in the field of medical image segmentation. However, the 2D CNN ignores the 3D information of medical images, while the 3D CNN suffers from high computational resource demands. This paper proposes a new architecture called dimension-fusion-UNet (D-UNet), which combines 2D and 3D convolution innovatively in the encoding stage. The proposed architecture achieves a better segmentation performance than 2D networks, while requiring significantly less computation time in comparison to 3D networks. Furthermore, to alleviate the data imbalance issue between positive and negative samples for the network training, we propose a new loss function called Enhance Mixing Loss (EML). This function adds a weighted focal coefficient and combines two traditional loss functions. The proposed method has been tested on the ATLAS dataset and compared to three state-of-the-art methods. The results demonstrate that the proposed method achieves the best quality performance in terms of DSC = 0.5349 ± 0.2763 and precision = 0.6331 ± 0.295).

Accurate segmentation of anatomical brain structures is crucial for many neuroimaging applications, e.g., early brain development studies and the study of imaging biomarkers of neurodegenerative diseases. Although multi-atlas segmentation (MAS) has achieved many successes in the medical imaging area, this approach encounters limitations in segmenting anatomical structures associated with poor image contrast. To address this issue, we propose a new MAS method that uses a hypergraph learning framework to model the complex subject-within and subject-to-atlas image voxel relationships and propagate the label on the atlas image to the target subject image. To alleviate the low-image contrast issue, we propose two strategies equipped with our hypergraph learning framework. First, we use a hierarchical strategy that exploits high-level context features for hypergraph construction. Because the context features are computed on the tentatively estimated probability maps, we can ultimately turn the hypergraph learning into a hierarchical model. Second, instead of only propagating the labels from the atlas images to the target subject image, we use a dynamic label propagation strategy that can gradually use increasing reliably identified labels from the subject image to aid in predicting the labels on the difficult-to-label subject image voxels. Compared with the state-of-the-art label fusion methods, our results show that the hierarchical hypergraph learning framework can substantially improve the robustness and accuracy in the segmentation of anatomical brain structures with low image contrast from magnetic resonance (MR) images.

Foreground detection is fundamental in surveillance video analysis and meaningful toward object tracking and higher level tasks, such as anomaly detection and activity analysis. Nevertheless, existing methods are still limited in accurately detecting the foreground due to the complex scene settings. To robustly handle the diverse background variations and foreground challenges, this paper proposes a Background REpresentation approach With Dictionary Learning and Historical Pixel Maintenance (BREW-DLHPM). Specifically, a dictionary learning problem is formulated at the frame level to adaptively represent the background signals with the varied structure information captured, while a pixel-level maintenance is exploited to grasp the dynamic nature of historical information under the help of the learned background. The simultaneous utilization of dictionary learning and historical pixel maintenance facilitates the accurate description of the background and thus guides a wise foreground detection decision. The proposed BREW-DLHPM has been evaluated on the prestigious change detection challenge data set against 11 state-of-the-art foreground detection approaches and encouraging performances have been achieved by our method.

This paper presents a novel structure gradient and texture decor relating regularization (SGTD) for image decomposition. The motivation of the idea is under the assumption that the structure gradient and texture components should be properly decor related for a successful decomposition. The proposed model consists of the data fidelity term, total variation regularization and the SGTD regularization. An augmented Lagrangian method is proposed to address this optimization issue, by first transforming the unconstrained problem to an equivalent constrained problem and then applying an alternating direction method to iteratively solve the sub problems. Experimental results demonstrate that the proposed method presents better or comparable performance as state-of-the-art methods do.

Dictionary learning based sparse modeling has been increasingly recognized as providing high performance in the restoration of noisy images. Although a number of dictionary learning algorithms have been developed, most of them attack this learning problem in its primal form, with little effort being devoted to exploring the advantage of solving this problem in a dual space. In this paper, a novel Fenchel duality based dictionary learning (FD-DL) algorithm has been proposed for the restoration of noise-corrupted images. With the restricted attention to the additive white Gaussian noise, the sparse image representation is formulated as an 2-1 minimization problem, whose dual formulation is constructed using a generalization of Fenchel’s duality theorem and solved under the augmented Lagrangian framework. The proposed algorithm has been compared with four state-of-the-art algorithms, including the local pixel grouping-principal component analysis, method of optimal directions, K-singular value decomposition, and beta process factor analysis, on grayscale natural images. Our results demonstrate that the FD-DL algorithm can effectively improve the image quality and its noisy image restoration ability is comparable or even superior to the abilities of the other four widely-used algorithms.

Video summarization enables convenient and efficient management of large volume of visual data. However, most existing summarization approaches are based on either the pixel domain information or conventional video compression standards. As the most recent and popular international video coding standard, H.264/AVC adopts a number of advanced techniques and brings not only opportunities but also challenges to video summarization. In this paper, we propose a real-time image storyboard generation algorithm for H.264/AVC compressed videos by using both compressed domain and pixel domain information jointly and adaptively. This algorithm extracts compressed domain information for visual content representation, video structuring and candidate representative frame selection. By fusing both compressed domain and pixel domain information, the redundancy in the candidate representative frames is further reduced. Our experimental results show that the proposed algorithm can efficiently produce image storyboards conforming to human interpretation of the essential content in generic videos.

With increased use of H.264/AVC in various applications including video surveillance systems, feature extraction and knowledge representation in compressed domain are becoming attractive. A real-time H.264/AVC compressed domain moving object segmentation and tracking algorithm for surveillance videos is proposed in this paper. This algorithm consists of moving object detection, bounding box matching, spatiotemporal merge and split reasoning and trajectory smoothing, with major innovation in incorporating the information provided by the prediction modes into the framework of motion detection and trajectory construction. The experimental results on both indoor and outdoor surveillance videos demonstrate that the adaptive use of the information from motion vectors, DCT coefficients and prediction modes can substantially improve the performance of moving object segmentation and tracking.