Enhancing the precision of segmenting coronary atherosclerotic plaques from CT Angiography (CTA) images is pivotal for advanced Coronary Atherosclerosis Analysis (CAA), which distinctively relies on the analysis of vessel cross-section images reconstructed via Curved Planar Reformation. This task presents significant challenges due to the indistinct boundaries and structures of plaques and blood vessels, leading to the inadequate performance of current deep learning models, compounded by the inherent difficulty in annotating such complex data. To address these issues, we propose a novel dual-consistency semi-supervised framework that integrates Intra-frame Topological Consistency (ITC) and Cross-frame Topological Consistency (CTC) to leverage labeled and unlabeled data. ITC employs a dual-task network for simultaneous segmentation mask and Skeleton-aware Distance Transform (SDT) prediction, achieving similar prediction of topology structure through consistency constraint without additional annotations. Meanwhile, CTC utilizes an unsupervised estimator for analyzing pixel flow between skeletons and boundaries of adjacent frames, ensuring spatial continuity. Experiments on two CTA datasets show that our method surpasses existing semi-supervised methods and approaches the performance of supervised methods on CAA. In addition, our method also performs better than other methods on the ACDC dataset, demonstrating its generalization.

Deep neural networks (DNNs) face substantial challenges in Long-Tail Visual Recognition (LTVR) due to the inherent class imbalances in real-world data distributions. The Mixture of Experts (MoE) framework has emerged as a promising approach to addressing these issues. However, in MoE systems, experts are typically trained to optimize a collective objective, often neglecting the individual optimality of each expert. This individual optimality usually contributes to the overall performance, as the goals of different experts are not mutually exclusive. We propose the Independent and Collaborative Learning (ICL) framework to optimize each expert independently while ensuring global optimality. First, Diverse Optimization Learning (DOL) is introduced to enhance expert diversity and individual performance. Then, we conceptualize experts as parallel circuit branches and introduce Competition and Collaboration Learning (CoL). Competition Learning amplifies the gradients of better-performing experts to preserve individual optimality, and Collaboration Learning encourages collaboration through mutual distillation to enhance optimal knowledge sharing. ICL achieves state-of-the-art accuracy in experiments on CIFAR-100/10-LT, ImageNet-LT, and iNaturalist 2018, respectively. Our code is available at https://github.com/PolarisLight/ICL.

Coronary artery extraction is a crucial prerequisite for computer-aided diagnosis of coronary artery disease. Accurately extracting the complete coronary tree remains challenging due to several factors, including presence of thin distal vessels, tortuous topological structures, and insufficient contrast. These issues often result in over-segmentation and under-segmentation in current segmentation methods. To address these challenges, we propose a topology-preserving three-stage framework for fully-connected coronary artery extraction. This framework includes vessel segmentation, centerline reconnection, and missing vessel reconstruction. First, we introduce a new centerline enhanced loss in the segmentation process. Second, for the broken vessel segments, we further propose a regularized walk algorithm to integrate distance, probabilities predicted by a centerline classifier, and directional cosine similarity, for reconnecting the centerlines. Third, we apply implicit neural representation and implicit modeling, to reconstruct the geometric model of the missing vessels. Experimental results show that our proposed framework outperforms existing methods, achieving Dice scores of 88.53% and 85.07%, with Hausdorff Distances (HD) of 1.07 mm and 1.63 mm on ASOCA and PDSCA datasets, respectively. Code will be available at https://github.com/YH-Qiu/CorSegRec.

Accurate segmentation of lesions plays a critical role in medical image analysis and diagnosis. Traditional segmentation approaches that rely solely on visual features often struggle with the inherent uncertainty in lesion distribution and size. To address these issues, we propose STPNet, a Scale-aware Text Prompt Network that leverages vision-language modeling to enhance medical image segmentation. Our approach utilizes multi-scale textual descriptions to guide lesion localization and employs retrieval-segmentation joint learning to bridge the semantic gap between visual and linguistic modalities. Crucially, STPNet retrieves relevant textual information from a specialized medical text repository during training, eliminating the need for text input during inference while retaining the benefits of cross-modal learning. We evaluate STPNet on three datasets: COVID-Xray, COVID-CT, and Kvasir-SEG. Experimental results show that our vision-language approach outperforms state-of-the-art segmentation methods, demonstrating the effectiveness of incorporating textual semantic knowledge into medical image analysis. The code has been made publicly on https://github.com/HUANGLIZI/STPNet.

Self-supervised pre-training followed by fine-tuning is a potent paradigm for few-shot learning, leveraging extensive unlabeled data with remarkable efficacy. Current self-supervised methods often lean towards Vision Transformers (ViTs) rather than CNN-Transformer hybrid architectures, which generally demonstrate superior performance. However, this reliance on ViTs can lead to poor perception of local features by the model. The challenge lies in designing a suitable proxy task for hybrid architectures like CNN-Transformers, which have significant structural differences. Additionally, the current organization of CNN-Transformer hybrid backbones is often sequential, hindering collaboration during pre-training and the acquisition of robust representations. To address these issues, we propose Self-Supervised Collaborative CNN-Transformer (S2CCT) for few-shot medical image segmentation. This framework introduces three innovative designs: (1) a composite proxy task based on image masking and image super-resolution tailored for CNN-Transformer hybrid architectures, enabling the backbone to acquire robust representations during pre-training that can be transferred to downstream tasks; (2) a parallel CNN-Transformer architecture that better attends to multi-scale features in images, making it more suitable for dense prediction tasks like image segmentation; (3) a sparse and dense feature fusion module to enhance collaboration between the two encoders. Experiments demonstrate that S2CCT outperforms previous state-of-the-art methods on two public medical image segmentation benchmarks, i.e., ACDC and KiTs19. The code and pretrained models will be released soon.

Semi-supervised learning is becoming increasingly popular in medical image segmentation because of its ability to exploit large amounts of unlabelled data to extract additional information. However, most existing semi-supervised segmentation methods focus only on extracting information from unlabelled data, ignoring the potential of labelled data to further improve model performance. In this paper, we propose a new framework for Intra-Sample Cross Reconstruction Networks (ICR-Net) that utilises labelled data to help the network extract information from unlabelled data, thereby guiding the network’s regularisation learning. Our method contains two modules: Intra-Sample Cross Reconstruction (ICR) module and Synergistic Consistency Constraints (SCC) module. The ICR module processes the labelled data features in a more fine-grained manner, thus enabling the network to learn and capture the key patterns and features in the inputs more efficiently, and the SCC guides the network’s regularised learning by formulating additional model regularisations. Experiments on the LA dataset and the pancreas dataset show that our proposed framework is more effective than current state-of-the-art methods in medical image segmentation tasks.

In semi-supervised medical image segmentation, the scarcity of labeled data makes models prone to learning bias, causing persistent errors in certain regions and eventual over-fitting, significantly impacting segmentation performance. These problematic regions, termed difficult areas, are inadequately addressed by existing methods. To address this, We propose the Difficulty Perception-Processing Heterogeneous Network (DPP-Net). It guides the model in accurately perceiving and rectifying difficult areas, overcoming learning bias. Specifically, we introduce the Global Mutual Perception (GMP) to establish a comprehensive information perception channel between sample data, enabling a more holistic and accurate perception of difficult areas. The Difficulty-Aware Rectification (DAR) structure ensures continuous monitoring of difficult areas during training, allowing for timely adjustments to errors. Additionally, the Adaptive Competitive Pseudo-Label (ACP) Augmentation strategy enhances pseudo-labels through adaptive confidence competition. Experimental results on two different medical image databases (CT and MRI) demonstrate that our approach outperforms several state-of-the-art methods.

Recent methods for neural surface representation and rendering have shown the ability to reconstruct surface but require lengthy training. The latest approach employs hash encoding to expedite training, but neglects reconstruction accuracy, leading to lower surface quality. To address this problem, we propose a fast neural surface reconstruction method called FFnsr, which incorporates two optimizations. First, FFnsr uses a predetermined linear growth function to replace the learnable parameter in previous volume rendering methods. This allows the network to concentrate on the rough shape during the initial stages and refine the details in the later stages. Simultaneously, FFnsr proposes a regularization scheme using second-order derivatives in the direction of the gradient, which can stabilize the training of the network and obtain a flatter surface. Our experimental results on DTU datasets demonstrate that FFnsr can generate high-quality and robust reconstruction results while maintaining high speed.

Three-dimensional reconstruction from multiview images is considered as a longstanding problem in computer vision and graphics. In order to achieve high-fidelity geometry and appearance of 3-D scenes, this article proposes a novel geometric object learning method for multiview reconstruction withfuzzy set theory. We establish anew neural 3D reconstruction theoretical framecalled neural fuzzy geometric representation (NeuFG), which is a special type of implicit representation of geometric scene that only takes value in [0, 1]. NeuFG is essentially a volume image, and thus can be visualized directly with the conventional volume rendering technique. Extensive experiments on two public datasets, i.e., DTU and BlendedMVS, show that our method has the ability of accurately reconstructing complex shapes with vivid geometric details, without the requirement of mask supervision. Both qualitative and quantitative comparisons demonstrate that the proposed method has superior performance over the state-of-the-art neural scene representation methods. The code will be released on GitHub soon.

Deep learning has been widely used in medical image segmentation and other aspects. However, the performance of existing medical image segmentation models has been limited by the challenge of obtaining sufficient high-quality labeled data due to the prohibitive data annotation cost. To alleviate this limitation, we propose a new text-augmented medical image segmentation model LViT (Language meets Vision Transformer). In our LViT model, medical text annotation is incorporated to compensate for the quality deficiency in image data. In addition, the text information can guide to generate pseudo labels of improved quality in the semi-supervised learning. We also propose an Exponential Pseudo label Iteration mechanism (EPI) to help the Pixel-Level Attention Module (PLAM) preserve local image features in semi-supervised LViT setting. In our model, LV (Language-Vision) loss is designed to supervise the training of unlabeled images using text information directly. For evaluation, we construct three multimodal medical segmentation datasets (image + text) containing X-rays and CT images. Experimental results show that our proposed LViT has superior segmentation performance in both fully-supervised and semi-supervised setting. The code and datasets are available at https://github.com/HUANGLIZI/LViT.

Most recent scribble-supervised segmentation methods commonly adopt a CNN framework with an encoder-decoder architecture. Despite its multiple benefits, this framework generally can only capture small-range feature dependency for the convolutional layer with the local receptive field, which makes it difficult to learn global shape information from the limited information provided by scribble annotations. To address this issue, this paper proposes a new CNN-Transformer hybrid solution for scribble-supervised medical image segmentation called ScribFormer. The proposed ScribFormer model has a triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer branch, and an attention-guided class activation map (ACAM) branch. Specifically, the CNN branch collaborates with the Transformer branch to fuse the local features learned from CNN with the global representations obtained from Transformer, which can effectively overcome limitations of existing scribble-supervised segmentation methods. Furthermore, the ACAM branch assists in unifying the shallow convolution features and the deep convolution features to improve model's performance further. Extensive experiments on two public datasets and one private dataset show that our ScribFormer has superior performance over the state-of-the-art scribble-supervised segmentation methods, and achieves even better results than the fully-supervised segmentation methods. The code is released at https://github.com/HUANGLIZI/ScribFormer.

Detecting pneumonia, especially coronavirus disease 2019 (COVID-19), from chest X-ray (CXR) images is one of the most effective ways for disease diagnosis and patient triage. The application of deep neural networks (DNNs) for CXR image classification is limited due to the small sample size of the well-curated data. To tackle this problem, this article proposes a distance transformation-based deep forest framework with hybrid-feature fusion (DTDF-HFF) for accurate CXR image classification. In our proposed method, hybrid features of CXR images are extracted in two ways: hand-crafted feature extraction and multigrained scanning. Different types of features are fed into different classifiers in the same layer of the deep forest (DF), and the prediction vector obtained at each layer is transformed to form distance vector based on a self-adaptive scheme. The distance vectors obtained by different classifiers are fused and concatenated with the original features, then input into the corresponding classifier at the next layer. The cascade grows until DTDF-HFF can no longer gain benefits from the new layer. We compare the proposed method with other methods on the public CXR datasets, and the experimental results show that the proposed method can achieve state-of-the art (SOTA) performance. The code will be made publicly available at https://github.com/hongqq/DTDF-HFF.

Segmentation of COVID-19 lesions can assist physicians in better diagnosis and treatment of COVID-19. However, there are few relevant studies due to the lack of detailed information and high-quality annotation in the COVID-19 dataset. To solve the above problem, we propose C2FVL, a Coarse-to-Fine segmentation framework via Vision-Language alignment to merge text information containing the number of lesions and specific locations of image information. Introducing text information allows the network to achieve better prediction results on challenging datasets. We conduct extensive experiments on two COVID-19 datasets, including chest X-ray and CT, and the results demonstrate that our proposed method outperforms other state-of-the-art segmentation methods.

Medical image segmentation plays a critical role in clinical decision-making, treatment planning, and disease monitoring. However, accurate segmentation of medical images is challenging due to several factors, such as the lack of high-quality annotation, imaging noise, and anatomical differences across patients. In addition, there is still a considerable gap in performance between the existing label-efficient methods and fully-supervised methods. To address the above challenges, we propose ScribbleVC, a novel framework for scribble-supervised medical image segmentation that leverages vision and class embeddings via the multimodal information enhancement mechanism. In addition, ScribbleVC uniformly utilizes the CNN features and Transformer features to achieve better visual feature extraction. The proposed method combines a scribble-based approach with a segmentation network and a class-embedding module to produce accurate segmentation masks. We evaluate ScribbleVC on three benchmark datasets and compare it with state-of-the-art methods. The experimental results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency. The datasets and code are released on GitHub.

Medial Axis Transform (MAT) has been recently adopted as the acceleration structure of broad-phase collision detection. Compared to traditional BVH-based methods, MAT can provide a high-fidelity volumetric approximation of 3D complex objects, resulting in higher collision culling efficiency. However, due to MAT’s non-hierarchical structure, it may be outperformed in collision-light scenarios because several cullings at the top level of a BVH may take a large number of cullings with MAT. We propose a collision detection method that combines MAT and BVH to address the above problem. Our technique efficiently culls collisions between dynamic and static objects. Experimental results show that our method has higher culling efficiency than pure BVH or MAT methods.

Abstract 3D point cloud segmentation is a non‐trivial problem due to its irregular, sparse, and unordered data structure. Existing methods only consider structural relationships of a 3D point and its spatial neighbours. However, the inner‐point interactions and long‐distance context of a 3D point cloud have been less investigated. In this study, we propose an effective plug‐and‐play module called the Long Short‐Distance Topologically Modelled (LSDTM) Graph Convolutional Neural Network (GCNN) to learn the underlying structure of 3D point clouds. Specifically, we introduce the concept of subgraph to model the contextual‐point relationships within a short distance. Then the proposed topology can be reconstructed by recursive aggregation of subgraphs, and importantly, to propagate the contextual scope to a long range. The proposed LSDTM can parse the point cloud data with maximisation of preserving the geometric structure and contextual structure, and the topological graph can be trained end‐to‐end through a seamlessly integrated GCNN. We provide a case study of triple‐layer ternary topology and experimental results on ShapeNetPart, Stanford 3D Indoor Semantics and ScanNet datasets, indicating a significant improvement on the task of 3D point cloud segmentation and validating the effectiveness of our research.

Cardiac medical image segmentation plays an important role in the diagnosis and clinical treatment of cardiovascular diseases. However, due to the variability of cardiac anatomy and the ambiguity between cardiac substructures, it is still difficult to quickly segment the entire heart from medical images. Most of the current researches utilize neural network structure to perform whole heart segmentation. Although good segmentation accuracy has been achieved, it usually requires a long training time. This paper aims to build a new whole heart segmentation model based on Deep Forest, called Multi-Resolution Deep Forest Framework(MRDFF), which performs segmentation through two stages. In the first stage, the heart region is located by rough binary classification, and similarity screening is used to reduce redundancy. The second stage subdivides the heart substructures based on the results of the first stage and uses multi-scale fusion to achieve high segmentation accuracy. The experimental results conducted on the public data set MM-WHS show that under the same training data and configuration, our model can be trained in only 4.5 hours, which is about 1/2 of the training time of neural network models, and can reach the accuracy not lower than neural network models, which shows the feasibility and efficiency of our model. The code will be made publicly available at https://github.com/xufeixf/MRDFF.

Recently, coronary heart disease has attracted more and more attention, where segmentation and analysis for vascular lumen contour are helpful for treatment. And intravascular optical coherence tomography (IVOCT) images are used to display lumen shapes in clinic. Thus, an automatic segmentation method for IVOCT lumen contour is necessary to reduce the doctors' workload while ensuring diagnostic accuracy. In this paper, we proposed a deep residual segmentation network of multi-scale feature fusion based on attention mechanism (RSM-Network, Residual Squeezed Multi-Scale Network) to segment the lumen contour in IVOCT images. Firstly, three different data augmentation methods including mirror level turnover, rotation and vertical flip are considered to expand the training set. Then in the proposed RSM-Network, U-Net is contained as the main body, considering its characteristic of accepting input images with any sizes. Meanwhile, the combination of residual network and attention mechanism is applied to improve the ability of global feature extraction and solve the vanishing gradient problem. Moreover, the pyramid feature extraction structure is introduced to enhance the learning ability for multi-scale features. Finally, in order to increase the matching degree between the actual output and expected output, the cross entropy loss function is also used. A series of metrics are presented to evaluate the performance of our proposed network and the experimental results demonstrate that the proposed RSM-Network can learn the contour details better, contributing to strong robustness and accuracy for IVOCT lumen contour segmentation.

The human eye's state of motion and content of interest can express people's cognitive status and emotional status based on their situation. When observing the surrounding things, the human eyes make different eye movements according to the observed objects which reflects human's attention and interest. In this paper, we capture and analyze patterns of human eye-gaze behavior and head motion and classify them into different categories. Besides, we compute and train the eye-object movement attention model and eye-object feature preference model based on different peoples' eye-gaze behaviors by using machine learning algorithms. These models are used to predict humans' object of interest and the interaction intention according to people's real-time situation. Furthermore, the eye-gaze behavior and head motion patterns can be used as a modality of non-verbal information in the computing of human emotional states based on the PAD affective computing model. Our methodology analyzes human emotion and cognition status from the aspect of eye-gaze behavior and head motion, understands the cognitive information that human eyes can express, and effectively improves the efficiency of human-computer interaction in different circumstances.

The classification of multi-class microarray datasets is a hard task because of the small samples size in each class and the heavy overlaps among classes. To effectively solve these problems, we propose a novel Error Correcting Output Code (ECOC) algorithm by Enhance Class Separability related Data Complexity measures during encoding process, named as ECOCECS. In this algorithm, two nearest neighbor related DC measures are deployed to extract the intrinsic overlapping information from microarray data. Our ECOC algorithm aims to search an optimal class split scheme by minimizing these measures. The class splitting process ends when each class is separated from others, and then the class assignment scheme is mapped as a coding matrix. Experiments are carried out on seven microarray datasets, and results demonstrate the effectiveness and robustness of our method in comparison with four state-of-art ECOC methods. In short, our work shows that it is promising to apply the DC theory to ECOC framework.

Vasculature geometry reconstruction from volumetric medical data is a crucial task in the development of computer guided minimally invasive vascular surgery systems. In this paper, a technique for reconstructing the geometry of vasculatures using bivariate implicit splines is developed. With the proposed technique, an implicit geometry representation of the vascular tree can be accurately constructed based on the voxels extracted directly from the surface of a certain vascular structure in a given volumetric medical dataset. Experimental results show that the geometric representation built using our method can faithfully represent the morphology and topology of vascular structures. In addition, both the qualitative and the quantitative validations have been performed to show that the reconstructed vessel geometry is of high accuracy and smoothness. An virtual angioscopy system has been implemented to indicate one of the strengths of our proposed method.