Colonoscopy video generation delivers dynamic, information-rich data critical for diagnosing intestinal diseases, particularly in data-scarce scenarios. High-quality video generation demands temporal consistency and precise control over clinical attributes, but faces challenges from irregular intestinal structures, diverse disease representations, and various imaging modalities. To this end, we propose ColoDiff, a diffusion-based framework that generates dynamic-consistent and content-aware colonoscopy videos, aiming to alleviate data shortage and assist clinical analysis. At the inter-frame level, our TimeStream module decouples temporal dependency from video sequences through a cross-frame tokenization mechanism, enabling intricate dynamic modeling despite irregular intestinal structures. At the intra-frame level, our Content-Aware module incorporates noise-injected embeddings and learnable prototypes to realize precise control over clinical attributes, breaking through the coarse guidance of diffusion models. Additionally, ColoDiff employs a non-Markovian sampling strategy that cuts steps by over 90% for real-time generation. ColoDiff is evaluated across three public datasets and one hospital database, based on both generation metrics and downstream tasks including disease diagnosis, modality discrimination, bowel preparation scoring, and lesion segmentation. Extensive experiments show ColoDiff generates videos with smooth transitions and rich dynamics. ColoDiff also produces customized contents tailored for diverse tasks, e.g., colitis, polyps, and adenomas for diagnosis. Incorporating synthetic videos into training promotes discriminative representation learning and improves diagnosis accuracy by 7.1%. ColoDiff presents an effort in controllable colonoscopy video generation, revealing the potential of synthetic videos in complementing authentic representation and mitigating data scarcity in clinical settings.

Congestive heart failure (CHF) is a chronic heart condition with high morbidity and mortality, manifesting as persistent abnormal rhythms and electrophysiological disturbances across multiple cardiac regions. Early and accurate detection of CHF using electrocardiograms (ECGs) is essential for clinical management. However, existing algorithms, typically tailored for short-term single-lead recordings, fail to capture multi-scale and multi-view cardiac abnormalities. Additionally, the pronounced class imbalance, with normal samples predominating, substantially compromises the sensitivity of mediocre models to CHF cases. To address these challenges, this article proposes a novel dual-view class-specific broad aggregation network (DCBA-Net) capable of extracting and integrating multi-scale temporal dynamics from long-term ECGs of limb lead II and chest lead V1. Specifically, an ECGNeXt architecture with multi-kernel depthwise convolutions (DWConvs) and channel attention mechanisms (CAMs) as the backbone is first constructed to extract both local and global disease-related features from the two ECG leads. Subsequently, in the information integration stage, a new objective function is designed to enhance interlead interactions by simultaneously enforcing view discrepancy and target consistency constraints. Furthermore, this function assigns class-specific coefficients to elevate the prominence of CHF cases, thereby alleviating the class imbalance problem. Finally, DCBA-Net aggregates consensus and complementary decisions on both leads for improved CHF detection. Experimental results on two publicly available databases show that DCBA-Net achieves 100% across all metrics under the intrapatient paradigm, with an accuracy of 99.4%, an$F1$-score of 98.98% and a G-mean of 99.1% under the interpatient paradigm, outperforming advanced results and demonstrating its immense potential as an auxiliary CHF diagnostic tool.

Colorectal cancer plays a dominant role in cancer-related deaths, primarily due to the absence of obvious early-stage symptoms. Whole-stage colorectal disease diagnosis is crucial for assessing lesion evolution and determining treatment plans. However, locality difference and disease progression lead to intra-class disparities and inter-class similarities for colorectal lesion representation. In addition, interpretable algorithms explaining the lesion progression are still lacking, making the prediction process a "black box". In this paper, we propose IPNet, a dual-branch interpretable network with progressive loss for whole-stage colorectal disease diagnosis. The dual-branch architecture captures unbiased features representing diverse localities to suppress intra-class variation. The progressive loss function considers inter-class relationship, using prior knowledge of disease evolution to guide classification. Furthermore, a novel Grain-CAM is designed to interpret IPNet by visualizing pixel-wise attention maps from shallow to deep layers, providing regions semantically related to IPNet's progressive classification. We conducted whole-stage diagnosis on two image modalities, i.e., colorectal lesion classification on 129,893 endoscopic optical images and rectal tumor T-staging on 11,072 endoscopic ultrasound images. IPNet is shown to surpass other state-of-the-art algorithms, accordingly achieving an accuracy of 93.15% and 89.62%. Especially, it establishes effective decision boundaries for challenges like polyp vs. adenoma and T2 vs. T3. The results demonstrate an explainable attempt for colorectal lesion classification at a whole-stage level, and rectal tumor T-staging by endoscopic ultrasound is also unprecedentedly explored. IPNet is expected to be further applied, assisting physicians in whole-stage disease diagnosis and enhancing diagnostic interpretability.

The three-dimensional (3D) reconstruction of the coronary artery from angiographic images is crucial for diagnosing and treating coronary artery disease. However, accurate reconstruction is challenging due to the non-simultaneous acquisition of angiographic images and the complex motion patterns of coronary arteries. State-of-the-art methods typically involve a two-stage process: manual selection of corresponding point pairs for spatial geometric calibration, followed by centerline reconstruction. However, overlap and foreshortening in 2D images complicate point selection, often requiring repeated adjustments, and the lack of sufficient point pairs can lead to reconstruction failure. In this paper, we propose a one-stage automatic approach that integrates calibration and 3D centerline reconstruction, eliminating the need for manual calibration. For each angiographic image, we constructed a 3D deformable curve corresponding to the 2D vessel centerline, strictly constrained by the projection lines. Unlike traditional methods that minimize 2D reprojection errors, our approach minimizes the 3D spatial distance between two 3D curves, simultaneously optimizing the spatial transformation and the two deformable 3D curves. The transformation is optimized through iterative curves registration, while the curves are evolved based on a cosine representation method. Both processes occur simultaneously and mutually reinforce each other, resulting in high-precision 3D reconstruction without manual calibration. The proposed approach was validated on 45 phantom and 107 clinical data. The mean space error was $0.085~\pm ~0.085$ mm for phantom data; and the mean reprojection error was $0.060~\pm ~0.027$ mm for clinical data. Results demonstrated that our approach achieves state-of-the-art accuracy while eliminating the need for manual intervention.

Multimodal analysis of placental ultrasound (US) and microflow imaging (MFI) could greatly aid in the early diagnosis and interventional treatment of placental insufficiency (PI), ensuring a normal pregnancy. Existing multimodal analysis methods have weaknesses in multimodal feature representation and modal knowledge definitions and fail on incomplete datasets with unpaired multimodal samples. To address these challenges and efficiently leverage the incomplete multimodal dataset for accurate PI diagnosis, we propose a novel graph-based manifold regularization learning (MRL) framework named GMRLNet. It takes US and MFI images as input and exploits their modality-shared and modality-specific information for optimal multimodal feature representation. Specifically, a graph convolutional-based shared and specific transfer network (GSSTN) is designed to explore intra-modal feature associations, thus decoupling each modal input into interpretable shared and specific spaces. For unimodal knowledge definitions, graph-based manifold knowledge is introduced to describe the sample-level feature representation, local inter-sample relations, and global data distribution of each modality. Then, an MRL paradigm is designed for inter-modal manifold knowledge transfer to obtain effective cross-modal feature representations. Furthermore, MRL transfers the knowledge between both paired and unpaired data for robust learning on incomplete datasets. Experiments were conducted on two clinical datasets to validate the PI classification performance and generalization of GMRLNet. State-of-the-art comparisons show the higher accuracy of GMRLNet on incomplete datasets. Our method achieves 0.913 AUC and 0.904 balanced accuracy (bACC) for paired US and MFI images, as well as 0.906 AUC and 0.888 bACC for unimodal US images, illustrating its application potential in PI CAD systems.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and ultrasound (US), which are two common modalities for clinical breast tumor diagnosis besides Mammograms, can provide different and complementary information for the same tumor regions. Although many machine learning methods have been proposed for breast tumor classification based on either single modality, it remains unclear how to further boost the classification performance by utilizing paired multi-modality information with different dimensions. In this paper, we propose MRI-US multi-modality network (MUM-Net) to classify breast tumor into different subtypes based on 3D MR and 2D US images. The key insight of MUM-Net is that we explicitly distill modality-agnostic features for tumor classification. Specifically, we first adopt a discrimination-adaption module to decompose features into modality-agnostic and modality-specific ones with min-max training strategies. Then, we propose a feature fusion module to increase the compactness of the modality-agnostic features by utilizing an affinity matrix with nearest neighbour selection. We build a paired MRI-US breast tumor classification dataset containing 502 cases with three clinical indicators to validate the proposed method. In three tasks including lymph node metastasis, histological grade and Ki-67 level, MUM-Net achieves AUC scores of 0.8581, 0.8965 and 0.8577, outperforming other counterparts which are based on single task or single modality by a wide margin. In addition, we find that the extracted modality-agnostic features can help the network focus on the tumor regions in both modalities.

3D coronary artery reconstruction (3D-CAR) in intravascular ultrasound (IVUS) sequences allows quantitative analyses of vessel properties. Existing methods treat two main tasks of the 3D-CAR separately, including the cardiac phase retrieval (CPR) and the membrane border extraction (MBE). They ignore the CPR-MBE connection that could achieve mutual promotions to both tasks. In this paper, we pioneer to achieve one-step 3D-CAR via a collaborative constraint generative adversarial network (GAN) named the AwCPM-Net. The AwCPM-Net consists of a dual-task collaborative generator and a dual-task constraint discriminator. The generator combines a self-supervised CPR branch with a semi-supervised MBE branch via a warming-up connection. The discriminator promotes dual-branch predictions simultaneously. The CPR branch requires no annotations and outputs inter-frame deformation fields used for identifying cardiac phases. Deformation fields are additionally constrained by the MBE branch and the discriminator. The MBE branch predicts membrane boundaries for each frame. Two aspects assist the semi-supervised segmentation: annotation augmentation by deformation fields of the CPR branch; information exploitation on unlabeled images enabled by GAN design. Trained and tested on an IVUS dataset acquired from atherosclerosis patients, the AwCPM-Net is effective in both CPR and MBE tasks, superior to state-of-the-art IVUS CPR or MBE methods. Hence, the AwCPM-Net reconstructs reliable 3D artery anatomy in the IVUS modality.

Recently, the use of portable equipment has attracted much attention in the medical ultrasound field. Handheld ultrasound devices have great potential for improving the convenience of diagnosis, but noise-induced artifacts and low resolution limit their application. To enhance the video quality of handheld ultrasound devices, we propose a low-rank representation multipathway generative adversarial network (LRR MPGAN) with a cascade training strategy. This method can directly generate sequential, high-quality ultrasound video with clear tissue structures and details. In the cascade training process, the network is first trained with plane wave (PW) single-/multiangle video pairs to capture dynamic information and then fine-tuned with handheld/high-end image pairs to extract high-quality single-frame information. In the proposed GAN structure, a multipathway generator is applied to implement the cascade training strategy, which can simultaneously extract dynamic information and synthesize multiframe features. The LRR decomposition channel approach guarantees the fine reconstruction of both global features and local details. In addition, a novel ultrasound loss is added to the conventional mean square error (MSE) loss to acquire ultrasound-specific perceptual features. A comprehensive evaluation is conducted in the experiments, and the results confirm that the proposed method can effectively reconstruct high-quality ultrasound videos for handheld devices. With the aid of the proposed method, handheld ultrasound devices can be used to obtain convincing and convenient diagnoses.

In the medical ultrasound field, ultrafast imaging has recently become a hot topic. However, the diagnostic reliability of ultrafast high-frame rate plane-wave (PW) imaging is reduced by its low-quality images. The medical ultrasound equipment on the market usually adopts the line-scanning mode, which can obtain high-quality images at a very low frame rate. In addition, many proven data-driven ultrasound image processing methods are trained by line-scan images. Since the gray-level distributions of line-scan images and PW images are very different, these gray-level distribution-sensitive methods cannot be generalized to ultrafast ultrasound imaging, which limits further applications. Hence, we propose an ultrasound-transfer generative adversarial network to improve the quality of PW images and extend the existing image processing methods to ultrafast ultrasound imaging by reconstructing PW images into line-scan images. This network adopts a residual dense generator with a self-attention system that fully uses the hierarchical features and generates details from all the relevant physiological information. A projection discriminator and spectral normalization are introduced to increase the discernibility and to maintain a balance between the generator and the discriminator. Moreover, we reorganize the transmit sequence of the transducer array to eliminate the negative influence of human movements and facilitate the convergence of the proposed model. The experimental results are evaluated with five metrics, which confirm the feasibility of the proposed method to obtain a line-scan-quality image with a very high frame rate. This technology could significantly popularize ultrafast medical ultrasound imaging.

BACKGROUND: Image registration is an important research topic in the field of image processing. Applying image registration to vascular image allows multiple images to be strengthened and fused, which has practical value in disease detection, clinical assisted therapy, etc. However, it is hard to register vascular structures with high noise and large difference in an efficient and effective method. RESULTS: Different from common image registration methods based on area or features, which were sensitive to distortion and uncertainty in vascular structure, we proposed a novel registration method based on network structure and circuit simulation. Vessel images were transformed to graph networks and segmented to branches to reduce the calculation complexity. Weighted graph networks were then converted to circuits, in which node voltages of the circuit reflecting the vessel structures were used for node registration. The experiments in the two-dimensional and three-dimensional simulation and clinical image sets showed the success of our proposed method in registration. CONCLUSIONS: The proposed vascular image registration method based on network structure and circuit simulation is stable, fault tolerant and efficient, which is a useful complement to the current mainstream image registration methods.

Segmentation of pheochromocytomas in contrast-enhanced computed tomography (CECT) images is an ill-posed problem due to the presence of weak boundaries, intratumoral degeneration, and nearby structures and clutter. Additional information from different phases of CECT images needs to be imposed for better mass segmentations. In this paper, a novel adaptive cosegmentation method is proposed by incorporating a localized region-based level set model (LRLSM). The energy function is formulated with consideration of adaptive tradeoff between the complementary local information from image pairs. Gradient direction and shape dissimilarity measure are integrated to guide the level set evolution. Automatic localization radius selection is added to further facilitate the segmentation. Then, two level set functions from each image pair are evolved and refined alternately to minimize the energy function. Experimental results in 50 CECT image pairs show that the adaptive LRLSM-based method is effective in segmentation of pheochromocytoma at two phases and produces better results, especially in the cases with weak boundaries, and complex foreground and background.

Identification of intracardiac masses in echocardiograms is one important task in cardiac disease diagnosis. To improve diagnosis accuracy, a novel fully automatic classification method based on the sparse representation is proposed to distinguish intracardiac tumor and thrombi in echocardiography. First, a region of interest is cropped to define the mass area. Then, a unique globally denoising method is employed to remove the speckle and preserve the anatomical structure. Subsequently, the contour of the mass and its connected atrial wall are described by the K-singular value decomposition and a modified active contour model. Finally, the motion, the boundary as well as the texture features are processed by a sparse representation classifier to distinguish two masses. Ninety-seven clinical echocardiogram sequences are collected to assess the effectiveness. Compared with other state-of-the-art classifiers, our proposed method demonstrates the best performance by achieving an accuracy of 96.91%, a sensitivity of 100%, and a specificity of 93.02%. It explicates that our method is capable of classifying intracardiac tumors and thrombi in echocardiography, potentially to assist the cardiologists in the clinical practice.

A computerized classification based on morphologic and texture features is proposed to increase the accuracy of the ultrasonic diagnosis of breast tumors. Firstly, tumor boundaries are obtained with the gray-level threshold segmentation algorithm and the dynamic programming method. Then five morphologic features and two texture features are extracted. Finally, an artificial neural network with the error back propagation algorithm is applied to classify breast tumors as benign or malignant. Experiments on 168 cases show that the proposed system yields the high accuracy, sensitivity and specificity. Therefore, it is concluded that this system performs well in the ultrasonic classification of breast tumors.