Surgical workflow recognition (SWR) stands as a pivotal component in computer-assisted surgery and is dedicated to identifying phases from surgical videos. Many deep learning-based methods have been proposed for this task and achieved acceptable SWR results. However, these methods usually implicitly extract and aggregate spatio-temporal features, so that it is challenging for these methods to adequately use some spatial information that is strongly relevant to surgical phase in SWR task, such as the information from the surgical instruments. To address this issue, an Explicit Surgical Instrument Prompting (ESIP) approach is proposed for SWR task. ESIP leverages surgical instrument segmentation to generate instrument-specific visual prompts, which explicitly guide the extraction of crucial intra-frame spatial features through a frozen pre-trained backbone, then enable effective inter-frame spatio-temporal feature extraction and aggregation. Unlike multi-task approaches that jointly perform SWR with auxiliary tasks within a shared network framework, ESIP is a single-task SWR approach dedicated to optimize framework itself for more adequate feature extraction. Furthermore, to accomplish the segmentation prompting efficiently, this paper presents SAM-based segmentation with prompt tuning strategy to explicitly integrate segmentation features into spatial features. Experimental results on Cholec80, M2CAI and AutoLaparo datasets demonstrate that our ESIP method achieves the best performance in comparison with 16 SOTA methods, with a Precision of 91.8%, 89.5% and 89.6%, Recall of 92.2%, 89.5% and 76.9%, Jaccard of 83.3%, 77.0% and 67.3%, respectively.

Deformable image registration (DIR) is critical for accurate clinical diagnosis and effective treatment planning. However, patient movement, significant intensity differences, and large breathing deformations hinder accurate anatomical alignment in multi-modal image registration. These factors exacerbate the entanglement of anatomical and modality-specific style information, thereby severely limiting the performance of multi-modal registration. To address this, we propose a novel LAtent Diffusion-based Domain-Adaptive feature disentangling (LADDA) framework for unsupervised multi-modal medical image registration, which explicitly addresses the representation disentanglement. First, LADDA extracts reliable anatomical priors from the Latent Diffusion Model (LDM), facilitating downstream content-style disentangled learning. A Domain-Adaptive Feature Disentangling (DAFD) module is proposed to promote anatomical structure alignment further. This module disentangles image features into content and style information, boosting the network to focus on cross-modal content information. Next, a Neighborhood-Preserving Hashing (NPH) is constructed to further perceive and integrate hierarchical content information through local neighbourhood encoding, thereby maintaining cross-modal structural consistency. Furthermore, a Unilateral-Query-Frozen Attention (UQFA) module is proposed to enhance the coupling between upstream prior and downstream content information. The feature interaction within intra-domain consistent structures improves the fine recovery of detailed textures. The proposed framework is extensively evaluated on large-scale multi-center datasets, demonstrating superior performance across diverse clinical scenarios and strong generalization on out-of-distribution (OOD) data.

The integration of laboratory tests and medical images is crucial in making accurate disease prediction. However, imaging data exhibits temporal sparsity, compared to frequently collected laboratory tests. This temporal sparsity limits effective multi-modal interaction, which in turn degrades the prediction accuracy. We address this issue by generating additional medical images at more time points, conditioned on the laboratory tests. Inspired by the pivotal role of organs in mediating laboratory tests and imaging abnormalities, we propose an Organ-Centric Modal-Shared Image Generator. It converts laboratory tests into imaging abnormalities through two key components: 1) Organ-Centric Graph: It positions organs as central nodes connecting laboratory tests and imaging abnormalities; and 2) Knowledge-Guided Modal-Shared Trajectory Module: It binds multi-modal features across time into a unified organ state trajectory. Experimental results demonstrate that our method improves multi-modal prediction performance across various diseases. Code is available at https://github.com/LyapunovStability/Lab_Guide_Med_Image _Gen.

Semi-supervised medical image segmentation has advanced significantly by utilizing pseudo-labeled annotations. However, ensuring pseudo-label accuracy remains challenging, often causing misclassification and confirmation bias. Existing methods mainly use prediction uncertainty to exclude or downweight uncertain regions, but these areas frequently coincide with diagnostically important zones, such as lesion cores or tissue boundaries. Neglecting them can thus degrade segmentation performance. To address this, we propose a Dynamic Contrastive Correction Network (DCCN) that corrects uncertain regions instead of ignoring them. DCCN aligns features from high-uncertainty areas with dynamically assigned classes via contrastive learning, reconstructing the uncertain feature space. Additionally, a Multi-layer Sampling (MLS) module leverages boundary-aware sampling to focus contrastive learning on uncertain tissue boundaries. Experiments on two public datasets show that DCCN surpasses previous SOTA methods and effectively mitigates the challenges of high-uncertainty regions.

Mortality prediction is necessary for patients in the Intensive Care Unit (ICU). Clinical time series provide essential insights for making accurate predictions. However, existing prediction models often struggle with domain shifts when applied across different domains. Privacy concern hinders model calibration due to the forbidden data sharing across domains. Test-Time Training (TTT) has been increasingly researched to tackle the above issues by updating a source model to each single target sample before inference. While massive vision-based TTT methods are proposed, deploying TTT in clinical time series still faces the unique challenge of temporal imbalance: Time points tend to cluster around specific periods in some patients. Neglecting the temporal imbalance in TTT can make the model biased toward dense local pattern, resulting in unsatisfactory prediction. To overcome this challenge, we propose a novel Test-Time Training method with Diversified Local Aggregation Consistency (DLAC-TTT). During the test-time update, DLAC-TTT focuses on the distinct temporal distributions within each patient, enforcing their local diversity and global consistency through aggregation. In this way, it can mitigate the over-reliance on specific local patterns and well integrate diverse local patterns for global learning. Extensive experiments show that DLAC-TTT can boost the generalization performance across real-world clinical datasets from different medical institutes.

The increasing prevalence of non-alcoholic fatty liver disease (NAFLD) has caused public concern in recent years. The high prevalence and risk of severe complications make monitoring NAFLD progression a public health priority. Fibrosis staging from liver biopsy images plays a key role in demonstrating the histological progression of NAFLD. Fibrosis mainly involves the deposition of fibers around vessels. Current deep learning-based fi-brosis staging methods learn spatial relationships between tissue patches but do not explicitly consider the relation-ships between vessels and fibers, leading to limited performance and poor interpretability. In this paper, we propose an eXplicit vessel-fiber modeling method for Fibrosis staging from liver biopsy images, namely XFibrosis. Specifically, we transform vessels and fibers into graph-structured representations, where their micro-structures are depicted by vessel-induced primal graphs andfiber-induced dual graphs, respectively. Moreover, the fiber-induced dual graphs also represent the connectivity information between vessels caused by fiber deposition. A primal-dual graph convolution module is designed to facilitate the learning of spatial relationships between vessels and fibers, allowing for the joint exploration and interaction of their micro-structures. Experiments conducted on two datasets have shown that explicitly modeling the relationship between vessels and fibers leads to improved fibrosis staging and en-hanced interpretability.

Single-source domain generalization in medical image segmentation is a challenging yet practical task, as domain shift commonly exists across medical datasets. Previous works have attempted to alleviate this problem through adversarial data augmentation or random-style transformation. However, these approaches neither fully leverage medical information nor consider the morphological structure alterations. To address these limitations and enhance the fidelity and diversity of the augmented data, we propose a Morphology-Mixup Stylized data generation (MMS) method, which expands source data from a new morphological perspective, guided by the characteristics of medical imaging. Specifically, we design a Mixed Dual-stream Auto-Encoder (MDs-AE) to simulate the morphology changes between medical image slices and mix the morphology of two slices. In addition, we introduce a feature consistency strategy to improve the effectiveness of morphology mixing. The trained MDs-AE with a random styler is used to generate data that vary in both morphology and style to enhance the generalization ability of the segmentation network. Extensive experimental results demonstrate that MMS is effective and outperforms the state-of-the-art on three cross-domain segmentation tasks.

This work investigates the Single Domain Generalization (SDG), which generalizes a model from a single source domain to multiple unseen target domains. Most existing SDG methods focus on expanding the source domain by either transforming the source samples into different styles or optimizing adversarial noise perturbations applied to the source samples. However, these methods generate fictitious samples using specific image transformation, resulting in insufficient domain expansion. In this paper, we propose a progressive domain expansion method, namely domain dilation (DD) for SDG. This method dilates the source domain from two perspectives: enriching source domain diversity and generating various pseudo domains. To enrich source domain diversity, we generate fictitious samples with diverse styles. To obtain various pseudo domains, this paper generates pseudo domains with a new distribution by maximizing the domain difference from the source domain. Our method outperforms the state-of-the-art methods on prevalent single domain generalization benchmarks through extensive experiments, offering improved results.

Multi-modal learning has shown advantages in improving weakly-supervised medical image segmentation (WS- MIS). However, most current works are based on paired data, which is infeasible to collect in certain scenarios. Although modal translation can be used to generate paired data, it often leads to low-quality translations, such as local deformations or irrational textures, without prior knowledge. This paper proposes a discriminative-aware image translation method, which introduces class activation maps (CAMs) to localize discriminative areas, thus overcoming the lack of pixel-wise annotations in WS-MIS. In addition, we design a CAM-correlation constraint that facilitates multi-modal complementary information exchange to enhance the consistency between CAMs generated from different modalities. Experimental results show that our method outperforms recent weakly-supervised segmentation works when using unpaired multi-modal data.

Automatic lesion segmentation is important for assisting doctors in the diagnostic process. Recent deep learning approaches heavily rely on large-scale datasets, which are difficult to obtain in many clinical applications. Leveraging external labelled datasets is an effective solution to tackle the problem of insufficient training data. In this paper, we propose a new framework, namely LatenTrans, to utilize existing datasets for boosting the performance of lesion segmentation in extremely low data regimes. LatenTrans translates non-target lesions into target-like lesions and expands the training dataset with target-like data for better performance. Images are first projected to the latent space via aligned style-based generative models, and rich lesion semantics are encoded using the latent codes. A novel consistency-aware latent code manipulation module is proposed to enable high-quality local style transfer from non-target lesions to target-like lesions while preserving other parts. Moreover, we propose a new metric, Normalized Latent Distance, to solve the question of how to select an adequate one from various existing datasets for knowledge transfer. Extensive experiments are conducted on segmenting lung and brain lesions, and the experimental results demonstrate that our proposed LatenTrans is superior to existing methods for cross-disease lesion segmentation.

Medical time series have been widely employed for disease prediction. Missing data hinders accurate prediction. While existing imputation methods partially solve the problem, there are two challenges for medical time series: (1) High dimensionality: Existing imputation methods existing methods suffer from the trade-off between accuracy and computational efficiency. (2) Irregularity: Medical time series exhibit the dynamic temporal relationship that changes over varying sampling densities. However, existing methods mainly take the stationary mechanism, which struggles with capturing the dynamic temporal relationships. To overcome the above deficiencies, we propose a Density-Aware Temporal Attentive Step-wise Diffusion Model (DA-TASWDM), which imputes each time step based on a non-iterative diffusion model and captures inter-step dependency with the density-aware time similarity. Specifically, DA-TASWDM exploits two novel modules: (1) Density-Aware Temporal Attention (DA-TA): It correlates inter-step values from the time embedding similarity adjusted with varying sampling densities. (2) Non-Iterative Step-wise Diffusion Imputer (NI-SWDI): It directly recovers the missing values at each time step from noise without diffusion iteration. Compared with the existing methods, DA-TASWDM can achieve promising accuracy without sacrificing computational efficiency. Extensive experimental results on three real-world datasets demonstrate that our method can significantly outperform state-of-the-art methods in both imputation and post-imputation performance.

Coronavirus disease 2019 (COVID-19) has become a severe global pandemic. Accurate pneumonia infection segmentation is important for assisting doctors in diagnosing COVID-19. Deep learning-based methods can be developed for automatic segmentation, but the lack of large-scale well-annotated COVID-19 training datasets may hinder their performance. Semi-supervised segmentation is a promising solution which explores large amounts of unlabelled data, while most existing methods focus on pseudo-label refinement. In this paper, we propose a new perspective on semi-supervised learning for COVID-19 pneumonia infection segmentation, namely pseudo-label guided image synthesis. The main idea is to keep the pseudo-labels and synthesize new images to match them. The synthetic image has the same COVID-19 infected regions as indicated in the pseudo-label, and the reference style extracted from the style code pool is added to make it more realistic. We introduce two representative methods by incorporating the synthetic images into model training, including single-stage Synthesis-Assisted Cross Pseudo Supervision (SA-CPS) and multi-stage Synthesis-Assisted Self-Training (SA-ST), which can work individually as well as cooperatively. Synthesis-assisted methods expand the training data with high-quality synthetic data, thus improving the segmentation performance. Extensive experiments on two COVID-19 CT datasets for segmenting the infections demonstrate our method is superior to existing schemes for semi-supervised segmentation, and achieves the state-of-the-art performance on both datasets. Code is available at: https://github.com/FeiLyu/SASSL.

Automatic liver tumor segmentation is of great importance for assisting doctors in liver cancer diagnosis and treatment planning. Recently, deep learning approaches trained with pixel-level annotations have contributed many breakthroughs in image segmentation. However, acquiring such accurate dense annotations is time-consuming and labor-intensive, which limits the performance of deep neural networks for medical image segmentation. We note that Couinaud segment is widely used by radiologists when recording liver cancer-related findings in the reports, since it is well-suited for describing the localization of tumors. In this paper, we propose a novel approach to train convolutional networks for liver tumor segmentation using Couinaud segment annotations. Couinaud segment annotations are image-level labels with values ranging from 1 to 8, indicating a specific region of the liver. Our proposed model, namely CouinaudNet, can estimate pseudo tumor masks from the Couinaud segment annotations as pixel-wise supervision for training a fully supervised tumor segmentation model, and it is composed of two components: 1) an inpainting network with Couinaud segment masks which can effectively remove tumors for pathological images by filling the tumor regions with plausible healthy-looking intensities; 2) a difference spotting network for segmenting the tumors, which is trained with healthy-pathological pairs generated by an effective tumor synthesis strategy. The proposed method is extensively evaluated on two liver tumor segmentation datasets. The experimental results demonstrate that our method can achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods while requiring significantly less annotation effort.

Automatic liver tumor segmentation could offer assistance to radiologists in liver tumor diagnosis, and its performance has been significantly improved by recent deep learning based methods. These methods rely on large-scale well-annotated training datasets, but collecting such datasets is time-consuming and labor-intensive, which could hinder their performance in practical situations. Learning from synthetic data is an encouraging solution to address this problem. In our task, synthetic tumors can be injected to healthy images to form training pairs. However, directly applying the model trained using the synthetic tumor images on real test images performs poorly due to the domain shift problem. In this paper, we propose a novel approach, namely Synthetic-to-Real Test-Time Training (SR-TTT), to reduce the domain gap between synthetic training images and real test images. Specifically, we add a self-supervised auxiliary task, i.e., two-step reconstruction, which takes the output of the main segmentation task as its input to build an explicit connection between these two tasks. Moreover, we design a scheduled mixture strategy to avoid error accumulation and bias explosion in the training process. During test time, we adapt the segmentation model to each test image with self-supervision from the auxiliary task so as to improve the inference performance. The proposed method is extensively evaluated on two public datasets for liver tumor segmentation. The experimental results demonstrate that our proposed SR-TTT can effectively mitigate the synthetic-to-real domain shift problem in the liver tumor segmentation task, and is superior to existing state-of-the-art approaches.