Reliable recognition and localization of surgical instruments in endoscopic video recordings are foundational for a wide range of applications in computer- and robot-assisted minimally invasive surgery (RAMIS), including surgical training, skill assessment, and autonomous assistance. However, robust performance under real-world conditions remains a significant challenge. Incorporating surgical context - such as the current procedural phase - has emerged as a promising strategy to improve robustness and interpretability. To address these challenges, we organized the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) sub-challenge as part of the Endoscopic Vision (EndoVis) challenge at MICCAI 2024. We introduced a novel, multi-center dataset comprising thirteen full-length laparoscopic cholecystectomy videos collected from three distinct medical institutions, with unified annotations for three interrelated tasks: surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation. Unlike existing datasets, ours enables joint investigation of instrument localization and procedural context within the same data while supporting the integration of temporal information across entire procedures. We report results and findings in accordance with the BIAS guidelines for biomedical image analysis challenges. The PhaKIR sub-challenge advances the field by providing a unique benchmark for developing temporally aware, context-driven methods in RAMIS and offers a high-quality resource to support future research in surgical scene understanding.

Unsupervised anomaly detection has gained significant attention in the field of medical imaging due to its capability of reducing the need for costly pixel-level annotation. To achieve this, existing approaches usually utilize generative models to produce healthy references of the diseased images and then identify the abnormalities by comparing healthy references and original diseased images. However, intrinsic characteristics of brain images, e.g., the low contrast and the intricate anatomical structure, make reconstruction challenging. To address those challenges, we propose a Masked Diffusion Model (MDiff), which incorporates a hierarchical patch partition strategy into the diffusion model for precise reconstruction of detailed content. Aligned with this strategy, we treat the perturbed upper-level patch as masked and introduce a masked modeling mechanism into MDiff's diffusion U-Net. This mechanism operates on sub-level patches, enhancing the model's ability to process contextual information surrounding the perturbed upper-level patch. To further improve the quality of healthy references, we integrate a memory module within the mechanism's encoder to retrieve the most relevant memory items as contextual information, while employing the learnable query embedding in its decoder to prevent the network from learning identical shortcuts. Experiments on tumor and multiple sclerosis lesion data demonstrate MDiff's effectiveness.

The integration of multimodal data including pathology images and gene profiles is widely applied in precise survival prediction. Despite recent advances in multimodal survival models, collecting complete modalities for multi-modal fusion still poses a significant challenge, hindering their application in clinical settings. Current approaches tackling incomplete modalities often fall short, as they typically compensate for only a limited part of the knowledge of missing modalities. To address this issue, we propose a Distilled Prompt Learning framework (DisPro) to utilize the strong robustness of Large Language Models (LLMs) to missing modalities, which employs two-stage prompting for compensation of comprehensive information for missing modalities. In the first stage, Unimodal Prompting (UniPro) distills the knowledge distribution of each modality, preparing for supplementing modality-specific knowledge of the missing modality in the subsequent stage. In the second stage, Multimodal Prompting (MultiPro) leverages available modalities as prompts for LLMs to infer the missing modality, which provides modality-common information. Simultaneously, the unimodal knowledge acquired in the first stage is injected into multimodal inference to compensate for the modality-specific knowledge of the missing modality. Extensive experiments covering various missing scenarios demonstrated the superiority of the proposed method. The code is available at https://github.com/Innse/DisPro.