Large language models (LLMs) are increasingly deployed as intelligent agents capable of executing complex real-world tasks through external tool interactions, but effective tool selection remains challenging due to the inherent limitations of real-world training data. These datasets suffer from severe tool imbalance following long-tail distributions, data scarcity for specialized tools, logic conflicts between user queries and available tools, rapidly evolving toolsets, and the presence of subpar samples including partially correct and dirty examples. Existing supervised fine-tuning (SFT) approaches struggle with these multifaceted challenges as they require abundant high-quality data, treat all labeled examples as ground truth regardless of quality, and lack the flexibility to generalize beyond specific query-tool pairings seen during training. While reinforcement learning (RL) offers a promising alternative through outcome-based learning, vanilla approaches like Group Relative Policy Optimization (GRPO) suffer from training instability due to conflicting reward signals and inefficient learning from weak signals. To address these issues, we propose TOOL-CURE, a novel method with two key improvements to GRPO: Proficiency-Scaled Curriculum Learning (PSCL), which organizes training into a two-stage curriculum that builds foundational skills on easier samples before progressing to harder ones, and Online Policy Guarding via Sample Screening (OPGSS), which continuously assesses rollout quality and masks dirty samples to prevent noisy gradients from destabilizing policy updates. Our approach enables stable and efficient learning from heterogeneous real-world data, resulting in a robust tool-selection agent that demonstrates significant improvements in accuracy and generalization capability. Our code is available at: https://github.com/einnullnull/TOOL-CURE.git.

Surgical procedures are inherently complex and dynamic, with intricate dependencies and various execution paths. Accurate identification of the intentions behind critical actions, referred to as Primary Intentions (PIs), is crucial to understanding and planning the procedure. This paper presents a novel framework that advances PI recognition in instructional videos by combining top-down grammatical structure with bottom-up visual cues. The grammatical structure is based on a rich corpus of surgical procedures, offering a hierarchical perspective on surgical activities. A grammar parser, utilizing the surgical activity grammar, processes visual data obtained from laparoscopic images through surgical action detectors, ensuring a more precise interpretation of the visual information. Experimental results on the benchmark dataset demonstrate that our method outperforms existing surgical activity detectors that rely solely on visual features. Our research provides a promising foundation for developing advanced robotic surgical systems with enhanced planning and automation capabilities.

Accurate and reliable annotation of anatomical landmarks in laparoscopic surgery remains a challenge due to varying degrees of landmark visibility and changing shapes of human tissues during a surgical procedure in videos. In this paper, we propose a knowledge-driven framework that integrates prior surgical expertise with visual data to address this problem. Inspired by visual reasoning knowledge of tool-anatomy interactions, our framework models a spatio-temporal graph to represent the static topology of tool and tissue and dynamic transitions of landmarks' temporal behavior. By assigning explainable features of the surgical scene as node attributes in the graph, the surgical context is incorporated into the knowledge space. An attention-guided message passing mechanism across the graph dynamically adjusts the focus in different scenarios, enabling robust tracking of landmark states throughout the surgical process. Evaluations on the clinical dataset demonstrate the framework's ability to effectively use the inductive bias of explainable features to label landmarks, showing its potential in tackling intricate surgical tasks with improved stability and reliability.

Surgical workflow analysis integrates perception, comprehension, and prediction of the surgical workflow, which helps real-time surgical support systems provide proper guidance and assistance for surgeons. This article promotes the idea of critical actions, which refer to the essential surgical actions that progress towards the fulfillment of the operation. Fine-grained workflow analysis involves recognizing current critical actions and previewing the moving tendency of instruments in the early stage of critical actions. Aiming at this, we propose a framework that incorporates operational experience to improve the robustness and interpretability of action recognition in in-vivo situations. High-dimensional images are mapped into an experience-based explainable feature space with low dimensions to achieve critical action recognition through a hierarchical classification structure. To forecast the instrument's motion tendency, we model the motion primitives in the polar coordinate system (PCS) to represent patterns of complex trajectories. Given the laparoscopy variance, the adaptive pattern recognition (APR) method, which adapts to uncertain trajectories by modifying model parameters, is designed to improve prediction accuracy. The in-vivo dataset validations show that our framework fulfilled the surgical awareness tasks with exceptional accuracy and real-time performance.

Robot-Assisted Minimally Invasive Surgery (RAMIS), which introduced robot-actuated invasive tools to increase the dexterity and efficiency of traditional MIS, has become popular. Investigations on how to achieve autonomy in RAMIS have drawn vast intention recently, which urges further insights into the process of the surgical procedures. In this paper, the definition of critical actions, which discriminates the essential stages from regular surgical actions, is proposed to help decompose the complicated surgical processes. A critical intra-operative moment of the surgical workflow, which is called the keyframe, is introduced to indicate the beginning or ending moments of the critical actions. A keyframe detection method is proposed for critical action identification based on a new in-vivo dataset labeled by expert surgeons. Surgeons' criteria for critical actions are captured by the explainable features, which can be extracted from the raw laparoscopic images with a two-stage network. Motivated by the surgeon's decision process of keyframes, a hierarchical structure is designed for keyframe identification by checking the spatial-temporal characteristics of the explainable features. Experimental results show that the reliability of the proposed method for keyframe detection achieves unanimous agreement by expert surgeons.