Non-rigid registration of intraoperative tissue is essential for surgical navigation and scene reconstruction in minimally invasive surgery. However, accurate registration remains challenging due to significant tissue deformation and partial overlaps caused by laparoscope movement. We propose an Overlap-Aware Online-Adaptive Non-Rigid Registration Method (OANRM) to address these challenges. The framework introduces a Hierarchical Matching Network (HMNet) that simultaneously predicts overlapping regions and their correspondences through a novel similarity-based approach. Our method uniquely incorporates an online adaptation mechanism that continuously fine-tunes the network parameters using unsupervised losses, enabling robust performance across varying surgical scenarios without requiring additional training data. A Transform Displacement Deformation Prediction (TDDP) module further enhances the framework by handling non-overlapping regions through integrating Random Sample Consensus with distance-based interpolation. The method is validated on both artificial datasets with controlled deformations and clinical datasets from real surgical procedures. Experimental results demonstrate that OANRM achieves state-of-the-art performance, significantly outperforming existing methods in handling complex tissue deformations and varying overlap ratios. https://github.com/AIGCer0807/OANRM.

Resection of pathological tissue is a common procedure in surgical oncology for treating tumors. In robot-assisted electrosurgery, the use of predefined markers to guide autonomous robotic resection is gaining traction. Accurate tracking of these markers and minimizing electrocautery damage are critical for the safe and effective autonomous resection of tumors. This paper introduces a safety enhanced autonomous resection method for laparoscopic surgery, designed to mitigate the risks posed by tissue deformation during the resection process. Initially, we pre-plan the cutting path and design a switching strategy for navigation waypoints based on a preview tracking mechanism. Then, we develop a depth-fused navigation controller and a safe withdrawal motion controller. Next, an inertial tracking mechanism is established to evaluate tissue deformation over short periods. Finally, we develop a confidence generator to fuse the two controllers, ensuring that tissue deformation during the resection process does not cause additional electrocautery damage. Simulation and phantom experiments were conducted, demonstrating the effectiveness of our proposed method. This work represents a significant step toward achieving autonomous robotic resection.

In minimally invasive surgery (MIS), tissue retraction is critical for exposing the surgical site and facilitating pathological tissue excision. However, tissue retraction in MIS is subject to multiple constraints, including restricted field of view, non-damaging tissue retraction force, and limited instrument operating range, which hinder the safe and continuous retraction of tissue during robotic-assisted surgery. This paper introduces a novel autonomous retraction method for MIS, capable of safely retracting tissue within the aforementioned multiple constraints, assisting the surgeon in tissue excision. The method takes into account information from three different modalities: vision, force, and position. Based on these, three distinct control objectives are defined: retraction angle, retraction force, and safety space constraints, with an individual controller and modelpredictive evaluation function designed for each objective. We propose a novel multi-objective motion fusion strategy designed to balance three distinct control objectives. This strategy evaluates the sensitivity of each objective to changes in the system state by comparing the gradients of their respective prediction evaluation functions and fuses the control inputs of the individual controllers. The proposed method allows for rapid addition or removal of objectives without altering the algorithmic framework. Experiments with phantoms andex vivoanimal tissue are conducted on robotic platform to validate the effectiveness of the proposed method in various configurations.

The development of autonomous laparoscope control in robot-assisted surgery has emerged as a significant research area, particularly due to its potential to reduce assistant fatigue and minimize miscommunication between the surgeon and assistant. A notable challenge, however, is the tendency of autonomous control strategies to override the surgeon’s direct command occasionally. To address this issue, we propose a novel haptic feedback-based cooperative control strategy that enhances the surgeon’s command of laparoscopic field of view (FOV) movement in robot-assisted laparoscopic surgery. Specifically, we first established a dynamic model of the laparoscope-holding robot, which serves as a link between the movement of the laparoscopic FOV and the surgical instruments to deliver haptic feedback to the surgeon. Next, a motion observer was developed to transform 30 Hz visual feedback into 1 kHz haptic feedback by integrating visual tracking data with kinematic information, ensuring smoother and more continuous haptic feedback. Finally, we propose two distinct collaboration modes: the plane tracking mode (PTM) ensures instruments remain within the laparoscopic image, and the space tracking mode (STM) synchronizes the laparoscope with instrument movement. The laboratory experiments validated the effectiveness of the proposed method in enhancing the cooperative performance of robot-assisted laparoscope systems while animal experiments demonstrated the feasibility of the PTM design.

Laparoscope-holding robots significantly enhance the stability and precision of visualization in minimally invasive surgeries. Most existing robots of this kind depend on visual servo systems and struggle with efficient, rapid adjustments in the field-of-view (FOV), especially when identifying organs and needles outside the FOV. This paper presents a laparoscope-holding robot system capable of employing both vision-driven and force-driven mechanisms for continuous and large-scale FOV adjustments, respectively. The system features an integrated tactile handle, enabling the reception of human-robot interaction forces during surgical navigation. We propose a hybrid control method that leverages both force and vision inputs for laparoscopic FOV adjustments. This approach integrates a virtual wrench, generated from visual information, and an interaction wrench, obtained from the tactile handle, into the robot's dynamic model, which complies with remote center of motion constraints. The interaction wrench's gain is adjusted with the gripping force on the integrated tactile handle, ensuring that unintended movements caused by accidental contacts are prevented, thus safeguarding operational safety. The proposed method eliminates the need to switch control modes, enabling simultaneous visual tracking and tactile interaction guidance. Experimental results demonstrate that the proposed method not only allows for FOV adjustments with surgical instrument guiding but also adapts well to large-scale FOV adjustment tasks.

Numerous studies have emphasized the application of autonomous intelligence in human-robot shared control to enhance surgical convenience and efficiency. However, the neglect of human dominance may reduce surgical safety. This paper developed a safety-enhanced human-robot shared control method by intelligently allocating control authority, with the surgeon remaining the leader during the surgical procedure. Three controllers are designed initially, including a master hand position (MP) controller and a master hand velocity (MV) controller related to the surgeon's manipulation, and a planned trajectory tracking (PT) controller related to the robot. In precision surgical manipulation scenarios, precise tracking of the human's operation is achieved by combining MP and MV controllers, while a combination of MV and PT controllers is developed in high-efficiency surgical scenarios, which relaxes the requirement for precise tracking of hand position and enables precise robot assistance guided by the velocity of human hand. The autonomous scenarios and controllers switching are accomplished through a motion fusion mechanism, which is achieved via optimizing evaluation functions that are reliant on future states. Furthermore, a force feedback mechanism is proposed to help human understand the intent of autonomous control to improve safety. The feasibility and effectiveness of this method have been validated through simulations and experiments.

Depression is a mental disorder that causes pain to people and society and is also the largest cause of disability in the world. Intelligent early screening of depression is of great benefit for patients to obtain better diagnoses and treatment. However, previous low-precision detection methods based on facial vision heavily rely on computing resources, which hinders the wide application of automatic depression diagnoses. Therefore, this article proposes an intelligent method for multiscene automatic depression symptom detection, which uses an efficient and convenient cloud-edge collaboration framework combined with negative emotion monitoring and cross-scale facial feature analysis. We deploy a shallow model (EdgeER) on the edge server and a deep model (C-DepressNet) on the cloud server. EdgeER is used to quickly detect negative user emotions and screen user data. C-DepressNet is used to analyze degrees of depression with high precision. The experimental results show that our cloud-edge collaboration framework has superior performance in depression detection accuracy and service response times.

Gastroscopy is a widely adopted method for locating gastric lesions and performing the early screening and diagnosis of gastric cancer (GC). However, the effectiveness of traditional GC screening methods depends on the medical skills of the gastroscopy specialist. A lack of knowledge and experience may lead to misdiagnosis and mistreatment, especially in small-scale hospitals. Recently, there has been a significant increase in studies on data-driven computer-aided diagnosis techniques. In this article, we propose a novel intelligent decision-making method for GC screening (ID-GCS), a multimodal semantic fusion-based data-driven decision-making system. ID-GCS exploits a hybrid attention mechanism to extract textual semantics from multimodal gastroscopy reports and performs semantic fusion to integrate the semantics of textual gastroscopy reports and images, resulting in improved interpretability of gastroscopy findings. We evaluated ID-GCS using a real gastroscopy report dataset, and experimental results show that compared with state-of-the-art methods, ID-GCS achieves better sensitivity and accuracy in GC screening.

Abstract Compared to patients readmitted to general wards, readmitted patients in the intensive care unit (ICU) are exposed to higher mortality rates and prolonged hospital stays. Moreover, the readmission of ICU patients brings pressing challenges for ICU management. Most models are devoted to identifying the risk factors and developing classification models that can predict whether ICU patients will be readmitted. Though these models are prominent, they do not provide estimates for the frequency of readmissions. This paper establishes a prediction model, hybrid feature selection‐LightGBM (HFS‐LightGBM), to evaluate the probability and frequency of ICU patient readmissions empirically. In terms of feature selection, a hybrid feature selection (HFS) algorithm for LightGBM combines the filter and wrapper methods. Pearson's correlation coefficient is employed in the filter procedure. Then we adopt the targeted LightGBM classifier along with the recursive feature elimination and cross‐validated (RFECV) to produce the optimal feature subset. Additionally, the hyperparameters of the HFS‐LightGBM are optimized. The HFS‐LightGBM is employed on the real‐world ICU dataset containing 1722 patients' electronic health records. This model outperforms the current prevailing readmission models. The identified frequency can assist doctors in making specific interventions for patients to reduce the ICU readmission rate.

Three-dimensional display and virtual reality technology have been applied in minimally invasive surgery to provide doctors with a more immersive surgical experience. One of the most popular systems based on this technology is the Da Vinci surgical robot system. The key to build the in vivo 3-D virtual reality model with a monocular endoscope is an accurate estimation of depth and motion. In this article, a fully unsupervised learning method for depth and motion estimation using the continuous monocular endoscopic video is proposed. After the detection of highlighted regions, EndoMotionNet and EndoDepthNet are designed to estimate ego-motion and depth, respectively. The timing information between consecutive frames is considered with a long short-term memory layer by EndoMotionNet to enhance the accuracy of ego-motion estimation. The estimated depth value of the previous frame is used to estimate the depth of the next frame by EndoDepthNet with a multimode fusion mechanism. The custom loss function is defined to improve the robustness and accuracy of the proposed unsupervised-learning-based method. Experiments with the public datasets verify that the proposed unsupervised-learning-based continuous depth and motion estimation method can effectively improve the accuracy of depth and motion estimation, especially after processing the frame.

Automatic acetowhite lesion segmentation in colposcopy images (cervigrams) is essential in assisting gynecologists for the diagnosis of cervical intraepithelial neoplasia grades and cervical cancer. It can also help gynecologists determine the correct lesion areas for further pathological examination. Existing computer-aided diagnosis algorithms show poor segmentation performance because of specular reflections, insufficient training data and the inability to focus on semantically meaningful lesion parts. In this paper, a novel computer-aided diagnosis algorithm is proposed to segment acetowhite lesions in cervigrams automatically. To reduce the interference of specularities on segmentation performance, a specular reflection removal mechanism is presented to detect and inpaint these areas with precision. Moreover, we design a cervigram image classification network to classify pathology results and generate lesion attention maps, which are subsequently leveraged to guide a more accurate lesion segmentation task by the proposed lesion-aware convolutional neural network. We conducted comprehensive experiments to evaluate the proposed approaches on 3045 clinical cervigrams. Our results show that our method outperforms state-of-the-art approaches and achieves better Dice similarity coefficient and Hausdorff Distance values in acetowhite legion segmentation.

Recent studies have shown that there exhibits significantly imbalanced medical resource allocation across public hospitals. Patients, regardless of their diseases, tend to choose hospitals and physicians with a better reputation, which often overloads major hospitals while leaving others underutilized. Guiding patients to hospitals that can serve their treatment needs both timely and with good quality can make the best use of precious medical resources. Unfortunately, it remains one of the major challenges both for research and in practice. In this article, we propose a novel diversity-enhanced hierarchical physician recommendation approach to address this issue. We adopt matrix factorization to estimate physician competency and exploit implicit similarity relationships to improve the competency estimation of physicians that we are of little information of. We then balance the patient preference and physician diversity using two novel heuristic algorithms. We evaluate our proposed approach and compare it with the state of the art. Experiments show that our approach significantly improves both accuracy and recommendation diversity over existing approaches.

Abstract Gastroscopy is important for finding suspicious stomach lesions, screening for gastric cancer, and providing early diagnoses. Due to the differences in the levels of diagnosis and treatment among gastroscope doctors, clinical diagnosis based on gastroscopy is limited by low diagnostic sensitivity and specificity to gastric cancer. An assistive system for gastroscopy report analysis can be helpful to improve the success rate of gastric cancer detection. In this study, a homogeneous ensemble decision support system for gastric cancer screening (Endo‐GCS) that performs word segmentation, feature extraction, and gastric cancer screening on text‐based gastroscopy reports is proposed. The proposed Endo‐GCS method establishes a progressive local weighting algorithm that improves the overall prediction performance of the homogeneous ensemble model in gastric cancer screening. An optimal threshold estimation algorithm is developed to minimize the negative impact of misdiagnosis and missed diagnoses. Through a comparative experimental study using real gastroscopy report data, the pathological examination conclusion is the gold standard. The sensitivity of the proposed Endo‐GCS method is 88.27%, the specificity is 77.84%, and the accuracy is 82.11%, which significantly improved the sensitivity 65.49% and the accuracy 80.5% of the gastroscopic diagnosis results, respectively.

In vivo manipulation of microparticles, such as biological cells and drugs, has attracted considerable attention in recent years. This paper presents the development of robot-aided manipulation technology that can control targeted microparticles to move a relatively long distance in an in vivo environment. The field of view can be updated online, such that the controlled microparticle can be tracked automatically in transportation. Simultaneous localization and mapping for in vivo applications are first investigated. Based on the in vivo map, an artificial potential field-based controller with disturbance compensation is developed to navigate microparticles in vivo. Experiments on navigating single cells in living zebrafish embryos by using optical tweezers manipulator are performed to demonstrate the effectiveness of the proposed control approach in a dynamic in vivo environment.

As the rapid development of precision medicine, in-vivo manipulation of micro/nano-scaled particles has attracted increasing attention in recent years. The collision is one of the main reasons that falls the in-vivo particle transportation fail. In this paper, we develop an in-vivo cell transportation control approach using a robotically controlled optical tweezers manipulation system, where a so-called wide-area image gradient algorithm is used to avoid collision of the transported cell with other obstacles. The controller exhibits advantages of the reduced online calculation for collision avoidance, fast response, high accuracy, as well as an ability to compensate the environmental disturbance caused by blood flow. Simulation and experimental results are presented to demonstrate the effectiveness of the proposed controller.

In vivo manipulation of biological cells has attracted considerable attention in recent years. This process is particularly useful for precision medicine, such as cancer target therapy. Robotics technology is becoming necessary to stably and effectively manipulate and control single target cells in a complex in vivo environment. This paper presents a robot-aided optical tweezers-based manipulation technology that serves a function in the transport of single biological cells in vivo. An enhanced disturbance compensation controller is developed to minimize the effect of fluids (e.g., blood flow) on the cell. The method has exhibited advantages of flexibility in adjusting cell tracking trajectory online and the capability to minimize steady-state error and eliminate overshoot. Simulations and experiments of tracking single target cells in living zebrafish embryos have demonstrated the effectiveness of the proposed approach in a dynamic in vivo environment.

As rapid development of precision medicine, in vivo manipulation of micro/nano-scaled particles have attracted increasing attention in recent years. To accommodate complex in-vivo environment, robot-aided automated manipulation technology is highly demanded in trapping and controlling micro/nano-particles stably and effectively. This paper presents an in-vivo cell manipulation system, where a disturbance compensation controller is utilized to minimize the effect of fluid (e.g., blood flow) on the cell. The controller has exhibited advantages in adjusting cell tracking trajectory online, minimizing the steady-state error, and eliminating overshoot. Simulation and experimental results verify the performance of the controller.