Ultrasound image acquisition in conventional transesophageal echocardiography (TEE) requires complex manual operation of the probe in the esophagus based on the interpretation of ultrasound images and in-depth knowledge of the cardiac anatomy. In this work, we formulate the TEE probe guidance task as a reinforcement learning (RL) problem, and present the first learning-based solution to 3-DOF control of a TEE probe based on the ultrasound image feedback, named RL-TEE, in order to mimic the visual search and navigation strategies of expert echocardiographers. The probe-tissue interaction in TEE is carefully modeled in our framework by considering both the requirements for navigation towards the standard views and compliance in the esophageal environment. Furthermore, we propose a hybrid deep Q-network model that augments a convolutional neural network backbone with self-attention mechanisms to better capture spatial information in ultrasound images to guide navigation decisions. The presented methods are preliminarily validated in a TEE simulation environment built with data from 25 subjects to acquire four standard views of the heart. Our results show that the proposed method can effectively learn to accurately and compliantly guide the probe movement for TEE standard view acquisition tasks and has a good generalization ability to unseen patient data. Note to Practitioners—The motivation of this paper is to realize 3-DOF movement guidance of a TEE probe to acquire the standard views of the heart based on the real-time images, which can be applied to existing robotic control systems or used to assist novice echocardiographers in TEE examination, thereby relieving operator workload and improving ease of use. This paper suggests a novel approach that uses the deep RL technique to achieve automatic interpretation of TEE images and intelligent guidance of the probe movement. The RL framework is designed to take into account both the navigation efficiency and compliance with the esophageal environment for the targeted intracorporeal application. A hybrid deep Q-network model that augments a convolutional neural network with attention mechanisms is designed to better capture spatial information from ultrasound images to predict the probe movement. The effectiveness of the framework is preliminarily validated in extensive experiments in a simulation environment built with real patient data. The proposed method can be applied in clinical use to provide real-time TEE probe guidance for novice echocardiographers, and can be integrated with a robotic system to fully automate the TEE acquisition, thereby relieving the doctors from tedious manual operation to focus on the diagnosis and treatment.

Current wireless capsule endoscopy (WCE) is limited in the long examination time and low flexibility since the capsule is passively moved by the natural peristalsis. Efforts have been made to facilitate active locomotion of WCE using magnetic actuation and localization technologies. In this work, we investigate the trajectory following problem of a robotic capsule under rotating magnetic actuation, in order to realize efficient and accurate navigation of the capsule in the narrow, complex intestinal environments. Specifically, four control strategies are developed based on the PD controller, adaptive controller (AC), model predictive controller (MPC) and robust multi-stage model predictive controller (RMMPC). In particular, the RMMPC method takes into account the uncertainty in the intestinal environment by modeling the intestinal peristalsis and friction in the controller design. We evaluate the proposed methods in simulation as well as in real-world experiments in several tubular environments, including plastic phantoms with complex shapes and an ex-vivo pig colon. The results have demonstrated the potential of the proposed control methods to realize accurate and efficient inspection of the intestine using active WCE. Our methods can be integrated with current WCE to improve the diagnostic accuracy and efficiency of the GI tract.Note to Practitioners—The motivation of this paper is to solve the trajectory following problem for active WCE in the human intestine to realize accurate, efficient and repeatable inspection of the gastrointestinal (GI) tract. We present four different control strategies for 5-DOF control of a robotic capsule endoscope actuated by a reciprocally rotating permanent magnet to make the capsule follow a predefined trajectory in a tubular environment. The accuracy and efficiency of the approach are validated in simulation and real-world experiments. The proposed trajectory following strategies can be integrated into existing WCE products to allow automatic and repeatable examination of the GI tract, and can also be extended to the locomotion of other tethered or untethered magnetic devices in the tubular environments for different medical and industrial applications. In the future, our proposed approach is expected to be combined with image-based automatic diagnosis of the GI tract to provide doctors with better tools for digestive examinations.

This article presents a closed-loop magnetic manipulation framework for robotic transesophageal echocardiography (TEE) acquisitions. Different from previous work on intracorporeal robotic ultrasound acquisitions that focus on continuum robot control, we first investigate the use of magnetic control methods for more direct, intuitive, and accurate manipulation of the distal tip of the probe. We modify a standard TEE probe by attaching a permanent magnet and an inertial measurement unit (IMU) sensor to the probe tip and replacing the flexible gastroscope with a soft tether containing only wires for transmitting ultrasound and IMU data and show that six-degree-of-freedom (DOF) localization and five-DOF closed-loop control of the probe can be achieved with an external permanent magnet based on the fusion of internal inertial measurement and external magnetic field sensing data. The proposed method does not require complex structures or motions of the actuator and the probe compared with existing magnetic manipulation methods. We have conducted extensive experiments to validate the effectiveness of the framework in terms of localization accuracy, update rate, workspace size, and tracking accuracy. In addition, our results obtained on a realistic cardiac tissue-mimicking phantom show that the proposed framework is applicable in real conditions and can generally meet the requirements for teleoperated TEE acquisitions.

Simultaneous magnetic actuation and localization is a promising technology to realize active wireless capsule endoscopy. In this article, an adaptive approach is proposed to efficiently propel and precisely locate a magnetically actuated capsule in unknown complex tubular environments. In order to track the capsule in real time in a large workspace, we improve upon our previous external sensor-based magnetic localization strategies by proposing an adaptive method to automatically activate an optimal subset of sensors during the capsule movement and developing a simplified multiple objects tracking algorithm to estimate the six-dimensional pose of the capsule in real time. Moreover, based on a study of capsule locomotion in tubular environments with different shapes, we propose to dynamically adjust the pose of the actuator to achieve efficient and robust propulsion of the capsule in an unknown tubular environment. The effectiveness of our proposed system is validated in extensive experiments on phantoms andex vivoanimal organs. Compared with the state-of-the-art, our method can realize the closed-loop control of a magnetic capsule without requiring sensing modules to be placed inside the capsule, which can reduce the space constraints and energy consumption of the capsule. Moreover, the system and algorithms for adaptive actuation and localization can provide an improved propulsion speed, workspace range, and localization update frequency as well as a comparable localization accuracy.

Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans. Furthermore, we propose a confidence-based approach to encode the optimization of image quality in the learning process. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine. Experimental results demonstrate that our method can perform reproducible US probe navigation towards the standard scan plane with an accuracy of 4.91mm/4.65° in the intra-patient setting, and accomplish the task in the intra- and inter-patient settings with a success rate of 92% and 46%, respectively. The results also show that the introduction of image quality optimization in our method can effectively improve the navigation performance.

Active wireless capsule endoscopy (WCE) under magnetic actuation is a promising technology to reduce the inspection time and relieve the burden of physicians. In this paper, we propose a reciprocally rotating magnetic actuation method for trajectory following of a capsule and develop its dynamic model. For the trajectory following task, we investigate the closed-loop tracking control strategies based on different controllers to actuate the capsule in the complex environments. The effectiveness of our method is validated in extensive experiments in a simulation environment as well as in an ex-vivo pig colon. The results demonstrate that the proposed method can accurately and efficiently actuate the capsule to follow the desired trajectory in the complex environments, achieving tracking errors on the order of millimeter. Moreover, the experiments on the ex-vivo pig colon show that the proposed reciprocally rotating magnetic actuation method has the potential to reduce the clinical risks and improve the safety and clinical acceptability of this technology.

Simultaneous magnetic actuation and localization (SMAL) is a promising technology for accelerating and positioning the capsule in the human intestine for wireless capsule endoscopy (WCE). In this article, we propose a novel system that uses a rotating magnetic actuator and an external sensor array to achieve closed-loop SMAL for a capsule with two embedded magnetic rings. First, the state of the capsule is detected as “Stuck,” “Synchronous,” or “Missing” by studying the relationship between the theoretical actuating magnetic field and the measured total magnetic field. Then, the undesired interference with the localization system caused by the actuator is eliminated with an integral filter-based approach. Different models are proposed to solve the pose of the capsule according to the different states of the capsule, and the localization result is used to update the pose of the actuator to close the loop. Extensive experiments on phantoms and animal organs with different environmental conditions are carried out to validate the proposed framework. The state detection accuracy achieves 96.7%, and the capsule can be located with an accuracy of 5.5 mm and 5.2° in position and orientation, respectively. Experimental results show the feasibility of our proposed system and demonstrate the robustness, accuracy, actuation efficiency, and closed-loop performance of the system.Note to Practitioners—The motivation of this article is to solve the problem of controlling the movement of a wireless capsule endoscope in the human intestine to assist intestinal diagnosis and treatment. We present a feasible system design and corresponding algorithms to achieve closed-loop simultaneous actuation and localization of a robotic capsule. The design of a simple structure placed inside the capsule (i.e., two magnetic rings) and an external sensor array mounted on the examination bed can reduce the size and power consumption of the capsule, and the use of a rotating actuator helps improve the actuation efficiency. The proposed SMAL framework, which is composed of state detection, interference removal, multimodel localization, and actuator updating, can close the actuation-localization loop and improve the accuracy and robustness of the system. We demonstrate the superiority of the proposed framework compared with others through extensive experiments. In the future, our SMAL system can be combined with image- or ultrasound-based automatic diagnosis and is expected to provide doctors with better tools for digestive examinations.

Wireless Capsule Endoscopy (WCE) has the advantage of reducing the invasiveness and pain of gastrointestinal examinations. In this work, we propose a system aimed at autonomously accelerating and locating the WCE inside the intestine for clinical applications. A rotating magnet controlled by a robotic arm is placed outside the patient's body to actuate the capsule with an internal magnetic ring, and the magnetic fields of the two sources are measured by an external sensor array. The original Multiple Objects Tracking method is improved by combining Normal Vector Fitting, Bézier Curve Gradient, and Spherical Linear Interpolation to estimate the 6-D pose of the WCE from a 5-D pose sequence. In order to close the actuation-localization loop, a strategy is presented to react to different states of the capsule. The proposed method is validated via experiments on phantoms as well as on animal intestines. The localization of the capsule shows an accuracy of 3.5mm in position and 9.4° in orientation, and the average update frequency of the estimated 6-D pose reaches 25Hz.