This work presents a reinforcement learning-based switching control mechanism to autonomously move a ferromagnetic object (representing a milliscale robot) around obstacles within a constrained environment in the presence of disturbances. This mechanism can be used to navigate objects (e.g., capsule endoscopy, swarms of drug particles) through complex environments when active control is a necessity but where direct manipulation can be hazardous. The proposed control scheme consists of a switching control architecture implemented by two sub-controllers. The first sub-controller is designed to employ the robot’s inverse kinematic solutions to do an environment search for the to-be-carried ferromagnetic particle while being robust to disturbances. The second sub-controller uses a customized rainbow algorithm to control a robotic arm, i.e., the UR5 robot, to carry a ferromagnetic particle to a desired position through a constrained environment. For the customized Rainbow algorithm, Quantile Huber loss from the Implicit Quantile Networks (IQN) algorithm and ResNet are employed. The proposed controller is first trained and tested in a real-time physics simulation engine (PyBullet). Afterward, the trained controller is transferred to a UR5 robot to remotely transport a ferromagnetic particle in a real-world scenario to demonstrate the applicability of the proposed approach. The experimental results on the UR5 robot show an average success rate of 98.86% over 30 episodes for randomly generated trajectories, demonstrating the viability of the proposed approach for real-life applications. In addition, two classical path finding approaches, Attractor Dynamics and the execution extended Rapidly-Exploring Random Trees (ERRT), are also investigated and compared to the RL-based method. The proposed RL-based algorithm is shown to achieve performance comparable to that of the tested classical path planners whilst being more robust to deploy in dynamical environments. Note to Practitioners —Deep reinforcement learning methods have been widely applied in computer games and simulations. However, employing these algorithms for practical, real-world applications such as robotics becomes challenging due to the difficulty of obtaining training samples. This paper predominantly focuses on bridging the gap between simulations and the real-world implementation of a reinforcement learning algorithm for a robotic application in the context of miniaturized drug delivery robots and robotic capsule endoscopes. This paper presents the derivation and experimental validation of a reinforcement learning-based algorithm for controlling a magnetically-actuated small-scale robot within a simplified model of the large intestine in the presence of disturbances. We demonstrate the possibility of training a high-fidelity reinforcement learning algorithm fully within a simulated environment before deploying it as-is in a real-world scenario by carrying out different experiments and simulations. Implementing the presented control framework complements a large body of this work, and the results offer a feasibility study of using reinforcement learning algorithms in practice.

This paper establishes a physics-based simulation framework for steering a magnetically actuated guidewire based on the linear elasticity and dipoles theories. Interaction wrenches resulting from an external magnetic field and embedded magnets in a continuum rod, i.e., guidewire, serves as actuators for steering. In the presented framework, a simplified integration scheme based on the finite-volume method is employed to model guidewire using the linear elasticity theory and forces resulting from the interference of magnetic fields to provide a rapid model reconstruction. Furthermore, orienting the external magnetic field is employed to steer a guidewire into a constrained environment. Finally, simulations illustrate the approach performance on a soft rod where an external magnetic field is orientated to form the desired shape for a continuum rod and steer it within an environment. The results open up possibilities to construct a rapid model for continuum manipulators in practice.

Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.

BACKGROUND: Miniaturized accelerometers incorporated in pacing leads attached to the myocardium, are used to monitor cardiac function. For this purpose functional indices must be extracted from the acceleration signal. A method that automatically detects the time of aortic valve opening (AVO) and aortic valve closure (AVC) will be helpful for such extraction. We tested if deep learning can be used to detect these valve events from epicardially attached accelerometers, using high fidelity pressure measurements to establish ground truth for these valve events. METHOD: A deep neural network consisting of a CNN, an RNN, and a multi-head attention module was trained and tested on 130 recordings from 19 canines and 159 recordings from 27 porcines covering different interventions. Due to limited data, nested cross-validation was used to assess the accuracy of the method. RESULT: The correct detection rates were 98.9% and 97.1% for AVO and AVC in canines and 98.2% and 96.7% in porcines when defining a correct detection as a prediction closer than 40 ms to the ground truth. The incorrect detection rates were 0.7% and 2.3% for AVO and AVC in canines and 1.1% and 2.3% in porcines. The mean absolute error between correct detections and their ground truth was 8.4 ms and 7.2 ms for AVO and AVC in canines, and 8.9 ms and 10.1 ms in porcines. CONCLUSION: Deep neural networks can be used on signals from epicardially attached accelerometers for robust and accurate detection of the opening and closing of the aortic valve.

In laparoscopic liver resection, surgeons conventionally rely on anatomical landmarks detected through a laparoscope, preoperative volumetric images and laparoscopic ultrasound to compensate for the challenges of minimally invasive access. Image guidance using optical tracking and registration procedures is a promising tool, although often undermined by its inaccuracy. This study evaluates a novel surgical navigation solution that can compensate for liver deformations using an accurate and effective registration method. The proposed solution relies on a robotic C-arm to perform registration to preoperative CT/MRI image data and allows for intraoperative updates during resection using fluoroscopic images. Navigation is offered both as a 3D liver model with real-time instrument visualization, as well as an augmented reality overlay on the laparoscope camera view. Testing was conducted through a pre-clinical trial which included four porcine models. Accuracy of the navigation system was measured through two evaluation methods: liver surface fiducials reprojection and a comparison between planned and navigated resection margins. Target Registration Error with the fiducials evaluation shows that the accuracy in the vicinity of the lesion was 3.78±1.89 mm. Resection margin evaluations resulted in an overall median accuracy of 4.44 mm with a maximum error of 9.75 mm over the four subjects. The presented solution is accurate enough to be potentially clinically beneficial for surgical guidance in laparoscopic liver surgery.

Collaborative robots are becoming more common on factory floors as well as regular environments, however, their safety still is not a fully solved issue. Collision detection does not always perform as expected and collision avoidance is still an active research area. Collision avoidance works well for fixed robot-camera setups, however, if they are shifted around, Eye-to-Hand calibration becomes invalid making it difficult to accurately run many of the existing collision avoidance algorithms. We approach the problem by presenting a stand-alone system capable of detecting the robot and estimating its position, including individual joints, by using a simple 2D colour image as an input, where no Eye-to-Hand calibration is needed. As an extension of previous work, a two-stage transfer learning approach is used to re-train a multi-objective convolutional neural network (CNN) to allow it to be used with heterogeneous robot arms. Our method is capable of detecting the robot in real-time and new robot types can be added by having significantly smaller training datasets compared to the requirements of a fully trained network. We present data collection approach, the structure of the multi-objective CNN, the two-stage transfer learning training and test results by using real robots from Universal Robots, Kuka, and Franka Emika. Eventually, we analyse possible application areas of our method together with the possible improvements.

The success of minimally invasive interventions and the remarkable technological and medical progress have made endoscopic image enhancement a very active research field. Due to the intrinsic endoscopic domain characteristics and the surgical exercise, stereo endoscopic images may suffer from different degradations which affect its quality. Therefore, in order to provide the surgeons with a better visual feedback and improve the outcomes of possible subsequent processing steps, namely, a 3-D organ reconstruction/registration, it would be interesting to improve the stereo endoscopic image quality. To this end, we propose, in this paper, two joint enhancement methods which operate in the wavelet transform domain. More precisely, by resorting to a joint wavelet decomposition, the wavelet subbands of the right and left views are simultaneously processed to exploit the binocular vision properties. While the first proposed technique combines only the approximation subbands of both views, the second method combines all the wavelet subbands yielding an inter-view processing fully adapted to the local features of the stereo endoscopic images. Experimental results, carried out on various stereo endoscopic datasets, have demonstrated the efficiency of the proposed enhancement methods in terms of perceived visual image quality.

Percutaneous needle insertion guided by ultrasound imaging is routinely performed in hospitals today. Automating these procedures could increase placement accuracy and lower time usage of health care personnel to perform these procedures. An important step in the automation is the estimation of the needle orientation and position in the ultrasound image. One approach to estimate the needle orientation and position is to have the needle aligned with the image plane of the ultrasound probe. Aligning the needle with the plane is difficult, even with accurate measurements and calibration of both needle and probe. In this paper we propose a visual servoing method to move the ultrasound probe, using a robot to align the image plane of the probe with the needle. The method segments the needle and updates a set of visual features based on a model of the needle. A state machine is used to keep track of the alignment process, and different visual features are used to control the probe in the different states. Both simulation using simple synthetic images and experiments in a water tank are conducted to validate the proposed method. The simulation shows that the proposed method manages to align the probe plane with the needle. The alignment process is slower in the actual robot experiment, but still aligns the probe plane with the needle. The method is shown to work under simplified conditions, and is a step towards a method that could be applied in a clinical setting.

Ultrasound (US) guided biopsy is a medical procedure routinely performed in clinical practice. This task could be performed by robotic systems to improve the precision in the execution and then the safety for the patient. Both robotic and human procedures could greatly benefit from real-time localization of the needle in US images. This information could guide the robot or the specialists to the correct target point avoiding critical structures. Unfortunately US data provide very low quality images of the needle making this task quite complex, even more if you want to perform the localization on-line during the image acquisition. In this work we present a needle localization method able to extract the needle orientation and the tip position in real time from B-mode US images. To evaluate the performance of the algorithm in a precise way we use an optical tracking system to measure the position and the orientation of the needle and the US probe. In such a way the comparison is not human dependent (i.e. there are no radiologists manually selecting the needle tip) and fully repeatable. The results show an improvement in term of localization accuracy compared to previous works in literature.

A wavelet network (WN) friction model has been developed for robots where the friction is coupled, such that it is a function of the velocity of multiple joints. Wavelets have the ability to estimate random friction maps without any prior modeling while preserving linearity in the model parameters. The WN friction model was compared against the Coulomb+viscous (CV) model through experiments with the PHANTOM Omni haptic device (SensAble Technologies, MA, USA); however, the theory is valid for any serial-chain robotic manipulator. Ability to estimate applied motor torques was used as the performance metric, quantified using relative RMS values. During training of the WN model it outperformed the CV model in all cases, with an improvement in relative RMS ranging from 0.4 to 7.5 percentage points, illustrating the potential of the WN friction model. However, during testing of the WN model on an independent data set results were mixed, highlighting the challenge of achieving sufficient training.

A hybrid model/learning-based dynamic equation has been developed for robots in order to take advantage of the attractive features of both techniques. The regular dynamic equation is used to model the ideal behavior, while wavelets are used to learn the joint friction. The friction may be coupled, such that it is a function of the velocity of multiple joints. Furthermore, a system identification method has been designed to improve contact force estimation, using manual hand excitation in addition to motor excitation. The hybrid dynamic equation was implemented for the PHANTOM Omni haptic device (SensAble Technologies, MA, USA), and experiments were performed to compare using manual excitation with using motor excitation only. The results showed that manual excitation improved contact force estimation performance, with improvements in the relative RMS values ranging from 21% to 35%.