In this article, we present a novel method for self-calibrating a pan–tilt–zoom (PTZ) camera system model, specifically suited for long-range multitarget tracking with maneuvering low-cost PTZ cameras. Traditionally, such camera systems cannot provide accurate mappings from pixels to directions in the platform frame due to imprecise pan/tilt measurements or lacking synchronization between the pan/tilt unit and the video stream. Using a direction-only bundle adjustment (BA) incorporating pan/tilt measurements, we calibrate camera intrinsics, rolling shutter characteristics, and pan/tilt mechanics, and obtain clock synchronization between the video stream and pan/tilt telemetry. We call the resulting method pan/tilt camera extrinsic and intrinsic estimation (PTCEE). In a thorough simulation study, we show that the proposed estimation scheme identifies model parameters with subpixel precision across a wide range of camera setups. Leveraging the map of landmarks from the BA, we propose a method for estimating camera orientation in real-time, and demonstrate pixel-level mapping precision on real-world data. Through the proposed calibration and orientation schemes, PTCEE enables high-precision target tracking during camera maneuvers in many low-cost systems, which was previously reserved for high-end systems with specialized hardware.

This work presents a reinforcement learning-based switching control mechanism to autonomously move a ferromagnetic object (representing a milliscale robot) around obstacles within a constrained environment in the presence of disturbances. This mechanism can be used to navigate objects (e.g., capsule endoscopy, swarms of drug particles) through complex environments when active control is a necessity but where direct manipulation can be hazardous. The proposed control scheme consists of a switching control architecture implemented by two sub-controllers. The first sub-controller is designed to employ the robot’s inverse kinematic solutions to do an environment search for the to-be-carried ferromagnetic particle while being robust to disturbances. The second sub-controller uses a customized rainbow algorithm to control a robotic arm, i.e., the UR5 robot, to carry a ferromagnetic particle to a desired position through a constrained environment. For the customized Rainbow algorithm, Quantile Huber loss from the Implicit Quantile Networks (IQN) algorithm and ResNet are employed. The proposed controller is first trained and tested in a real-time physics simulation engine (PyBullet). Afterward, the trained controller is transferred to a UR5 robot to remotely transport a ferromagnetic particle in a real-world scenario to demonstrate the applicability of the proposed approach. The experimental results on the UR5 robot show an average success rate of 98.86% over 30 episodes for randomly generated trajectories, demonstrating the viability of the proposed approach for real-life applications. In addition, two classical path finding approaches, Attractor Dynamics and the execution extended Rapidly-Exploring Random Trees (ERRT), are also investigated and compared to the RL-based method. The proposed RL-based algorithm is shown to achieve performance comparable to that of the tested classical path planners whilst being more robust to deploy in dynamical environments. Note to Practitioners —Deep reinforcement learning methods have been widely applied in computer games and simulations. However, employing these algorithms for practical, real-world applications such as robotics becomes challenging due to the difficulty of obtaining training samples. This paper predominantly focuses on bridging the gap between simulations and the real-world implementation of a reinforcement learning algorithm for a robotic application in the context of miniaturized drug delivery robots and robotic capsule endoscopes. This paper presents the derivation and experimental validation of a reinforcement learning-based algorithm for controlling a magnetically-actuated small-scale robot within a simplified model of the large intestine in the presence of disturbances. We demonstrate the possibility of training a high-fidelity reinforcement learning algorithm fully within a simulated environment before deploying it as-is in a real-world scenario by carrying out different experiments and simulations. Implementing the presented control framework complements a large body of this work, and the results offer a feasibility study of using reinforcement learning algorithms in practice.

This paper establishes a physics-based simulation framework for steering a magnetically actuated guidewire based on the linear elasticity and dipoles theories. Interaction wrenches resulting from an external magnetic field and embedded magnets in a continuum rod, i.e., guidewire, serves as actuators for steering. In the presented framework, a simplified integration scheme based on the finite-volume method is employed to model guidewire using the linear elasticity theory and forces resulting from the interference of magnetic fields to provide a rapid model reconstruction. Furthermore, orienting the external magnetic field is employed to steer a guidewire into a constrained environment. Finally, simulations illustrate the approach performance on a soft rod where an external magnetic field is orientated to form the desired shape for a continuum rod and steer it within an environment. The results open up possibilities to construct a rapid model for continuum manipulators in practice.

Percutaneous needle insertion guided by ultrasound imaging is routinely performed in hospitals today. Automating these procedures could increase placement accuracy and lower time usage of health care personnel to perform these procedures. An important step in the automation is the estimation of the needle orientation and position in the ultrasound image. One approach to estimate the needle orientation and position is to have the needle aligned with the image plane of the ultrasound probe. Aligning the needle with the plane is difficult, even with accurate measurements and calibration of both needle and probe. In this paper we propose a visual servoing method to move the ultrasound probe, using a robot to align the image plane of the probe with the needle. The method segments the needle and updates a set of visual features based on a model of the needle. A state machine is used to keep track of the alignment process, and different visual features are used to control the probe in the different states. Both simulation using simple synthetic images and experiments in a water tank are conducted to validate the proposed method. The simulation shows that the proposed method manages to align the probe plane with the needle. The alignment process is slower in the actual robot experiment, but still aligns the probe plane with the needle. The method is shown to work under simplified conditions, and is a step towards a method that could be applied in a clinical setting.

Ultrasound (US) guided biopsy is a medical procedure routinely performed in clinical practice. This task could be performed by robotic systems to improve the precision in the execution and then the safety for the patient. Both robotic and human procedures could greatly benefit from real-time localization of the needle in US images. This information could guide the robot or the specialists to the correct target point avoiding critical structures. Unfortunately US data provide very low quality images of the needle making this task quite complex, even more if you want to perform the localization on-line during the image acquisition. In this work we present a needle localization method able to extract the needle orientation and the tip position in real time from B-mode US images. To evaluate the performance of the algorithm in a precise way we use an optical tracking system to measure the position and the orientation of the needle and the US probe. In such a way the comparison is not human dependent (i.e. there are no radiologists manually selecting the needle tip) and fully repeatable. The results show an improvement in term of localization accuracy compared to previous works in literature.

We have built a test unit containing a GPS, an inertial measurement unit, magnetometers and a barometer. When the GPS looses its signals, the other sensors are used to keep track of the position. Unscented Kalman filters have been used to test the performance of different aid sensor configurations. In addition to the four different sensor configurations, the gyroscope output was filtered, making it eight different test scenarios. Simulations show that filtering the gyroscope output and using inertial sensors in addition to a barometer had the lowest position error, with a standard deviation of the position of 8.5 m. For unfiltered gyroscope output the best result was obtained when using all sensors. This had a standard deviation of the position of approximatly 42 m. The preliminary real world test results had a much larger error than the simulation.