Needle insertion procedures are commonly used for diagnostic and therapeutic purposes. In this paper, an image-guided control system is developed to robotically steer flexible needles with an asymmetric tip. Knowledge about needle deflection is required for accurate steering. Two different models to predict needle deflection are presented. The first is a kinematics-based model, and the second model predicts needle deflection that is based on the mechanics of needle-tissue interaction. Both models predict deflection of needles that undergo multiple bends. The maximum targeting errors of the kinematics-based and the mechanics-based models for 110-mm insertion distance using a φ 0.5-mm needle are 0.8 and 1.7 mm, respectively. The kinematics-based model is used in the proposed image-guided control system. The control system accounts for target motion during the insertion procedure by detecting the target position in each image frame. Five experimental cases are presented to validate the real-time control system using both camera and ultrasound images as feedback. The experimental results show that the targeting errors of camera and ultrasound image-guided steering toward a moving target are 0.35 and 0.42 mm, respectively. The targeting accuracy of the algorithm is sufficient to reach the smallest lesions (φ 2 mm) that can be detected using the state-of-the-art ultrasound imaging systems.

A system is developed that can reconstruct the pose of flexible endoscopic instruments that are used in advanced flexible endoscopes using solely the endoscopic images. Four markers are placed on the instrument, whose positions are measured in the image. These measurements are compared to a three-dimensional rendered model of the instrument. The pseudo-inverse of the interaction matrix between the state of the model and the marker positions in the image is used to update the state such that the model will track the real instrument. An experiment was performed in which the instrument was moved inside a colon model, while the tip position was simultaneously measured with an electromagnetic tracking system. The root mean square errors of the position estimation were 2.3 mm, 2.2 mm and 1.7 mm in the horizontal (x), vertical (y) and away-from-camera (z) directions, respectively.

This video shows a system that estimates the pose of a flexible endoscopic instrument, based on the endoscopic images. A three-dimensional rendering of the instrument is matched to the actual instrument that is observed through the endoscopic camera. This system was evaluated in an anatomical model of a colon. The estimated position of the tip of the instrument was compared to measurements performed with an electromagnetic tracker. The errors of the position estimation were 2 mm, 2.2 mm and 1.7 mm in the horizontal (x), vertical (y) and away-from-camera (z) directions, respectively.

The goal of this study is to improve the efficiency and efficacy of the standard colonoscopy procedure. This is done by addressing the intuitiveness of colonoscope control. For this purpose an interface in the form of a grip was designed that allows the user to intuitively steer and drive the colonoscope. The Grip controls the orientation of the tip as if the colonoscope were a stiff instrument that pivots at the anus of a patient. To test the principle, experiments were conducted on a simulator operated by novice subjects. Initial experiments show a significant decrease in introduction time of 156 seconds (p<0.005). This technology will enhance current colonoscopy practice and open up possibilities for future applications of colonoscopy.

A position and orientation sensing system is developed for the feedback control of endoscopic instruments in advanced flexible endoscopes. The images that are taken by the endoscopic camera are used to match a kinematic model to the observed instrument. Using the pseudo-inverse of the Jacobian of the forward kinematics, the estimated state of the model is continuously updated so as to match feature points from the images to the model. An experiment was performed inside a colon model, in which reference markers with known locations were touched with the instrument. The root mean square position estimation errors were 1.7 mm, 1.2 mm and 3.6 mm in the horizontal (x), vertical (y), and away-from-camera (z) directions, respectively.

Passivity of virtual environments running in discrete time is a sufficient condition for stability of the system. The framework for passive sampled Port-Hamiltonian systems allows multi-dimensional virtual environments exhibiting internal dynamic behavior to be computed on a discrete medium in a passive manner. It is shown that a causality analysis is required in the framework to detect if any of the model elements have, a time dependent change of energy function in the energy balance of the system. The Standard Linear Solid model, which is often used to simulate the visco-elastic interaction with soft biological tissue is used as an example. Simulated and experimental results are provided to demonstrate the benefit of the described framework. It is shown that using this approach a multi-dimensional model which is passive in the continuous domain remains passive in the discrete domain, whereas a standard discretization approach can become non-passive.

Manually steering the tip of a flexible endoscope to navigate through an endoluminal path relies on the physician's dexterity and experience. In this paper we present the realization of a robotic flexible endoscope steering system that uses the endoscopic images to control the tip orientation towards the direction of the lumen. Two image-based control algorithms are investigated, one is based on the optical flow and the other is based on the image intensity. Both are evaluated using simulations in which the endoscope was steered through the lumen. The RMS distance to the lumen center was less than 25% of the lumen width. An experimental setup was built using a standard flexible endoscope, and the image-based control algorithms were used to actuate the wheels of the endoscope for tip steering. Experiments were conducted in an anatomical model to simulate gastroscopy. The image intensity-based algorithm was capable of steering the endoscope tip through an endoluminal path from the mouth to the duodenum accurately. Compared to manual control, the robotically steered endoscope performed 68% better in terms of keeping the lumen centered in the image.

This video shows the results of the project on the mechatronic development of the Twente humanoid head. The mechanical structure consists of a neck with four degrees of freedom (DOFs) and two eyes (a stereo pair system) which tilt on a common axis and rotate sideways freely providing a three more DOFs. The motion control algorithm is designed to receive, as an input, the output of a biological-inspired vision processing algorithm and to exploit the redundancy of the joints for the realization of the movements. The expressions of the humanoid head are implemented by projecting light from the internal part of the translucent plastic cover.