Contact force Jacobian relates the changes in the contact force to the changes in the actuation of a robotic catheter in contact with a surface. In this paper, we present an analytical method for calculating the contact force Jacobian for the Cosserat rod model of an MRI-actuated robotic catheter. First, the Cosserat rod model of the MRI-actuated robotic catheter under tip contact position constraint is introduced. For the analytical derivation of contact force Jacobian, the initial value problem parameter derivatives are defined and calculated analytically. Finally, simulation results show that the presented analytical method calculates the contact force Jacobian in significantly shorter computation time with comparable accuracy, compared to direct numerical computation.

In this work, an efficient homology guided belief space planning method for obstacle-cluttered environments is presented. The proposed planner follows a two-step approach. First, a h-signature guided rapidly-exploring random tree (HRRT) algorithm is proposed to provide nominal trajecto-ries in different homology classes by constructing homology aware sub-trees in a parallel manner. The HRRT planner is extended to a h-signature guided RRT*algorithm, where an inter-homology-class rewire procedure is proposed, increasing the probability of discovering homology classes in narrow space/passages. The iLQG-based belief space planning algorithm is then employed to find locally optimal trajectories minimizing uncertainties in each homology class.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

This paper proposes a path correction method for surgical robotic systems performing needle handoff manipulations as part of autonomous execution of surgical suturing. During handoff motions, the position and orientation of the needle is subject to perturbations from the idealized planned pose due to uncertainties in camera-robot calibration and needle localization. If, after a perturbation, the system needs to perform subsequent needle regrasp(s), but the robot still follows the originally planned trajectory out of the path planner [1], it has a lower chance of gripping the needle properly. In order to accommodate these unpredictable needle pose perturbations, the proposed path correction method works locally to direct the needle from the wrong pose to the original pose by partial replanning of the robot motion. The reliability of the proposed method is evaluated with three sets of experiments in a simulation environment.

Contact force quality is one of the most critical factors for safe and effective lesion formation during cardiac ablation. The contact force and contact stability plays important roles in determining the lesion size and creating a gap-free lesion. In this paper, the contact stability of a novel magnetic resonance imaging (MRI)-actuated robotic catheter under tissue surface motion is studied. The robotic catheter is modeled using a pseudo-rigid-body model, and the contact model under surface constraint is provided. Two contact force control schemes to improve the contact stability of the catheter under heart surface motions are proposed and their performance are evaluated in simulation.

This paper studies the contact stability and contact safety of a robotic intravascular cardiac catheter under blood flow disturbances while in contact with tissue surface. A probabilistic blood flow disturbance model, where the blood flow drag forces on the catheter body are approximated using a quasi-static model, is introduced. Using this blood flow disturbance model, probabilistic contact stability and contact safety metrics, employing a sample based representation of the blood flow velocity distribution, are proposed. Finally, the contact stability and contact safety of a MRI-actuated robotic catheter are analyzed using these models in a specific example scenario under left pulmonary inferior vein (LIV) blood flow disturbances.

In magnetic resonance imaging (MRI) guided robotic catheter ablation procedures, reliable tracking of the catheter within the MRI scanner is needed to safely navigate the catheter. This requires accurate registration of the catheter to the scanner. This paper presents a differential, multi-slice image-based registration approach utilizing active fiducial coils. The proposed method would be used to preoperatively register the MRI image space with the physical catheter space. In the proposed scheme, the registration is performed with the help of a registration frame, which has a set of embedded electromagnetic coils designed to actively create MRI image artifacts. These coils are detected in the MRI scanner's coordinate system by background subtraction. The detected coil locations in each slice are weighted by the artifact size and then registered to known ground truth coil locations in the catheter's coordinate system via least-squares fitting. The proposed approach is validated by using a set of target coils placed withing the workspace, employing multi-planar capabilities of the MRI scanner. The average registration and validation errors are respectively computed as 1.97 mm and 2.49 mm. The multi-slice approach is also compared to the single-slice method and shown to improve registration and validation by respectively 0.45 mm and 0.66 mm.

The development of autonomous or semi-autonomous surgical robots stands to improve the performance of existing teleoperated equipment, but requires fine hand-eye calibration between the free-moving endoscopic camera and patient-side manipulator arms (PSMs). A novel method of solving this problem for the da Vinci® robotic surgical system and kinematically similar systems is presented. First, a series of image-processing and optical-tracking operations are performed to compute the coordinate transformation between the endoscopic camera view frame and an optical-tracking marker permanently affixed to the camera body. Then, the kinematic properties of the PSM are exploited to compute the coordinate transformation between the kinematic base frame of the PSM and an optical marker permanently affixed thereto. Using these transformations, it is then possible to compute the spatial relationship between the PSM and the endoscopic camera using only one tracker snapshot of the two markers. The effectiveness of this calibration is demonstrated by successfully guiding the PSM end effector to points of interest identified through the camera. Additional tests on a surgical task, namely grasping a surgical needle, are also performed to validate the proposed method. The resulting visually-guided robot positioning accuracy is better than the earlier hand-eye calibration results reported in the literature for the da Vinci® system, while supporting intraoperative update of the calibration and requiring only devices that are already commonly used in the surgical environment.

This paper presents algorithms for three-dimensional tracking of surgical needles using the stereo endoscopic camera images obtained from the da Vinci® Surgical Robotic System. The proposed method employs Bayesian state estimation, computer vision techniques, and robot kinematics. A virtual needle rendering procedure is implemented to create simulated images of the surgical needle under the da Vinci ® robot endoscope, which makes it possible to measure the similarity between the rendered needle image and the real needle. A particle filter algorithm using the mentioned techniques is then used for tracking the surgical needle. The performance of the tracking is experimentally evaluated using an actual da Vinci® surgical robotic system and quantitatively validated in a ROS/Gazebo simulation thereof.

An MRI-actuated catheter is a novel robotic catheter system that utilizes the MR scanner for both remote steering and catheter tracking. In order to develop the mathematical model and the planning algorithm of the catheter in parallel to the MR tracking system, an alternative catheter tracking method is needed. This paper presents a catheter tracking algorithm based on the particle filter and the catadioptric camera system. The motion model of the particle filter is based on the quasi-static kinematics of the catheter. The measurement model calculates the weights of the particles according to the normalized cross-correlation of the segmented image from camera and a virtual rendering of the catheter. The efficacy of the tracking algorithm is demonstrates via experimental results.

This paper presents an approach to surgical tool tracking using stereo vision for the da Vinci®Surgical Robotic System. The proposed method is based on robot kinematics, computer vision techniques and Bayesian state estimation. The proposed method employs a silhouette rendering algorithm to create virtual images of the surgical tool by generating the silhouette of the defined tool geometry under the da Vinci®robot endoscopes. The virtual rendering method provides the tool representation in image form, which makes it possible to measure the distance between the rendered tool and real tool from endoscopic stereo image streams. Particle Filter algorithm employing the virtual rendering method is then used for surgical tool tracking. The tracking performance is evaluated on an actual da Vinci®surgical robotic system and a ROS/Gazebo-based simulation of the da Vinci®system.

This paper presents a free-space open-loop dynamic response analysis for an MRI -guided magnetically-actuated steerable intra-vascular catheter system. The catheter tip is embedded with a set of current carrying micro-coils. The catheter is directly actuated via the magnetic torques generated on these coils by the magnetic field of the magnetic resonance imaging (MRI)scanner. The relationship between the input current commands and catheter tip deflection angle presents an inherent nonlinearity in the proposed catheter system. The system nonlinearity is analyzed by utilizing a pendulum model. The pendulum model is used to describe the system nonlinearity and to perform an approximate input-output linearization. Then, a black-box system identification approach is performed for frequency response analysis of the linearized dynamics. The optimal estimated model is reduced by observing the modes and considering the Nyquist frequency of the camera system that is used to track the catheter motion. The reduced model is experimentally validated with 3D open-loop Cartesian free-space trajectories. This study paves the way for effective and accurate free-space closed-loop control of the robotic catheter with real-time feedback from MRI guidance in subsequent research.

In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently, using a surgical robot can be challenging, fatiguing, and time-consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both the patient's outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper presents novel stereo image processing algorithms for the detection, initialization, and tracking of a surgical suture thread. A nonuniform rational B-spline (NURBS) curve is used to model a thin, deformable, and dynamic length thread. The NURBS model is initialized and grown from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image matching energy between the projected stereo NURBS image and the segmented thread image. The algorithms are evaluated using suture threads, a calibrated test pattern, and a simulated thread image. In addition, the accuracies of the algorithms presented are validated as they track a suture thread undergoing translation, deformation, and apparent length changes. All of the tracking is in real time.

An MRI-actuated catheter is a novel robotic catheter system that utilizes the MRI for both remote steering and visualization for catheter ablation of atrial fibrillation. Planning and control of the catheter requires a sufficiently fast yet accurate model of the catheter. The pseudo-rigid-body (PRB) model offers a reasonable trade-off between speed and accuracy by approximating the continuum catheter as rigid links connected by flexible joints, thus reducing the infinite degrees of freedom of the continuum mechanism to a finite one. In this paper, a PRB model of the MRI-actuated catheter is validated experimentally by comparing the deflections of the PRB model with the deflections of the catheter prototype.

This paper presents an approach to supervisory control of a DaVinci surgical robot. At present, such robots are controlled by teleoperation, with dissimilar kinematics of the operator interface vs. the robot. As a result, it can be difficult for the operator to visualize the kinematic restrictions on the robot, particularly for desired extended, precise trajectories, such as circular needle driving. The interface presented here constitutes a means to elevate the operator from teleoperation mode to supervisory mode. The operator interacts directly with a point-cloud display, allowing selection of task specifications from which the system automatically computes and executes precise trajectories to achieve the task goals. The intent is to allow the operator to focus on task specifications and rely on automation to achieve faster and more precise execution.

The recent introduction of Magnetic Resonance Imaging (MRI)-actuated steerable catheters lays the ground work for increasing the efficacy of cardiac catheter procedures. The MRI, while capable of imaging the catheter for tracking and control, does not fulfill all of the needs required to identify and develop a complete catheter model. Specifially, the frequency response of the catheter must be identified to ensure stable control of the catheter system. This requires a higher frequency imaging than the MRI can achieve. This work uses a catadioptric stereo camera system consisting of a mirror and a single camera in order to track a MRI actuated catheter inside a MRI machine. The catadioptric system works in parallel to the MRI and is capable of recording the catheter at 60 fps for post processing. The accuracy of the catadioptric system is verified in imaging conditions that would be found inside the MRI. The stereo camera is then used to track a catheter as it is actuated inside the MRI.

In this paper, a new task-oriented active-sensing method is presented. Most active sensing methods choose sensing actions that minimize the uncertainty of the state according to some information-theoretic measure. While this is reasonable for most applications, minimizing state uncertainty may not be most relevant when the state information is used to perform a task. This is because the uncertainty in some subspace of the state space could have more impact on the performance of the task than the others at a given time. The active-sensing method presented in this paper takes the task into account when selecting sensing actions by minimizing the uncertainty in future task action.

Robotically Assisted Minimally Invasive Surgery (RAMIS) offers many advantages over manual surgical techniques. Most of the limitations of RAMIS stem from its non-intuitive user interface and costs. One way to mitigate some of the limitations is to automate surgical subtasks (e.g. suturing) such that they are performed faster while allowing the surgeon to plan the next step of the procedure. One component of successful suture automation is minimizing the internal tissue deformation forces generated by driving a needle through tissue. Minimizing the internal tissue forces requires segmenting the tissue deformation forces from other components of the needle tissue interaction (e.g. friction force). This paper proposes an Unscented Kalman Filter which can successfully model the force components, in particular the internal deformation force, generated by a needle as it is driven through a sample of tissue.

This paper presents algorithms for selection of needle grasp and for selection of entry ports of robotic instruments, for autonomous robotic execution of the minimally invasive surgical suturing task. A critical issue for automatic execution of surgical tasks, such as suturing, is the choice of needle grasp for the robotic system. Inappropriate needle grasp increases operating time requiring multiple regrasps to complete the desired task. In robotic minimally invasive surgery, the entry port that the surgical robot goes through into the patient's body has a significant role on the performance of the robot. Improper entry port affects the robot's dexterity, manipulability and reachability. The proposed methods use manipulability, dexterity and torque metrics for needle grasp selection, and employ needle grasp robustness and target location robustness metrics for port selection. The results of a case study simulation in thoracoscopic surgery is also presented to demonstrate the proposed methods. This paper is motivated by the problem of automating low-level surgical tasks in robotic surgery, such as, suturing, retraction, dissection, and providing exposure. Specifically, this paper focuses on needle grasp and entry port selection for automating robotic surgical suturing. Selection of an appropriate way of grasping a needle is critical for successfully and robustly completing autonomous suturing. To the best authors' knowledge, there are no earlier studies in the literature which focus on the needle grasp selection problem. The proposed approach determines how to grasp the needle by optimizing the surgical system's manipulation performance. The existing approaches in the literature for selecting entry ports for the robotic surgical tools only consider the teleoperated robotic minimally invasive surgery, in which the surgeons directly control the robotic instruments. However, automated performance of suturing introduces additional challenges due to uncertainties in needle localization and grasping. This paper proposes two new performance metrics on selecting port locations from the perspective of autonomously performing surgical suturing, without direct involvement of the human user. This paper also presents preliminary experiments which demonstrate the effectiveness of the proposed methods.

This paper presents a kinematic study of a pseudorigid-body model (PRBM) of MRI-compatible, magnetically actuated, steerable catheters. It includes a derivation of a mathematical model of the PRBM of the catheter, singularity studies of the model, and a new manipulability measure. While the forward kinematics of the model presented here is applicable to PRBMs for other applications, actuation method is unique to the particular design. Hence, a careful study of singularities and manipulability of the model is required. The singularities are studied from the underlying equations of motion with intuitive interpretations. The proposed manipulability measure is a generalization of the inverse condition number manipulability measure of robotic manipulators. While the PRBM is an approximation of the flexible catheter, kinematic studies of the PRBM still provide some insight into feasibility and limitations of the catheter, which is beneficial to the design and motion planning of the catheter.

In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently using a surgical robot can be challenging, fatiguing, and time consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both patient outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper proposes an algorithm which uses a Non-Uniform Rational B-Spline (NURBS) curve to model a suture thread. The NURBS model is initialized from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image match energy between the projected stereo NURBS image and the segmented thread image. The algorithm is able to accurately track a suture thread as it translates, deforms, and changes length in real-time.

This paper presents algorithms for optimal selection of needle grasp, for autonomous robotic execution of the minimally invasive surgical suturing task. In order to minimize the tissue trauma during the suturing motion, the best practices of needle path planning that are used by surgeons are applied for autonomous robotic surgical suturing tasks. Once an optimal needle trajectory in a well-defined suturing scenario is chosen, another critical issue for suturing is the choice of needle grasp for the robotic system. Inappropriate needle grasp increases operating time requiring multiple re-grasps to complete the desired task. The proposed methods use manipulability, dexterity and torque metrics for needle grasp selection. A simulation demonstrates the proposed methods and recommends a variety of grasps. Then a realistic demonstration compares the performances of the manipulator using different grasps.

This paper presents the three dimensional kinematic modeling of a novel steerable robotic ablation catheter system. The catheter, embedded with a set of current-carrying micro-coils, is actuated by the magnetic forces generated by the magnetic field of the MRI scanner. This paper develops a 3D model of the MRI actuated steerable catheter system by using finite differences approach. For each finite segment, a quasi-static torque-deflection equilibrium equation is calculated using beam theory. By using the deflection displacements and torsion angles, the kinematic modeling of the catheter system is derived. The proposed models are evaluated by comparing the simulation results of the proposed model with the experimental results of a proof-of-concept prototype.

This paper presents a motion planning algorithm for Magnetic Resonance Imaging (MRI) actuated catheters for catheter ablation of atrial fibrillation. The MRI-actuated catheters is a new robotic catheter concept which utilizes MRI for remote steering and guidance. Magnetic moments generated by a set of coils wound near the tip are used to steer the catheter under MRI scanner magnetic field. The catheter during an ablation procedure is modeled as a constrained robotic manipulator with flexible joints, and the proposed motion-planning algorithm calculates a sequence of magnetic moments based on the manipulator model to move the tip of the catheter along a predefined trajectory on the surface of the left atrium. The difficulties in motion planning of the catheter are due to kinematic redundancy and underactuation. The proposed motion planning algorithm overcomes the challenges by operating in the task space instead of the configuration space. The catheter is then regulated around this nominal trajectory using feedback control to reduce the effect of uncertainties.

Adverse and anomalous (A&A) events are a serious concern in medical robots. We describe a system that can rapidly detect such events and predict their occurrence. As part of this system, we describe simulation, data collection and user interface tools we build for a robot for small animal biopsies. The data we collect consists of both the hardware state of the robot and variables in the software controller. We use this data to train dynamic Bayesian network models of the joint hardware-software state-space dynamics of the robot. Our empirical evaluation shows that (i) our models can accurately model normal behavior of the robot, (ii) they can rapidly detect anomalous behavior once it starts, (iii) they can accurately predict a future A&A event within a time window of it starting and (iv) the use of additional software variables beyond the hardware state of the robot is important in being able to detect and predict certain kinds of events.

This paper develops a path plan for suture needles used with solid tissue volumes in endoscopic surgery. The path trajectory is based on the best practices that are used by surgeons. The path attempts to minimize the interaction forces between the tissue and the needle. Using surgical guides as a basis, two different techniques for driving a suture needle are developed. The two techniques are compared in hardware experiments by robotically driving the suture needle using both of the motion plans.

Robotic-assisted beating heart surgery aims to allow surgeons to operate on a beating heart without stabilizers as if the heart is stationary. The robot actively cancels heart motion by closely following a point of interest (POI) on the heart surface - a process called active relative motion canceling. Due to the high bandwidth of the POI motion, it is necessary to supply the controller with an estimate of the immediate future of the POI motion over a prediction horizon in order to achieve sufficient tracking accuracy. In this paper, two least-squares-based prediction algorithms, using an adaptive filter to generate future position estimates, are implemented and studied. The first method assumes a linear system relation between the consecutive samples in the prediction horizon. On the contrary, the second method performs this parametrization independently for each point over the whole the horizon. The effects of predictor parameters and variations in heart rate on tracking performance are studied with constant and varying heart rate data. The predictors are evaluated using a three-degree-of-freedom (DOF) test bed and prerecorded in vivo motion data. Then, the one-step prediction and tracking performances of the presented approaches are compared with an extended Kalman filter predictor. Finally, the essential features of the proposed prediction algorithms are summarized.

Robotic motion planning algorithms used for task automation in robotic surgical systems rely on availability of accurate models of target soft tissue's deformation. Relying on generic tissue parameters in constructing the tissue deformation models is problematic; because, biological tissues are known to have very large (inter- and intra-subject) variability. A priori mechanical characterization (e.g., uniaxial bench test) of the target tissues before a surgical procedure is also not usually practical. In this paper, a method for estimating mechanical parameters of soft tissue from sensory data collected during robotic surgical manipulation is presented. The method uses force data collected from a multiaxial force sensor mounted on the robotic manipulator, and tissue deformation data collected from a stereo camera system. The tissue parameters are then estimated using an inverse finite element method. The effects of measurement and modeling uncertainties on the proposed method are analyzed in simulation. The results of experimental evaluation of the method are also presented.

This paper presents a model of needle tissue interaction forces that a rigid suture needle experiences during surgical suturing. The needle-tissue interaction forces are modeled as the sum of lumped parameters. The model has three main components; friction, tissue compression, and cutting forces. The tissue compression force uses the area that the needle sweeps out during a suture to estimate both the force magnitude and force direction. The area that the needle sweeps out is a direct result of driving the needle in a way that does not follow the natural curve of the needle. The friction force is approximated as a static friction force along the shaft of the needle. The cutting force acts only on the needle tip. The resulting force and torque model is experimentally validated using a tissue phantom. These results indicate that the proposed lumped parameter model is capable of accurately modeling the forces experienced during a suture.

In robotic assisted beating heart surgery, the goal is to develop a robotic system that can actively cancel heart motion by closely following a point of interest (POI) on the heart surface, a process called Active Relative Motion Canceling (ARMC). In order to track and cancel POI motion precisely, control algorithms require good quality heart motion data. In this paper, a novel method is described which uses a particle filter to estimate the three-dimensional location of POI on heart surface by using measurements obtained from sonomicrometry along with an accelerometer. The new method employs a differential probability approach to increase the accuracy of the particle filter. The performance of the proposed method is evaluated by simulations.

Haptic interface research benefits from accurate human arm models for control and system design. The literature contains many human arm dynamic models but lacks detailed variability analyses. Without accurate measurements, variability is modeled in a very conservative manner, leading to less than optimal controller and system designs. This paper not only presents models for human arm dynamics but also develops inter- and intrasubject variability models for a stylus-based haptic device. Data from 15 human subjects (nine male, six female, ages 20-32) were collected using a Phantom Premium 1.5a haptic device for system identification. In this paper, grip-force-dependent models were identified for 1-3-N grip forces in the three spatial axes. Also, variability due to human subjects and grip-force variation were modeled as both structured and unstructured uncertainties. For both forms of variability, the maximum variation, 95 %, and 67 % confidence interval limits were examined. All models were in the frequency domain with force as input and position as output. The identified models enable precise controllers targeted to a subset of possible human operator dynamics.

Telerobotics is one of the most traditional fields of robotics and it played a crucial role in the history of robotics and of the mankind, especially in the areas of space and undersea exploration and of remote material handling. On the other hand, teleoperation is still a very active research area and many problems are still open. In particular, the design of the control strategy for coupling local and remote site is of paramount importance for implementing telepresence, namely the feeling of being directly interacting with the remote environment. The IEEE RAS Technical Committee on Telerobotics would like to propose a half-day tutorial for illustrating several successful control strategies for implementing high performance bilateral teleoperation systems.

Given the ease that humans have with using a keyboard and mouse in typical, non-colocated computer interaction, many studies have investigated the value of colocating the visual field and haptic workspaces using immersive virtual reality (VR) modalities. Significant understanding has been gained by previous work comparing physical tasks against VR tasks, visuo-haptic co-location versus non-colocation, and even visuo-haptic rotational misalignments in VR. However, few studies have explored all of these paradigms in context with each other and it is difficult to do inter-study comparisons because of the variation in tested motor tasks. Therefore, the goal for the current study was to characterize human performance of Fitts' point-to-point reaching task - an established test of manual performance - in the physical, co-located/non-colocated VR, and rotated VR visualization conditions. A key finding was the significant decrease observed in end-point error for tasks performed in a co-located virtual reality environment. The results also showed cyclic performance degradations due to rotational visuo-haptic misalignments that were consistent with trends reported by the literature.

Human-computer/machine interface research benefits from accurate human arm models for stability analysis, control, and system design. The current study developed models for human arm dynamics and variability specific to stylus-based kinesthetic haptic interfaces. Data from nine human subjects (5 male, 4 female, ages 20–30) were collected using a three degree-of-freedom haptic device in the X, Y, and Z axes along with a range of grip forces (1–3N) for parametric system identification of the human arm and hand. Variability models that accounted for subject and grip force variation were also identified. The arm and hand model structure consisted of a third-order linear parametric transfer function that was paired with a previously derived second-order model for the haptic robot. The variability was modeled as multiplicative unstructured uncertainty using transfer functions. All of the model parameters were identified in the frequency domain and have force as input and position as output.

In bilateral teleoperation system, conventional position sensor based force reflection method, known as position error feedback, may generate inaccurate force reflection directions, when motion of a slave robot is constrained by unexpected obstacles and link collisions. Restriction Space Projection method is a novel position sensor based force reflection framework that was proposed to address this issue. It provides accurate force reflection in unstructured environments when motion of a slave robot is constrained by unexpected obstacles and link collisions, regardless of kinematic dissimilarity between the master and slave manipulators of the bilateral teleoperation system. This paper discusses the applications and limitations of the Restriction Space Projection method through examples.

We are developing a personal micronavigation system that uses high-resolution gait-corrected inertial measurement units. The goal of this project is to develop a navigation system that use secondary inertial variables, such as velocity, to enable long-term precise navigation in the absence of Global Positioning System (GPS) and beacon signals. In this scheme, measured zero velocity durations from the ground reaction sensors are used to reset the accumulated integration errors from the accelerometers and gyroscopes in position calculation. We achieved an average position error of 4 meters at the end of half-hour walks.

In biomedical research it is difficult to perceive tumors or cells and perform biopsies manually. Robotics technology can offer a reliable solution for accurate needle insertion. A novel 5 degrees of freedom (DOF) robot for inserting needles into small animal subjects was developed. The robot can realize dexterous alignment of the needle using two parallel mechanisms, and has a syringe mechanism to insert needles to subjects. Operations on small animals require high accuracy positioning during needle insertion. In this paper, kinematic calibration of the 5 DOF robot using an optical tracker as an external sensor is performed to enhance accuracy of the system.

This paper investigates a novel position-sensor-based force reflection framework for multi-degree-of-freedom (DOF) bilateral teleoperation systems in unstructured environments. The conventional position-sensor-based force reflection method, which is known as position error feedback, may generate grossly inaccurate force reflection directions during collisions involving the slave manipulator links. The proposed restriction space projection framework calculates the instantaneous restriction space to provide the accurate force reflection, regardless of kinematic dissimilarity (KDS) conditions of bilateral teleoperation systems. Simulation results confirmed the validity of the proposed framework in a KDS bilateral teleoperation system under various constraint conditions.

In order to perform coronary artery bypass graft surgery, a stationary heart is necessary. A human cannot achieve manual tracking of the complex heartbeat motion. Robotics technology can overcome such limitations. In the robotic-assisted beating heart surgery, the robot actively cancels heart motion by closely following a point of interest on the heart surface-a process called active relative motion canceling. As a result, surgeon can operate on the beating heart as if it is stationary. In this paper, a generalized estimation algorithm, that uses an adaptive filter to generate future position estimates is studied. The predictor is parameterized on-line and adaptively to minimize the prediction error in the mean-square sense. The predictor is evaluated using a 3-degree- of-freedom test-bed system and prerecorded heart motion data.

In the robotic-assisted off-pump coronary artery bypass graft (CABG) surgery, surgeon performs the operation with intelligent robotic instruments controlled through teleoperation that replace conventional surgical tools. The robotic tools actively cancel the relative motion between the surgical instruments and the point of interest on the beating heart. Measuring the motion of the heart during this operation is an important part of this scheme. In this paper, a novel whisker sensor design to measure the heart motion in three dimensions (3D) is presented. The proposed whisker sensor is a flexible contact sensor. Low stiffness of the sensor prevents damage on the tissue it contacts. This paper explains the design concept, and reports the simulation and measurement results of the prototype whisker position sensor

An increasingly common new modality in human-computer interaction is haptic interfacing, especially in the field of medical simulation. The order-of-magnitude difference in update rates between graphical deformable object simulations and haptic interfaces can be bridged using local low-order approximations. However, providing force feedback using local models complicates collision detection and response with the virtual tool, since the user interacts with lower-order proxies rather than the full simulated objects. A novel approach focusing on rolling contact with stick-slip friction is presented where all collision detection and response with the virtual tool is performed at the local level at the haptic time-scale, utilizing linearized low-order local models that approximate the behavior of the full model for the short time steps and small deformations involved.

This paper presents a quantitative comparison framework for bilateral teleoperation systems which have different dynamic characteristics and sensory configurations for a given task dependant performance objective, mu-synthesis is used to develop the framework since it can efficiently treat systems containing uncertainties and disturbances. The framework consists of i) a feasibility test, and ii) a comparison methodology using prioritized task dependent performance objectives. This framework is applied to a bilateral teleoperation system including an uncertain human operator and environment in a practical case study. The validity of the proposed quantitative framework is confirmed through experiments. The proposed framework can be used as a tool to design bilateral teleoperation systems, especially when there are constraints in designing drive mechanisms and choosing sensory configurations.

Robot assisted heart surgery allows surgeons to operate on a heart while it is still beating as if it had been stopped. The robot actively cancels heart motion by closely following a point of interest (POI) on the heart surface -- a process called active relative motion canceling (ARMC). Due to the high bandwidth of the POI motion, it is necessary to supply the controller with an estimate of the immediate future of the POI over a prediction horizon. In this paper, a prediction algorithm, using an adaptive filter to generate future position estimates, is implemented and studied. The effects of predictor parameters on tracking performance are studied. Finally, the predictor is evaluated using a 3 degrees of freedom test-bed and prerecorded heart motion data.

This paper focuses on the development of control algorithms for intelligent robotic tools that assist off-pump coronary artery bypass graft (CABG) surgery. In the robotic-assisted CABG surgery, the surgeon operates on the beating heart using intelligent robotic instruments. Robotic tools actively cancel the relative motion between the surgical instruments and the point of interest on the beating heart, dynamically stabilizing the heart for the operation. This algorithm is called active relative motion canceling (ARMC). Here, a model-based intelligent ARMC algorithm employing biological signals, such as electrocardiogram, to achieve effective motion cancellation is proposed. Finally, experimental results of the algorithm on a 3-degree-of-freedom robotic test-bed system are reported.

This paper presents a quantitative comparison framework for bilateral teleoperation systems (BTSs) that have different dynamic characteristics and sensory configurations for a given task-dependent performance objective (TDPO).$\mu$-synthesis is used to develop the framework since it can efficiently treat systems containing uncertainties and disturbances. The framework consists of: 1) a feasibility test and 2) a comparison methodology using prioritized TDPOs. As the formulation used is based on$\mu$-synthesis, the system, operator, and environment models are represented in the form of linear nominal models with frequency-dependent multiplicative uncertainties. This framework is applied to a BTS including an uncertain human operator and environment in a practical case study. The validity of the proposed quantitative framework is confirmed through experiments. The proposed framework can be used as a tool to design BTSs, especially when there are constraints in designing drive mechanisms and choosing sensory configurations.

Robotics technology promises an enhanced way of performing off-pump coronary artery bypass graft (CABG) surgery. In the robotic-assisted CABG surgery, surgeon performs the operation with intelligent robotic instruments controlled through teleoperation in place of conventional surgical tools. The robotic tools actively cancel the relative motion between the surgical instruments and the point-of-interest on the beating heart, in contrast to traditional off-pump CABG where the heart is passively constrained to dampen the beating motion. As a result, the surgeon operates on the heart as if it were stationary. This algorithm is called active relative motion canceling (ARMC). In this paper, the use of biological signals, such as electrocardiogram (ECG), to achieve better motion canceling in the model-based intelligent ARMC algorithm is proposed. An ECG contains records for the electrical activity of the heart, which forms a series of waves and complexes. Real time identification of these waves and complexes improve the estimation of the future heart motion and improve the performance of the ARMC algorithm. Finally, the experimental results of the algorithm implemented on a 3-DOF robotic test-bed system are reported

It is unknown whether electroencephalography (EEG) signal characteristics in stroke survivors with motor deficits register enough activity for use with brain-computer interfaces (BCIs). This research studied pre-movement EEG from shoulder-elbow movement in stroke survivors to identify signal characteristics potentially useful for robot-assisted stroke rehabilitation. Pre-movement event-related desynchronization (ERD) was examined in the alpha band mu rhythm for control (n = 7) and stroke subjects (n = 11). Subjects were all right-hand dominant; stroke subjects used their impaired arm and controls were assigned a side to match stroke subjects. Both non-dominant-arm-tested stroke and control subjects exhibited greater ERD intensity vs. those using their dominant arm (p < 0.05). Also, pre-movement ERD was detected in stroke survivors, which suggests at the possibility of using ERD as a BCI system control signal. However, the peak ERD of stroke survivors was significantly lower than that of healthy subjects (p < 0.05), which brings doubt to whether the intensity of ERD in stroke survivors is large enough to be used as a BCI system control signal

This paper presents the architectural details of an evolving open source/open architecture software framework for developing organ-level surgical simulations. Our goal is to facilitate shared development of reusable models, to accommodate heterogeneous models of computation, and to provide a framework for interfacing multiple heterogeneous models. The framework provides an application programming interface for interfacing dynamic models defined over spatial domains. It is specifically designed to be independent of the specifics of the modeling methods used, and therefore facilitates seamless integration of heterogeneous models and processes. Furthermore, each model has separate geometries for visualization, simulation, and interfacing, allowing the model developer to choose the most natural geometric representation for each case. Input/output interfaces for visualization and haptics for real-time interactive applications have also been provided.

Since teleoperation systems are mostly executed in the extreme environment, there are constraints in designing the mechanism and choosing sensors. This paper presents a novel quantitative comparison method of teleoperators based on H/sub /spl infin// framework. The upper H/sub /spl infin// norm bound of the system including H/sub /spl infin// sub optimal controller is used as the performance index. As a case study, the method is applied to a real teleoperation system to study the effects of sensory configuration and back-drivability of the mechanism on the performance of the system in tasks, which involve different environment impedances. It can be important criteria to design a teleoperator from the control point of view.

An important area of research in the teleoperation literature is to develop systematic methods to quantitatively compare different manipulator designs in application critical tasks. Such quantitative methods are especially important during design of the manipulators to make an informed decision among various design alternatives. In this paper, a novel method to quantitatively compare different sensory schemes for a teleoperation system is introduced. This method evaluates the sensory schemes by comparing the norm of the a posteriori error covariance matrices of the Kalman filters for each configuration. The main advantage of this method is that it allows to quantitatively compare arbitrary sensory configurations.

In this letter, teleoperation controller design for haptic exploration and telemanipulation of soft environments is studied. First, a new measure for fidelity in teleoperation is introduced which quantifies the teleoperation system's ability to transmit changes in the compliance of the environment. This sensitivity function is appropriate for the application of telesurgery, where the ability to distinguish small changes in tissue compliance is essential for tasks such as detection of embedded vessels. The bilateral teleoperation controller design problem is then formulated in a task-based optimization framework as the optimization of this metric, with constraints on free-space tracking and robust stability of the system under environment and human operator uncertainties. The control design procedure is illustrated with a case study. The analysis is also used to evaluate the effectiveness of using a force sensor in a teleoperation system.

Previous research on teleoperation has focused on manipulation of hard objects. However, the design constraints are different in applications that involve manipulation of deformable objects, such as robotic telesurgery. In this paper a new measure for fidelity in teleoperation is introduced which quantifies the teleoperation system's ability to transmit changes in the compliance of the environment. This sensitivity function is highly appropriate for the application of telesurgery, where the ability to distinguish small changes in tissue compliance is essential for tasks such as tumor detection. The bilateral teleoperation controller design problem is then formulated as the optimization of this new metric with constraints on free space tracking requirements and robust stability of the system under environment and human operator uncertainties. The robust stability analysis can be applied to any teleoperator plant and guarantee stability given an uncertainty model. The analysis is also extended to evaluate effectiveness of using a force sensor in the teleoperation system.

An important missing piece in the medical robotics literature is the lack of systematic methods to quantitatively compare different manipulator designs, and to evaluate kinematic configurations chosen for telesurgical manipulators in application-critical tasks. Such a quantitative method is especially important during design stage to make an informed decision between various design alternatives. In the paper, a quantitative method to evaluate the kinematic ability of surgical manipulators to perform the critical tasks of suturing and knot tying is presented. The proposed method does not require a physical prototype. This is achieved by running typical tool motions during these tasks through the inverse kinematics of the manipulators and checking if the system can accommodate the desired motions. The system can perform a given motion if the whole trajectory lies continuously within the workspace of the manipulator. Open surgical suturing motion data collected from experiments done with expert surgeons is used as the set of desired tool motions used in the analysis. The method is applied to compare two different wrist configurations of telesurgical slave manipulators, intended for use in minimally invasive surgery, by looking at the requirements on joint ranges and wrist manipulability during these motions.

Haptic interaction is an increasingly common form of interaction in virtual environment simulations. This medium introduces some new challenges. In this paper we study the problem arising from the difference between the sampling rate requirements of haptic interfaces and the significantly lower update rates of the physical models being manipulated. We propose a multirate simulation approach which uses a local linear approximation. The treatment includes a detailed analysis and experimental verification of the approach. The proposed method is also shown to improve the stability of the haptic interaction.

Proposes a mass-spring model of a human thigh based on real data acquired. It addresses both the difficulties of determining the parameters of this model to fit the measurements and the computational demands. Implicit integration is used to update the model through time. The motivation behind this work is to provide accurate force-feedback for an echographic simulator that could be used to train practitioners to detect a thrombosis.

In this paper, various aspects of robotic telesurgery are studied. After a general introduction to laparoscopic surgery and medical applications of robotics, the UC Berkeley/Endorobotics Inc./UC San Francisco Telesurgical Workstation, a master-slave telerobotic system for laparoscopic surgery, is introduced, followed by its kinematic analysis, control, and experimental results. Some conceptual and future issues on telesurgery are discussed, including teleoperation and hybrid control, focusing on the special requirements of telesurgery.