Deployable structures provide adaptability and versatility for applications such as temporary architectures, space structures, and biomedical devices. Jamming is a mechanical phenomenon with which dramatic changes in stiffness can be achieved by increasing the frictional and kinematic coupling between constituents in a structure by applying an external pressure. This study applies jamming, which has been primarily used in medium-scale soft robotics applications to large-scale deployable structures with components that are soft and compact during transport, but rigid upon deployment. It proposes a new jamming structure with a novel built-in actuation mechanism which enables high-performance at large scales: a composite beam made of rectangular segments along a cable which can be pre-tensioned and thus jammed. Two theoretical models are developed to provide insights into the mechanical behavior of the composite beams and predict their performance under loading. A scale model of a deployable bridge is built using the tension-based composite beams, and the bridge is deployed and assembled by air with a drone demonstrating the versatility and viability of the proposed approach for robotics applications.

A long-standing question in robot hand design is how accurate tactile sensing must be. This paper uses simulated tactile signals and the reinforcement learning (RL) framework to study the sensing needs in grasping systems. Our first experiment investigates the need for rich tactile sensing in the rewards of RL-based grasp refinement algorithms for multi-fingered robotic hands. We systematically integrate different levels of tactile data into the rewards using analytic grasp stability metrics. We find that combining information on contact positions, normals, and forces in the reward yields the highest average success rates of 95.4% for cuboids, 93.1% for cylinders, and 62.3% for spheres across wrist position errors between 0 and 7 centimeters and rotational errors between 0 and 14 degrees. This contact-based reward outperforms a non-tactile binary-reward baseline by 42.9%. Our follow-up experiment shows that when training with tactile-enabled rewards, the use of tactile information in the control policy's state vector is drastically reducible at only a slight performance decrease of at most 6.6% for no tactile sensing in the state. Since policies do not require access to the reward signal at test time, our work implies that models trained on tactile-enabled hands are deployable to robotic hands with a smaller sensor suite, potentially reducing cost dramatically.

Exosuits can reduce metabolic demand and improve gait. Controllers explicitly derived from biological mechanisms that reflect the user's joint or muscle dynamics should in theory allow for individualized assistance and enable adaptation to changing gait. With the goal of developing an exosuit control strategy based on muscle power, we present an approach for estimating, at real time rates, when the soleus muscle begins to generate positive power. A low-profile ultrasound system recorded B-mode images of the soleus in walking individuals. An automated routine using optical flow segmented the data to a normalized gait cycle and estimated the onset of concentric contraction at real-time rates (~130Hz). Segmentation error was within 1% of the gait cycle compared to using ground reaction forces. Estimation of onset of concentric contraction had a high correlation (R2=0.92) and an RMSE of 2.6% gait cycle relative to manual estimation. We demonstrated the ability to estimate the onset of concentric contraction during fixed speed walking in healthy individuals that ranged from 39.3% to 45.8% of the gait cycle and feasibility in two persons post-stroke walking at comfortable walking speed. We also showed the ability to measure a shift in onset timing to 7% earlier when the biological system adapts from level to incline walking. Finally, we provided an initial evaluation for how the onset of concentric contraction might be used to inform exosuit control in level and incline walking.

Commercial six-axis force-torque sensors suffer from being some combination of expensive, fragile, and hard-touse. We propose a new fiducial-based design which addresses all three points. The sensor uses an inexpensive webcam and can be fabricated using a consumer-grade 3D printer. Open-source software is used to estimate the 3D pose of the fiducials on the sensor, which is then used to calculate the applied force-torque. A browser-based (installation free) interface demonstrates ease-of-use. The sensor is very light and can be dropped or thrown with little concern. We characterize our prototype in dynamic conditions under compound loading, finding a mean R2of over 0.99 for the Fx, Fy, Mx, and Myaxes, and over 0.87 and 0.90 for the Fzand Mzaxes respectively. The open source design files allow the sensor to be adapted for diverse applications ranging from robot fingers to human-computer interfaces, while the sdesign principle allows for quick changes with minimal technical expertise. This approach promises to bring six-axis force-torque sensing to new applications where the precision, cost, and fragility of traditional strain-gauge based sensors are not appropriate. The open-source sensor design can be viewed at http://sites.google.com/view/fiducialforcesensor.

Flexures provide precise motion control without friction or wear. Variable impedance mechanisms enable adaptable and robust interactions with the environment. This paper combines the advantages of both approaches through layer jamming. Thin sheets of complaint material are encased in an airtight envelope, and when connected to a vacuum, the bending stiffness and damping increase dramatically. Using layer jamming structures as flexure elements leads to mechanical systems that can actively vary stiffness and damping. This results in flexure mechanisms with the versatility to transition between degrees of freedom and degrees of constraint and to tune impact response. This approach is used to create a 2-DOF, jamming-based, tunable impedance robotic wrist that enables passive hybrid force/position control for contact tasks. Keywords: Compliant Joint/Mechanism, Compliance and Impedance Control, Mechanism Design.

One of the key design parameters in tendon-driven continuum robots is the number of tendons and the tendon loading distribution. A load model is also helpful for avoiding slack in tendons that causes control inefficiency and inaccuracy. A quasistatic model of n-tendon continuum robots is derived using the Euler-Lagrange formulation. The model is employed to derive an analytical loading model for equidistant tendon tensions for any given beam configuration within the workspace. The model accounts for the bending and axial compliance of the manipulator as well as tendon compliance. Features of the proposed model are discussed and some of the potential applications are explained. Based on the proposed model, a slack avoidance algorithm with analytical formulation is developed to dynamically optimize the tendon loads while preventing slack in tendons for a given configuration. The proposed model is experimentally validated in a multitendon continuum robot system for four case studies of three- to six-tendon arrangements in open-loop control architecture. A stereo vision-based three-dimensional reconstruction system measures the beam configuration and properties for each of the threeto six-tendon continuum robots. The effect of number of tendons on the tension loads in n-tendon continuum robots is studied. A quantitative dimensionless relationship between the number of tendons, the maximum tendon loads, and the bending angles is developed that may be used as a design tool for tradeoff among the complexity and required force and size.

Robotic cardiac catheterization using ultrasound (US) imaging catheters provides real time imaging from within the heart while reducing the difficulty in manually steering a four degree-of-freedom (4-DOF) catheter. Accurate robotic catheter navigation in the heart is challenging due to a variety of disturbances including cyclical physiological motions, such as respiration. In this work we compensate for respiratory motion by using an Extended Kalman Filter (EKF) to predict target motion and by applying the predictions to steer the US imaging catheter. The system performance was measured in bench top experiments with phantom vasculature. The robotic system with predictive filtering tracked cyclically moving targets with 1.59 mm and 0.72° mean error. Accurately tracking moving structures can improve intra-procedural treatments and visualization.

A system for automatically pointing ultrasound (US) imaging catheters will enable clinicians to monitor anatomical structures and track instruments during interventional procedures. Off-the-shelf US catheters provide high quality US images from within the patient. While this method of imaging has been proven to be effective for guiding many interventional treatments, significant training is required to overcome the difficulty in manually steering the imager to point at desired structures. Our system uses closed-form four degree of freedom (DOF) kinematic solutions to automatically position the US catheter and point the imager. Algorithms for steering and imager pointing were developed for a range of useful diagnostic and interventional motions. The system was validated on a robotic test bed by steering the catheter within a water environment containing phantom objects. While the system described here was designed for pointing ultrasound catheters, these algorithms are applicable to accurate 4-DOF steering and orientation control of any long thin tendon-driven tool with single or bi-directional bending.

Cardiac catheterization with ultrasound (US) imaging catheters provides real time US imaging from within the heart, but manually navigating a four degree of freedom (DOF) imaging catheter is difficult and requires extensive training. Existing work has demonstrated robotic catheter steering in constrained bench top environments. Closed-loop control in an unconstrained setting, such as patient vasculature, remains a significant challenge due to friction, backlash, and physiological disturbances. In this paper we present a new method for closed-loop control of the catheter tip that can accurately and robustly steer 4-DOF cardiac catheters and other flexible manipulators despite these effects. The performance of the system is demonstrated in a vasculature phantom and an in vivo porcine animal model. During bench top studies the robotic system converged to the desired US imager pose with sub-millimeter and sub-degree-level accuracy. During animal trials the system achieved 2.0 mm and 0.65° accuracy. Accurate and robust robotic navigation of flexible manipulators will enable enhanced visualization and treatment during procedures.

Robotic manipulation in unstructured environments requires grasping a wide range of objects. Tactile sensing is presumed to provide essential information in this context, but there has been little work examining the tactile sensor signals produced during realistic manipulation tasks. This paper presents tactile sensor data from grasping a generic object in thousands of trials. Position error between the hand and object was varied to model the uncertainty in real-world grasping, and a grasp outcome prediction was done using only tactile sensors. Results show that tactile signals are highly variable despite good repeatability in grasping conditions. The observed variability appears to be intrinsic to the grasping process, due to the mechanical coupling between fingers as they contact the object in parallel, as well as numerous factors such as frictional effects and inaccuracies in the robot hand. Using a simple machine learning algorithm, grasp outcome prediction based purely on tactile sensors is not reliable enough for real-world responsibilities. These results have implications for improved tactile sensor system and controller design, as well as signal processing and machine learning methods.

Continuum robots offer better maneuverability and inherent compliance and are well-suited for surgical applications as catheters where gentle interaction with the environment is desired. However, sensing their shape and tip position is a challenge as traditional sensors cannot be employed in the same way that they are in rigid robotic manipulators. In this paper, a vision-based shape sensing algorithm for real-time 3D reconstruction of catheters based on the views of two arbitrary positioned cameras is presented. Customized high-speed algorithms are developed for the segmentation and feature extraction from the images. The algorithm is experimentally validated for accuracy by measuring the tip position, bending and orientation angles and for precision by estimating known 3D circular and elliptical shapes of the catheter. Experimental results demonstrate good accuracy and performance of the proposed high speed algorithms.

This paper presents the design and control of a teleoperated robotic system for dexterous micromanipulation tasks at the meso-scale, specifically open microsurgery. Robotic open microsurgery is an unexplored yet potentially a high impact area of surgical robotics. Microsurgical operations, such as microanastomosis of blood vessels and reattachment of nerve fibers, require high levels of manual dexterity and accuracy that surpass human capabilities. A 3-DoF robotic wrist is designed and built based on a spherical five-bar mechanism. The wrist is attached to a 3-axis commercial off-the-shelf linear stage, achieving a fully dexterous system. Design requirements are determined using motion data collected during a simulated microanastomosis operation. The wrist design is optimized to maximize workspace and manipulability. The system is teleoperated using a haptic device, and has the required bandwidth to replicate microsurgical motions. The system was successfully used in a micromanipulation task to stack 1 mm-diameter metal spheres. The micromanipulation system presented here may improve surgical outcomes during open microsurgery by offering better accuracy and dexterity to surgeons.

A system for actively changing the stiffness of a long, thin, flexible robotic manipulator has been designed for cardiologists to use in a range of diagnosis and treatment procedures. Low-stiffness manipulators, such as catheters, are ideal for steering through vasculature with low risk of tissue injury. However, such instruments are not well-suited for applying force to tissue. The proposed system solves this problem by using a series of bead-shaped vertebrae containing pull wires to actively change the stiffness of the catheter, similar to gooseneck surgical retractors. Individual wires steer the catheter to a desired location. All wires are then tensioned to create friction between each vertebra and prevent sliding, therefore resisting motion. While this design concept has been implemented manually in various settings for decades, fine robotic control of the friction and stiffness of the system relies on a thorough understanding of the friction properties between vertebral segments. We have developed an analytical model to understand the interactions between vertebrae and determine the relationships between system parameters and the overall stiffness of the catheter. Experiments validated the calculations from the model and the functionality of the system by applying known loads to the tip of the catheter and measuring the catheter displacement. The catheter stiffness was measured to range from 100 N/m to 800 N/m, which is sufficient for performing many surgical tasks on tissue. This system can be useful in minimally invasive procedures involving direct instrument contact with tissue by improving accuracy, safety, and work flow.

In this paper we have rapidly prototyped customized, highly-sensitive, mm-scale multi-axis force sensors for medical applications. Using a composite laminate batch fabrication process with biocompatible constituent materials, we have fabricated a fully-integrated, 10×10 mm three-axis force sensor with up to 5 V/N sensitivity and RMS noise on the order of ~1.6 mN, operational over a range of -500 to 500 mN in the x- and y-axes, and -2.5 to 2.5 N in the z-axis. Custom foil-based strain sensors were fabricated in parallel with the mechanical structure, obviating the need for post-manufacturing alignment and assembly. The sensor and its custom-fabricated signal conditioning circuitry fit within a 1×1×2 cm volume to realize a fully-integrated force transduction platform with potential haptics and control applications in minimally-invasive surgical tools. The form factor, biocompatibility, and cost of the sensor and signal conditioning makes this method ideal for rapid-prototyping low-cost, mm-scale distal force sensors. Sensor performance is validated in a simulated tissue palpation task using a robotic master-slave platform.

Grasping and manipulation in unstructured environments must handle a wide range of object properties and significant sensing errors. Underactuation and compliance have been shown to be an effective way to improve grasping performance under such uncertainty, but the degree of compliance plays an important role in both gently adapting to sensing errors and maintaining stable grasps of heavy objects. These demands limit the range of objects that can be grasped. We consider the role and required characteristics of tactile sensing as a compensation method when compliance alone is insufficient. By strategic use of contact sensing, it is possible to expand the capabilities of a hand to grasp effectively under a wide range of positioning errors using simple position-driven motors and low-cost hardware.

The lack of haptic feedback has negatively affected the surgeon's ability to palpate and diagnose tissue and differentiate its stiffness during surgical operations with commercially available robotic assisted surgical systems. A modular surgical instrument capable of non-invasive measurement of sideways tip/tissue interaction forces for direct application in robotic assisted minimally invasive surgery systems is presented in this paper. The proposed force measurement technique enables the actual non-invasive measurement of the sideways interaction forces at the tip jaws. The instrument has two actuation degrees of freedom (DOF) for the tip operation and grasping orientation. The tip functionality type (e.g., grasping, cutting, and dissecting) can also be changed quickly and easily. Experiments were conducted to evaluate functionalities of the proposed instrument in palpating tissues. The results are presented and analysed here that verify the capability of the proposed instrument in accurately measuring lateral tip/tissue interaction forces.

Performing surgery on fast-moving heart structures while the heart is freely beating is next to impossible. Nevertheless, the ability to do this would greatly benefit patients. By controlling a teleoperated robot to continuously follow the heart's motion, the heart can be made to appear stationary. The surgeon will then be able to operate on a seemingly stationary heart when in reality it is freely beating. The heart's motion is measured from ultrasound images and thus involves a non-negligible delay due to image acquisition and processing, estimated to be 150 ms that, if not compensated for, can cause the teleoperated robot's end-effector (i.e., the surgical tool) to collide with and puncture the heart. This research proposes the use of a Smith predictor to compensate for this time delay in calculating the reference position for the teleoperated robot. The results suggest that heart motion tracking is improved as the introduction of the Smith predictor significantly decreases the mean absolute error, which is the error in making the distance between the robot's end-effector and the heart follow the surgeon's motion, and the mean integrated square error.

Surgery on a freely beating-heart is extremely difficult as the surgeon must perform the procedure while following the heart's fast motion. However, by controlling a teleoperated robot to continuously follow the heart's motion, the surgeon can operate on a seemingly stationary heart. The heart's motion is calculated from ultrasound images and thus involves a non-negligible delay estimated to be 100 ms that, if not compensated for, can cause the robot end-effector (i.e., the surgical tool) to collide with and puncture the heart. This research proposes the use of a Smith predictor to compensate for this time delay. The results suggest that this improves heart motion tracking as the mean absolute error, the difference between the surgeon's motion and the distance between the heart and surgical tool, and the mean integrated square error decreased.

Intracardiac echocardiography (ICE) catheters enable high-quality ultrasound imaging within the heart, but their use in guiding procedures is limited due to the difficulty of manually pointing them at structures of interest. This paper presents the design and testing of a catheter steering model for robotic control of commercial ICE catheters. The four actuated degrees of freedom (4-DOF) are two catheter handle knobs to produce bi-directional bending in combination with rotation and translation of the handle. An extra degree of freedom in the system allows the imaging plane (dependent on orientation) to be directed at an object of interest. A closed form solution for forward and inverse kinematics enables control of the catheter tip position and the imaging plane orientation. The proposed algorithms were validated with a robotic test bed using electromagnetic sensor tracking of the catheter tip. The ability to automatically acquire imaging targets in the heart may improve the efficiency and effectiveness of intracardiac catheter interventions by allowing visualization of soft tissue structures that are not visible using standard fluoroscopic guidance. Although the system has been developed and tested for manipulating ICE catheters, the methods described here are applicable to any long thin tendon-driven tool (with single or bi-directional bending) requiring accurate tip position and orientation control.

Underactuated compliant robotic hands exploit passive mechanics and joint coupling to reduce the number of actuators required to achieve grasp robustness in unstructured environments. Reduced actuation requirements generally serve to decrease design cost and improve grasp planning efficiency, but overzealous simplification of an actuation topology, coupled with insufficient tuning of mechanical compliance and hand kinematics, can adversely affect grasp quality and adaptability. This paper presents a computational framework for reducing the mechanical complexity of robotic hand actuation topologies without significantly decreasing grasp robustness. Open-source grasp planning software and well-established grasp quality metrics are used to simulate a fully-actuated, 24 DOF anthropomorphic robotic hand grasping a set of daily living objects. DOFs are systematically demoted or removed from the hand actuation topology according to their contribution to grasp quality. The resulting actuation topology contained 22% fewer DOFs, 51% less aggregate joint motion, and required 82% less grasp planning time than the fully-actuated design, but decreased average grasp quality by only 11%.

Stroke is the leading cause of long-term disability in the United States, affecting over 795,000 people annually. In order to regain motor function of the upper body, patients are usually treated by regular sessions with a dedicated physical therapist. A cost-effective wearable upper body orthotics system that can be used at home to empower both the patients and physical therapists is described. The system is composed of a thin, compliant, lightweight, cost-effective soft orthotic device with an integrated cable actuation system that is worn over the upper body, an embedded limb position sensing system, an electric actuator package and controller. The proposed device is robust to misalignments that may occur during actuation of the compliant brace or when putting on the system. Through simulations and experimental evaluation, it was demonstrated i) that the soft orthotic cable-driven shoulder brace can be successfully actuated without the production of off-axis torques in the presence of misalignments and ii) that the proposed model can identify linear and angular misalignments online.

Micromanipulation methods used for complicated tasks such as microrobot assembly and microvascular surgery often lack the force reflection and contact localization capability necessary to achieve robust grasps of micro-scale objects without applying excessive forces. This absence of haptic feedback is especially prohibitive in cases where visual evidence of force application, such as object surface deformation, is imperceptible and where unstructured, dynamically changing environments require force sensing and modulation for safe, atraumatic object manipulation. This paper describes the design, fabrication, and experimental validation of a soft tactile sensor array for sub-millimeter contact localization and contact force measurement during micromanipulation. The geometry and placement of conductive liquid embedded channels within the sensor array are optimized to provide adequate sensitivity for representative micro-manipulation tasks. Mechanical testing of the sensor demonstrates a sensitivity of less than 50mN and contact localization resolution on the order of 100's of microns.

Catheter devices allow physicians to access the inside of the human body easily and painlessly through natural orifices and vessels. Although catheters allow for the delivery of fluids and drugs, the deployment of devices, and the acquisition of the measurements, they do not allow clinicians to assess the physical properties of tissue inside the body due to the tissue motion and transmission limitations of the catheter devices, including compliance, friction, and backlash. The goal of this research is to increase the tactile information available to physicians during catheter procedures by providing haptic feedback during palpation procedures. To accomplish this goal, we have developed the first motion compensated actuated catheter system that enables haptic perception of fast moving tissue structures. The actuated catheter is instrumented with a distal tip force sensor and a force feedback interface that allows users to adjust the position of the catheter while experiencing the forces on the catheter tip. The efficacy of this device and interface is evaluated through a psychophyisical study comparing how accurately users can differentiate various materials attached to a cardiac motion simulator using the haptic device and a conventional manual catheter. The results demonstrate that haptics improves a user's ability to differentiate material properties and decreases the total number of errors by 50% over the manual catheter system.

Recent developments in cardiac catheter technology promise to allow physicians to perform most cardiac interventions without stopping the heart or opening the chest. However, current cardiac devices, including newly developed catheter robots, are unable to accurately track and interact with the fast moving cardiac tissue without applying potentially damaging forces. This paper examines the challenges of implementing force control on a flexible robotic catheter. In particular, catheter friction and backlash must be compensated when controlling tissue interaction forces. Force controller designs are introduced and evaluated experimentally in a number of configurations. The controllers are based on the inner position loop force control approach where the position trajectory is adjusted to achieve a desired force on the target. Friction and backlash compensation improved force tracking up to 86% with residual RMS errors of 0.11 N while following a prerecorded cardiac tissue trajectory with accelerations of up to 3800 mm/s(2). This performance provides sufficient accuracy to enable a wide range of beating heart surgical procedures.

To determine object geometry in unstructured environments, sensors must be mechanically robust, must exert only low forces on objects during exploration, and must be able to scan large regions efficiently without risk of damaging objects or sensors. Joint-angle sensors on compliant joints provide an appealing option for this task. An algorithmic framework is presented that allows them to be used for contact detection and to determine object geometry without requiring tactile arrays or other complicated contact location sensors. This volumetric approach to using proprioceptive sensors provides improvements in accuracy over other existing approaches based on the intersection of planes and lines.

Surgical repair of the mitral valve is a difficult procedure that is often avoided in favor of less effective valve replacement because of the associated technical challenges facing non-expert surgeons. In the interest of increasing the rate of valve repair, an accurate, interactive surgical simulator for mitral valve repair was developed. With a haptic interface, users can interact with a mechanical model during simulation to aid in the development of a surgical plan and then virtually implement the procedure to assess its efficacy. Sub-millimeter accuracy was achieved in a validation study, and the system was successfully used by a cardiac surgeon to repair three virtual pathological valves.

Robotic catheters have the potential to revolutionize cardiac surgery by enabling minimally invasive structural repairs within the beating heart. This paper presents an actuated catheter system that compensates for the fast motion of cardiac tissue using 3D ultrasound image guidance. We describe the design and operation of the mechanical drive system and catheter module and analyze the catheter performance limitations of friction and backlash in detail. To mitigate these limitations, we propose and evaluate mechanical and control system compensation methods, including inverse and model-based backlash compensation, to improve the system performance. Finally, in vivo results are presented that demonstrate that the catheter can track the cardiac tissue motion with less than 1 mm RMS error. The ultimate goal of this research is to create a fast and dexterous robotic catheter system that can perform surgery on the delicate structures inside of the beating heart.

Robotic cardiac catheters have the potential to revolutionize heart surgery by extending minimally invasive techniques to complex surgical repairs inside the heart. However, catheter technologies are currently unable to track fast tissue motion, which is required to perform delicate procedures inside a beating heart. This paper proposes an actuated catheter tool that compensates for the motion of heart structures like the mitral valve apparatus by servoing a catheter guidewire inside a flexible sheath. We examine design and operation parameters that affect performance and establish that friction and backlash limit the tracking performance of the catheter system. Based on the results of these experiments and a model of the backlash behavior, we propose and implement compensation methods to improve trajectory tracking performance. The catheter system is evaluated with 3D ultrasound guidance in simulate in vivo conditions. The results demonstrate that with mechanical and control system design improvements, a robotic catheter system can accurately track the fast motion of the human mitral valve.

There is ample research on the stabilization of haptic teleoperation systems under communication time delay. Little attention, however, has so far been paid to the usefulness of delayed haptic feedback on task performance. While the usefulness of haptic feedback in no-delay teleoperation has been previously established, this paper investigates whether haptic feedback helps to improve task performance in the presence of delay. We consider peg-in-the-hole insertion, which is a dexterous manipulation task requiring high force levels at certain points during the task execution. Through a user study involving unilateral and bilateral teleoperation experiments under different delays, it is observed that in both unilateral and bilateral teleoperation, the task completion time increases as delay increases. It is also seen that haptic feedback helps reduce the amount and rate of energy transfer to the environment and the occurrence of larger robot/environment interaction forces. However, with the users mindful of minimizing contact forces, haptic feedback causes the task to take more time compared to no haptic feedback regardless of the time delay. Thus, for tasks where low completion times are crucial given a tolerance for larger forces, unilateral feedback may be sufficient.

The shape of the mitral valve annulus is used in diagnostic and modeling applications, yet methods to accurately and reproducibly delineate the annulus are limited. This paper presents a mitral annulus segmentation algorithm designed for closed mitral valves which locates the annulus in three-dimensional ultrasound using only a single user-specified point near the center of the valve. The algorithm first constructs a surface at the location of the thin leaflets, and then locates the annulus by finding where the thin leaflet tissue meets the thicker heart wall. The algorithm iterates until convergence metrics are satisfied, resulting in an operator-independent mitral annulus segmentation. The accuracy of the algorithm was assessed from both a diagnostic and surgical standpoint by comparing the algorithm's results to delineations made by a group of experts on clinical ultrasound images of the mitral valve, and to delineations made by an expert with a surgical view of the mitral annulus on excised porcine hearts using an electromagnetically tracked pointer. In the former study, the algorithm was statistically indistinguishable from the best performing expert (p=0.85) and had an average RMS difference of 1.81+/-0.78 mm to the expert average. In the latter, the average RMS difference between the algorithm's annulus and the electromagnetically tracked points across six hearts was 1.19+/-0.17 mm .

The manipulation of fast-moving, delicate tissues in beating heart procedures presents a considerable challenge to the surgeon. A robotic force tracking system can assist the surgeon by applying precise contact forces to the beating heart during surgical manipulation. Standard force control approaches cannot safely attain the required bandwidth for this application due to vibratory modes within the robot structure. These vibrations are a limitation even for single degree-of-freedom systems that drive long surgical instruments. These bandwidth limitations can be overcome by the incorporation of feed-forward motion terms in the control law. For intracardiac procedures, the required motion estimates can be derived from 3-D ultrasound imaging. Dynamic analysis shows that a force controller with feed-forward motion terms can provide safe and accurate force tracking for contact with structures within the beating heart. In vivo validation confirms that this approach confers a 50% reduction in force fluctuations when compared with a standard force controller and a 75% reduction in fluctuations when compared with manual attempts to maintain the same force.

Robot assistance can enhance minimally invasive image-guided surgery, but flexion of the thin surgical instrument shaft impairs accurate control by creating errors in the kinematic model. Two controller enhancements that can mitigate these errors are improved kinematic models that account for flexing and direct measurement of the instrument tip's position. This paper presents an experiment quantifying the benefits of these enhancements in an effort to inform development of an image-guided robot control system accurate in the presence of quasi-static instrument flexion. The study measured a controller's ability to guide a flexing instrument along user-commanded motions while preventing incursions into a forbidden region virtual fixture. Compared with the controller using neither enhancement, improved kinematics and reduced maximum incursion depth into the forbidden region by 28%, tip tracking by 67%, and both enhancements together by 83%.

The use of 3D ultrasound imaging for both imaging and tracking has enabled minimally-invasive, beating heart intracardiac procedures. However, the rapid movements of internal heart structures pose a serious challenge to the surgeon that is compounded by the presence of significant time delays in 3D ultrasound imaging. This paper investigates the concept of using a one-degree-of-freedom motion compensation system to synchronize instruments with tissue motions that are approximately one-dimensional. We first describe the motion of the mitral valve, which is well approximated by a 1D model. The subsequent development of a motion compensation system is described, which comprises a motion compensation instrument as well as an extended Kalman filter that can compensate for system delays. The benefits and robustness of the resulting system are demonstrated in user trials under tracking conditions with measurement noise, delay, and heart rate variability.

Beating heart procedures promise significant health benefits to patients but the fast motion of the heart poses a serious challenge to the surgeon. Robotic motion synchronization to heart movements could facilitate these surgeries, although for intracardiac procedures this requires the development of a predictive filter to compensate for the measurement noise and time delay present in 3D ultrasound imaging. In this paper, we present a quasiperiodic cardiac motion model and apply the extended Kalman filter to estimation of its parameters in real-time. We experimentally demonstrate high accuracy robot tracking to heart motion using this filter.

The inherent uncertainty associated with unstructured grasping tasks makes establishing a successful grasp difficult. Traditional approaches to this problem involve hands that are complex, fragile, require elaborate sensor suites, and are difficult to control. In this paper, we demonstrate a novel autonomous grasping system that is both simple and robust. The four-fingered hand is driven by a single actuator, yet can grasp objects spanning a wide range of size, shape, and mass. The hand is constructed using polymer-based shape deposition manufacturing, with joints formed by elastomeric flexures and actuator and sensor components embedded in tough rigid polymers. The hand has superior robustness properties, able to withstand large impacts without damage and capable of grasping objects in the presence of large positioning errors. We present experimental results showing that the hand mounted on a three degree of freedom manipulator arm can reliably grasp 5 cm-scale objects in the presence of positioning error of up to 100% of the object size and 10 cm-scale objects in the presence of positioning error of up to 33% of the object size, while keeping acquisition contact forces low.

This paper presents a robotic system capable of using three-dimensional ultrasound to guide a surgical instrument to a tracked target location. Tracking of both the surgical instrument and target was done using image based algorithms on the real-time 3D ultrasound data. The tracking techniques are shown to be especially amenable for execution on powerful graphics processor units. By harnessing a graphics card, it was possible to detect both a surgical instrument and a surgical target at a rate of 25 Hz. The high update rate permits the use of tracked instrument and target locations for controlling a robot. Validation of the system was done in a water tank, where the robot moved the instrument to the target site with a mean error of 1.2 mm

New 3D ultrasound visualization has enabled minimally invasive, beating-heart intracardiac procedures. However, rapid motion of internal heart structures limits the realization of these new procedures. This paper investigates the concept of using a single actuator to compensate for tissue motions which occur largely in one direction. We characterize mitral valve annulus motion and show that it is well approximated by a ID model. The subsequent development of a motion-compensating tool (MCT) is described. The resulting instrument was tested in user trials under a series of positional error and tracking delay conditions. Results indicate that the MCT provides an approximately 50% increase in dexterity and 50% decrease in applied force in comparison to a solid tool. The study also shows that MCT tracking efficacy is highly dependent on tracking delays, indicating the importance of predictive, cyclical control algorithms.

In applications such as space and surgical robotics, the use of thin, lightweight manipulators and cable-driven end- effectors results in flexibility of the manipulator. In bilateral teleoperation, however, any flexibility in a link or joint of the robot reduces the transparency of teleoperation. In this paper, we analyze master-slave teleoperation transparency under slave robot joint elasticity and evaluate the added benefits of using extra sensors at the end-effector of the elastic-joint robot. It is shown that velocity (or position) feedback from the output shaft of the elastic joint improves free-space position tracking performance, which in the absence of such feedback is hampered by the joint's anti-resonance. Also, when the interaction forces with an environment are measured by a force sensor and fed back to the user, end-effector velocity feedback improves hard-contact force tracking performance. If the operating trajectories correspond to low frequencies, both free-space position tracking and hard-contact force tracking are satisfactory regardless of end-effector feedback, yet the elasticity in the joint will be transmitted to the user during a hard contact task unless end-effector velocity feedback is used.

Force feedback has been suggested to provide a number of benefits to surgery. Few studies, however, have addressed the benefit of force feedback in the context of the complexities of true surgical tasks. When information is limited (such as depth information in endoscopically guided tasks), force feedback may provide additional information that improves performance. We investigate a two-handed, six degree of freedom, endoscopically guided, minimally invasive cannulation task (inserting one tube into another tube) to test this hypothesis. We used twelve subjects, six of whom were experienced minimally invasive surgeons. Results suggest that force feedback reduces applied forces for both subject groups, but only the surgically trained group can take advantage of this benefit without a significant increase in trial time. We hypothesize that this training difference is due to the interaction between visual-spatial motor abilities and the information contained in the mechanical interaction forces.

Search and rescue robots often use tethers to provide power and communication, but tethers get caught on debris and small robots have difficulty with the added drag of the tether. This work investigates a self-actuating tether capable of remaining free while traversing obstacles. We present a physical model of the pressure transients actuated tether. The model relates forward motion of the tether (independent of a dragging force) to the relevant design parameters of hose stiffness, flow diameter, tether length, applied pressure, and valve selection. We present an experiment to test and validate our model. The experimental results correspond within 15% to the expected values from our analysis and also validate the functional dependence of our model on the design parameters

This paper presents a real-time 3D ultrasoundguided robotic system, designed to autonomously navigate a surgical instrument to surgeon-specified target points. The system tracks the instrument in real-time (2 Hz) using image processing. Thus, it does not require tracking of the ultrasound scan head. It makes use of a line detection algorithm and a passive instrument marker, which together report the instrument's position and orientation (6 d.o.f.) from a single ultrasound image. Validation experiments are presented. The system controlled instrument position in a variety of image locations with a mean error of 0.8 mm

Integration of the correct tactile and kinesthetic force feedback response with an accurate computational model of a compliant environment is a formidable challenge. We examine several design issues that arise in the construction of a compliance renderer, specifically the interaction between impedances of tactile displays, impedances of robot arms, and the computational model. We also describe an implementation of a compliance rendering system combining a low-impedance robot arm for large workspace kinesthetic force feedback, a high-impedance shape display for distributed tactile feedback to the finger pad, and a real-time finite element modeler. To determine the efficacy of the integration of tactile and kinesthetic force feedback components, we conducted a study examining the user’s ability to discriminate stiffness. Subjects were able to reliably detect a 20% difference in rendered material stiffness using our compliance rendering system.

Robots are useful tools in minimally invasive surgery, providing benefits such as reduction in hand tremor, navigation, and workspace scaling. Unfortunately, minimally invasive configurations result in two likely sources of kinematic error: port displacement and instrument shaft flexion. For a quasistatic system, a measure is presented that relates the errors in the robot Jacobian to the angular difference between desired motions and actual motions. Simulations and experimental data demonstrate this measure for a laboratory system. One potential use for the presented measure is, for bounded errors, determining whether the system monotonically converges for all initial and desired positions in the workspace. In addition, the measure is useful for path planning, determining less error-prone paths.

Combining teletaction systems with telemanipulation systems promises to enhance task performance when interacting with remote environments. However, the force scaling inherent in the telemanipulation system affects the ability of the user to control the exploration force. The quality of the tactile signal is therefore impacted, affecting performance in tasks that benefit from spatially distributed force information. We compare performance localizing an embedded lump in a compliant environment using a telemanipulated teletaction system versus a directly manipulated teletaction system. Lump localization accuracy was found to be the same; however, time required to localize the lump was up to 150% longer for the telemanipulation trials. Based upon our results, we conclude that the ability to maintain an appropriate force in the remote environment is necessary to take full advantage of the spatially distributed force information from the tactile sensor.

In the world of search and rescue robotics, particularly for search, smaller is better. Small robots can get into tighter places and are more maneuverable. With diminishing size, however, providing adequate power and communications becomes a problem. Communication is problematic if the search site is a collapsed building were transmitted signals have to travel through layers of concrete and steel. Tethers are good for providing power and communication, but tethers get caught and small robots have difficulty with the added drag of the tether. This work proposes a self-actuating tether capable of moving its own weight and remaining free while traversing around corners. Tether motion is due to induced high pressure water transients formed by rapidly arresting flow through the tether. A number of tests performed on a constructed tether prototype are presented. A simplified model of the water transients to better understand design parameters is outlined and simulated, and force measurements are collected to validate the simulation results.

This paper examines the role of grasper compliance and kinematic configuration in unstructured environments, where object size and location may not be well known. A grasper consisting of two two-link planar fingers with compliant revolute joints was simulated as it passively deflects during contact with a target object. The kinematic configuration and joint stiffness values of the grasper were varied in order to maximize grasper workspace for a wide range of target object size. The results show a near-optimal result around the spring-rest angles of 25 and 45 degrees for the base and intermediate joints, respectively, when the joint stiffness ratio (base/intermediate) was small.

Properly selected port sites for robot-assisted coronary artery bypass graft (CABG) improve the efficiency and quality of these procedures. In clinical practice, surgeons select port locations using external anatomic landmarks to estimate a patient's internal anatomy. This paper proposes an automated approach to port selection based on a preoperative image of the patient, thus avoiding the need to estimate internal anatomy. Using this image as input, port sites are chosen from a grid of surgeon-approved options by defining a performance measure for each possible port triad. This measure seeks to minimize the weighted squared deviation of the instrument and endoscope angles from their optimal orientations at each internal surgical site. This performance measure proves insensitive to perturbations in both its weighting factors and moderate intraoperative displacements of the patient's internal anatomy. A validation study of this port site selection was performed. cardiac algorithm also Six surgeons dissected model vessels using the port triad selected by this algorithm with performance compared to dissection using a surgeon-selected port triad and a port triad template described by Tabaie et al., 1999. With the algorithm-selected ports, dissection speed increased by up to 43% (p = 0.046) with less overall vessel trauma. Thus, this algorithmic approach to port site selection has important clinical implications for robot-assisted CABG which warrant further investigation.

Locating arteries hidden beneath superficial tissue can be a difficult task in minimally invasive surgery. This paper reports the development of a system that finds the paths of arteries using tactile sensing. The surgeon begins by using the surgical robot to place the tactile sensor instrument on a known artery location. Signal processing algorithms locate the artery from its pulsatile pressure variation. An adaptive extrapolation algorithm then generates predicted locations for the artery based on previous measurements. After moving to the predicted location, if the artery is not located then a backtracking mechanism moves the sensor towards previously detected locations. Tests with model arteries show good tracking ability for circular arcs with curvatures as small as 80 mm, although problems with compliance in the system result in occasional loss of the artery path. Preliminary tests demonstrate the ability to transcutaneously track the radial artery in the human wrist.

Presents the design and testing of a multi-channel vibrotactile display. It is composed of a cylindrical handle with four embedded vibrating elements driven by piezoelectric beams. Vibrations are transmitted to the hands through arrays of pins. The device was tested in sensory substitution for conveying force information during a teleoperated peg insertion. Results show that the device is effective in reducing peak forces during the insertion task.

We present the results from our work in the development of a robotic arm with minimal impedance. The development of such an arm is useful for gentle exploration of unknown objects in unstructured environments. Similar to a human, the robotic arm should minimize the contact forces in the event of unanticipated contact with unknown objects in the absence of visual feedback. To accomplish this, our strategy is to develop a good model of the robotic system so that we can use low gains which in turn will lead to low impedance and hence low contact forces in manipulation tasks in unstructured environments. The paper demonstrates how good modeling and feedforward compensation can result in low interaction forces without any external force sensing. We present experimental results to demonstrate the validity of our model and approach.

During teleoperation, a human operator often receives only 2D visual feedback from the remote environment. While a variety of kinematic sensors exist on the remote manipulator for control purposes, the information inherent in these signals is rarely extracted and presented to the operator in a useful form. This paper investigates the extraction of such information through the development of an algorithm that estimates the geometric properties of a manipulated object and its environment using the remote robot sensors and knowledge of the task being performed. The focus of the paper is the development of contact constraint equations parametrized by the desired geometric properties and the automatic segmentation of the data stream according to the set of active contacts. The approach is validated for 3D peg-in-hole insertion using a desktop teleoperator system.

Minimally invasive surgery (MIS) constrains instrument motions to 4 DOF by precluding lateral motion at the incision. Robotic MIS systems can interpose arbitrary mappings between the surgeon's motions at the master controller and the motions of instrument tips within the patient's body. Our goal was to find the interface that was easiest to learn. We investigated the effects of different coordinate frame mappings (screen-mapped versus instrument-mapped) and master dexterities (6 DOF versus 4 DOF) by means of performance measures on simple surgical tasks. All four mode-dexterity combinations had approximately the same time-to-completion. The combination of instrument-based mapping and 4 DOF master had lower error rate and lower subjective workload. This mode most clearly reproduces the task constraints within the patient's body.

During teleoperation, the automatic identification of remote environment properties has the potential of improving performance by providing task-specific feedback to the operator. Similarly, virtual training systems can be calibrated using such an automatic identification procedure. For those properties which ran be described by parametrized constraint equations, this paper provides a method by which the active constraints can be determined during each portion of the remote manipulator's data stream. The parametrized properties can then be estimated from the appropriate data stream segments. The approach is validated for peg-in-hole insertion using a desktop teleoperator system. The proposed segmentation procedure is compared with manual segmentation to estimate the geometric properties of the peg and hole.

Vibrations can significantly enhance touch perception for virtual environment applications with minimal design complexity and cost. In order to create realistic vibrotactile feedback, we collected vibrations, forces, and velocities during various tasks executed with a stylus: tapping on materials, stroking textures, and puncturing membranes. Empirical models were fit to these waveforms and a library of model parameters was compiled. These models simulated tasks involving simultaneous display of forces and vibrations on a high-bandwidth force-feedback joystick. Vibration feedback adds little complexity to virtual environment algorithms. Human subjects interacting with the system showed improved execution and perception when performing surface feature discrimination tasks.

This paper proposes techniques for automatically identifying and modeling the kinematic constraints of manipulated objects. The existence of constraints is established by examination of the forces and torques normal and tangential to the object's motion. Constraints are modeled as joints with configuration and history dependent forces. To illustrate these concepts, several experimental examples are presented.

Skilled drummers can play drum rolls at frequencies well in excess of human motor control bandwidths, by allowing the drumstick to bounce passively against the drum head at least twice for each hand stroke. We present experimental evidence that drummers modulate grasp force to control the effective stiffness of the drumstick which in turn determines the bounce frequency. To confirm this explanation, we constructed a simple, single-joint robot that emulates the human drumming technique. McKibben "artificial muscle" pneumatic actuators were paired in an agonist-antagonist configuration, resulting in a slow robot with variable passive impedance. The robot can execute drum rolls across a frequency range comparable to human drumming (bounce interval=40-160 ms). The results demonstrate that modulation of passive impedance can permit a low bandwidth robot to execute certain types of fast manipulation tasks.

People can accomplish many high bandwidth tasks despite the slow response of the neuromuscular system. Reaction times are well over 100 ms, but skilled drummers can play a drum roll at over 30 Hz. Drummers take advantage of a passive dynamic interaction: they let the drumstick bounce against the drum head at least twice on each hand stroke. Because the bounce occurs too quickly for neuromuscular response, the control parameters that govern the interaction must be predetermined. In this paper, we present evidence that drummers vary the stiffness of their hands to control the bounce frequency. The drum roll is thus an example of the explicit use of impedance modulation to increase effective bandwidth in a manipulation task. To confirm this explanation, we have constructed a simple robot joint that performs a drum roll in much the same manner as a human drummer. It has slow pneumatic actuators which can modulate passive joint stiffness as well as position or torque. Our experiments show that drum roll frequency can be controlled by varying the robot's passive stiffness, as surmised for human drummers.

For a walking robot to negotiate rough terrain it must adjust its step length to hit suitable footholds while simultaneously regulating its forward speed. This paper develops an algorithm to achieve these aims for a planar dynamic biped in the context of smooth exchange of support. The basis of the algorithm is an asymmetric gait to adjust walking velocity combined with a set of conditions on the leg lengths which ensure smooth exchange of support for any step size. The algorithm was tested on level surfaces in simulation and on an experimental biped robot. The algorithm could track a 30% change in desired walking velocity and a 25% change in desired step length.

This paper presents initial results in integrating touch with vision for delicate manipulation tasks. A generalizable framework of behavioral primitives for tactile and visual feedback control is proposed. Since vision provides position and shape information at a distance, while tactile provides small-scale geometric and force information, we focus on the complimentary roles of vision and touch. We demonstrate that visual feedback can perform the rough positioning needed for tactile sensor feedback, and that grasp force and object orientation can be sensed and controlled with tactile sensing. A force sensor based approach provides a comparison measure, and we observe that the use of tactile sensing results in a more gentle grasp.

This paper presents a system for relaying distributed tactile information from the finger tips of a remote manipulator to the finger tips of a human teleoperator. Tactile array sensors measure the pressure distribution at the contact between the robot hand and the grasped object. A computer samples this pressure signal and applies signal processing algorithms. The resulting signal drives a tactile shape display mounted on the master manipulator. This device raises an array of pins against the operator's finger tip skin to recreate the object shape. This system has been tested on a force-reflecting teleoperated hand. Experiments confirm the system's ability to convey significant contact information.

This paper presents an experimental comparison of tactile array versus force-torque sensing for localizing contact during manipulation. The manipulation tasks involved rotating and translating objects using a planar two fingered manipulator. A pin and a box were selected as limiting cases of point and line contact against a cylindrical robot finger tip. Force-torque contact sensing results suffered from difficulties in calibration, transient forces, and low grasp force. Tactile array sensing was immune to these problems, and the effect of shear loading was only noticeable for a simple centroid algorithm. The results show that with care, both of these sensing schemes can determine the contact location within a millimeter during real manipulation tasks.

In this paper we examine exchange-of-support for a simple planar bipedal robot. We derive a set of conditions which will ensure that the robot's body does not experience an instantaneous change in velocity at exchange-of-support. These conditions specify leg lengths and velocities which the robot must achieve for impactless walking. We propose to smooth the single-support phase through the use of appropriate leg length programs and examine one such program. Finally, we describe an experimental robot we constructed to facilitate the testing of these ideas, and present brief experimental results.>

This paper presents a tactile sensor that provides transient event information at the finger-object interface. The multi-element stress rate sensor consists of piezoelectric polymer strips moulded into the surface of the rubber "skin" covering the robot finger tip. These piezoelectric elements provide localized information important to manipulation control. We provide experimental results confirming the sensor's ability to detect three parameters. Contact events have been detected for signaling transitions between position and force control. Detecting local skin curvature provided information regarding contact shape and area. Finally, incipient slip, which is generated by small micro slips prior to gross slip, has been detected. By sensing all of these parameters, the utility of the multi-element stress rate sensor seems promising.>

Dynamic tactile sensing, which is defined as sensing during motion for perception of high spatial and temporal frequencies, is presented. Applications include sensing fine surface features and textures and monitoring contact conditions for dextrous manipulation. One type of dynamic tactile sensor, the stress rate sensor, is described in detail. It uses piezoelectric polymer transducers to measure the changes in stress induced in the sensor's rubber skin as it traverses small surface features and textures. The signals are interpreted with the aid of a solid mechanics model of the contact interaction and a linear deconvolution filter. Experimental verification of the sensor's performance, including the detection of surface features only 6.5 mu m high, are presented.>

The ability to sense and respond to frictional variations is important for dexterous manipulation. It is demonstrated that it is possible to elicit rapid, nonhabituating and sustained grasp responses by means of a tactile display. Experiments in which subjects grasped and lifted an instrumented test object using the thumb and index finger are reported. While the object was held in air, rapid but small sliding movements were invoked between the object and either contact plate and caused a load force redistribution. This reliably triggered a grasp force increase similar to the ones elicited by natural slips occurring during normal manipulation. An important application of this finding is in relaying frictional information from a slave hand to a human operator. Furthermore, it may make it possible to reduce disparity between master and slave hands in force reflective telemanipulation systems.>

The author describes a teleoperated hand system developed to study the role of tactile and fine-force sensing in telemanipulation. Both master and slave manipulators are two-fingered hands designed for precision tasks that humans typically execute with a pinch grasp. A direct drive parallel linkage configuration and brushless DC servomotors permit smooth, accurate control of contact forces and small motions, which is essential for effective tactile sensing and display. Initial experiments demonstrated that an operator can perform precision tasks using this system, and that the ability to convey small force levels to the operator required careful attention to the coupling between the operator's finger tips and the master manipulator. The system has been used to demonstrate that a tactile display can convey frictional information sensed at the slave manipulator.>

Preliminary experiments are presented concerning the use of tactile sensing to enhance the flexibility and robustness of robotic manipulation. A simple two-fingered manipulator with very clean dynamics has been constructed to focus on tactile and force sensing in manipulation. Manipulation is characterized by constantly changing mechanical systems, as fingers make or break contact or start to roll or slide on the surface of a grasped object. It is important to detect these changes since control schemes must change to match the varying task requirements. Following the human model, it is shown that dynamic tactile sensors can reliably detect the changing contact conditions. In a simple grasp-lift-replace task, use of these sensors enables the manipulator to cope with uncertainty in object location and task forces.>

The authors present a scheme for sensing small accelerations of the outer skin covering the fingers of a manipulator. The sensor is constructed with a thin rubber skin covering a soft inner layer of foam rubber. This decouples the skin from the manipulator structure, isolating it from structural vibrations and facilitating the tracking of object surfaces. An accelerometer attached to the inner surface of the skin measures the large local accelerations produced when areas of the skin catch and snap back as the sensor moves against a surface. The authors present experimental confirmation of the ability to detect the onset of slip, and discuss the sensor response to various surface texture parameters.>

The authors are concerned with finding the magnitudes of applied moment and force which will cause a robot finger to slip on the surface of a grasped object. Friction and contact models used in previous grasp analyses are reviewed, and an improved model which includes torsion-shear interaction is described. Experimental measurements of the initiation of sliding as a function of loading are reported. These measurements suggest that a simple linear function of torsion and shear magnitudes will adequately predict the onset of the slip in many tasks. The use of this function is illustrated in two measures of slip susceptibility for grasp planning.>