Wearable devices can suppress or reduce hand tremor motion associated with neurological disorders such as Parkinson's disease. Tendon-driven transmission systems have been proposed as a way to decrease the size and weight of these devices; however, they have complex control system requirements due to their substantially nonlinear behavior. To address this issue, this study focused on the development of a comprehensive kinematic model of a wearable tremor suppression glove that more accurately calculates the tendon displacement during hand motion. The novelty lies in the identification of the threshold bending angle for each joint at which the driven tendon touches the arc of the joint. The derived kinematic model of the glove was verified by both simulation and benchtop experiments, and the proposed model was validated during single-joint and multijoint hand movements. The kinematic model shows a mean 2-D correlation coefficient of 0.96$\boldsymbol{\pm }$0.01 with the experimental data. Compared to the Euclidean norm model presented in the literature, it presents an average 83% improvement (a 4%–96% reduction in root mean square errors depending on the joint), which is most significant for increasing tendon displacements.

In spite of the remarkable benefits that minimally invasive surgery provides for patients, the absence of force feedback is still a significant disadvantage. Several studies have been performed to address this issue; however, an accurate sterilizable force sensing technology for measuring axial and grasping forces is still missing. In this work, an innovative partial grasper has been designed and developed for a laparoscopic needle driver that can measure axial and grasping force information at the grasper tip. Fiber Bragg Grating sensors are used in this work because of their sterilizability and high sensitivity. Accuracies of 0.19 N and 0.26 N were achieved for the grasping and axial sensors respectively.

For many procedures, open surgery is being replaced with minimally invasive surgical (MIS) techniques. The advantages of MIS include reduced operative trauma and fewer complications leading to faster patient recovery, better cosmetic results and shorter hospital stays. As the demand for MIS procedures increases, effective surgical training tools must be developed to improve procedure efficiency and patient safety. Motion tracking of laparoscopic instruments can provide objective skills assessment for novices and experienced users. The most common approaches to noncontact motion capture are optical and electromagnetic (EM) tracking systems, though each approach has operational limitations. Optical trackers are prone to occlusion and the performance of EM trackers degrades in the presence of magnetic and ferromagnetic material. The cost of these systems also limits their availability for surgical training and clinical environments. This thesis describes the development and validation of a novel, noncontact laparoscopic tracking system as an inexpensive alternative to current technology. This system is based on the fusion of inertial, magnetic and distance sensing to generate real-time, 6-DOF pose data. Orientation is estimated using a Kalman-filtered attitude-heading reference system (AHRS) and restricted motion at the trocar provides a datum from which position information can be recovered. The Inertial and Range-Enhanced Surgical (IRES) Tracker was prototyped, then validated using a MIS training box and by comparison to an EM tracking system. Results of IRES tracker testing showed similar performance to an EM tracker with position error as low as 1.25 mm RMS and orientation error <0.58 degrees RMS along each axis. The IRES tracker also displayed greater precision and superior magnetic interference rejection capabilities. At a fraction of the cost of current laparoscopic tracking methods, the IRES tracking system would provide an excellent alternative for use in surgical training and skills assessment.

Tumor localization in traditional lung resection surgery requires manual palpation of the deflated lung through a thoracotomy. It is a painful procedure that is not suitable for many patients. Therefore, a multisensory mechatronic device was designed to localize tumors using a minimally invasive approach. The device is sensorized with tactile, ultrasound and position sensors in order to obtain multimodal data of soft tissue in real time. This paper presents the validation of the efficiency and efficacy of this device via an ex vivo experimental study. Tumor pathology was simulated by embedding iodine-agar phantom tumors of varying shapes and sizes into porcine liver tissue. The device was then used to palpate the tissue to localize and visualize the simulated tumors. Markers were then placed on the location of the tumors and fluoroscopic imaging was performed on the tissue in order to determine the localization accuracy of the device. Our results show that the device localized 87.5% of the tumors with an average deviation from the tumor center of 3.42 mm.

In this paper, a robotics-assisted articulating ultrasonic surgical scalpel for minimally invasive soft tissue cutting and coagulation is designed and developed. For this purpose, the optimal design of a Langevin transducer with stepped horn profile is presented for internal-body applications. The modeling, optimization and design of the ultrasonic scalpel are performed through equivalent circuit theory and verified by finite element analysis. Moreover, a novel two degrees-of-freedom (DOFs) surgical end effector (1-DOF pitch, 1-DOF grip) with decoupled motions is developed that is compatible with the da Vinci® surgical system. The developed instrument is then driven using the dVRK (da Vinci® research kit) and the Classic da Vinci® surgical system.

In this paper, a novel formulation for robust surgical planning of robotics-assisted minimally invasive cardiac surgery based on patient-specific preoperative images is proposed. In this context, robustness is quantified in terms of the likelihood of intraoperative collisions and of joint limit violations. The proposed approach provides a more accurate and complete formulation than existing deterministic approaches in addressing uncertainty at the task level. Moreover, it is demonstrated that the dexterity of robotic arms can be quantified as a cross-entropy term. The resulting planning problem is rendered as a chance-constrained entropy maximization problem seeking a plan with the least susceptibility toward uncertainty at the task level, while maximizing the dexterity (cross-entropy term). By such treatment of uncertainty at the task level, spatial uncertainty pertaining to mismatches between the patient-specific anatomical model and that of the actual intraoperative situation is also indirectly addressed. As a solution method, the unscented transform is adopted to efficiently transform the resulting chance-constrained entropy maximization problem into a constrained nonlinear program without resorting to computationally expensive particle-based methods.

In this paper, a robotic solution is proposed to deal with the challenges caused by lung motion during needle insertion. To accomplish this goal, a macro-micro robotic tool is designed to compensate for tissue motion using the macro part, while performing the needle insertion independently with the micro part. The main application of this work is for robotics-assisted lung tumor biopsy, where the combined motions of respiration and heartbeat may compromise success. An impedance-based controller keeps the macro reference coordinate in contact with the moving soft tissue using measurements from small pressure sensors mounted at the tip of the macro shaft. The micro part, mounted at the end of the macro robot, manipulates the needle in the harmonized reference coordinate system. Preoperative identification of ex vivo soft tissue is performed to estimate the dynamic behavior of the tissue. The controller is then synthesized based on the identified model. The effects of identification error and high frequency uncertainty are addressed in the control design. A prototype was built to evaluate the proposed approach using: 1) two Mitsubishi PA-10 robots, one for manipulating the macro part and the other for mimicking tissue motion, 2) one motorized linear stage to handle the micro part, and 3) a Phantom Omni haptic device for remote manipulation. Experimental results demonstrate the performance of the motion compensation system.

The reduced access conditions present in Minimally Invasive Surgery (MIS) affect the feel of interaction forces between the instruments and the tissue being treated. This loss of haptic information compromises the safety of the procedure and must be overcome through training. Determining the skill level of trainees is critical for ensuring patient safety. The objective of this work was to evaluate the usefulness of force information for skills assessment during MIS. Experiments were performed using a set of sensorized instruments capable of measuring instrument position and tissue interaction forces. The results show that experience level has a strong correlation with force-based metrics. The proposed metrics can be automatically computed, are completely objective, and measure important aspects of performance.

In this paper, a vibration-assisted needle insertion technique has been proposed in order to reduce needle-tissue friction. The LuGre friction model was employed as a basis for the current study and the model was extended and analyzed to include the impact of high-frequency vibration on translational friction. Experiments were conducted to evaluate the role of insertion speed as well as vibration frequency on frictional effects. In the experiments conducted, an 18 GA brachytherapy needle was vibrated and inserted into an ex-vivo soft tissue sample using a pair of amplified piezoelectric actuators. Analysis demonstrates that the translational friction can be reduced by introducing a vibratory low-amplitude motion onto a regular insertion profile, which is usually performed at a constant rate.

In this paper, a computational framework for patient-specific preoperative planning of Robotics-Assisted Minimally Invasive Cardiac Surgery (RAMICS) is presented. It is expected that preoperative planning of RAMICS will improve the success rate by considering robot kinematics, patient-specific thoracic anatomy, and procedure-specific intraoperative conditions. Given the significant anatomical features localized in the preoperative computed tomography images of a patients thorax, port locations and robot orientations (with respect to the patients body coordinate frame) are determined to optimize qualities such as dexterity, reachability, tool approach angles and maneuverability. To address intraoperative geometric uncertainty, the problem is formulated as a Generalized Semi-Infinite Program (GSIP) with a convex lower-level problem to seek a plan that is less sensitive to geometric uncertainty in the neighborhood of surgical targets. It is demonstrated that with a proper formulation of the problem, the GSIP can be replaced by a tractable constrained nonlinear program that uses a multi-criteria objective function to balance between the nominal task performance and robustness to collisions and joint limit violations. Finally, performance of the proposed formulation is demonstrated by a comparison between the plans generated by the algorithm and those recommended by an experienced surgeon for several case studies.

In this paper, robust preoperative planning of RAMICS is formulated. The intent of the proposed planning framework is to improve surgical outcomes by contemplating the intraoperative conditions of the surgical procedure and the geometry of the patient's thoracic anatomy. This includes improvements in target reachability, instrument dexterity for critical surgical tasks, surgical task feasibility and visibility. Given the patient's preoperative computed tomography images of the chest, the planning framework aims to determine the optimal location of the access ports on the ribcage, along with the optimal pose of the robotic arms relative to the patient's anatomy. To minimize susceptibility of the results to intraoperative geometric uncertainty, the planning is formulated as a Generalized Semi-Infinite Program (GSIP) with a convex lower level problem and a multi-criteria objective function. By solving the GSIP, tolerable geometric uncertainty within the task space is increased by eliminating the likelihood of collisions and joint limit violation in a neighborhood of the surgical target.

This paper outlines a framework for the preoperative planning of robotics-assisted minimally invasive cardiac surgery with application to coronary artery bypass grafting. The intent of the proposed framework is to improve surgical outcomes by considering the intraoperative requirements of the robotic manipulators and the anatomical geometry of the patient's chest. This includes target reachability, instrument dexterity for critical surgical tasks and collision avoidance. Given the patient's preoperative chest computed tomography images, the planning framework aims to determine the optimal location of the access ports on the ribcage, along with the optimal pose of the robotic arms relative to the patient's anatomy. The proposed multi-objective optimality criteria consist of a measure of clearance as well as a new collective kinematic measure. The minimum distances among the robot arms provides a measure for the likelihood of collisions. The proposed kinematic measure is composed of two modified manipulability indices that are dimensionally homogeneous and, in contrast to previously-used measures, are more likely to yield isotropic force and torque distributions when optimized for surgical interventions. The results of a case study illustrate the compatibility of the framework with general guidelines used by experienced surgeons for port selection. Furthermore, the framework surpasses those guidelines by ensuring the feasibility of the solutions in the sense of collision avoidance and surgical target reachability.

This paper focuses on the problem of incorporating haptics-enabled teleoperation in minimally invasive tumor localization. Since the stiffness of a tumor is higher than that of the surrounding tissue, it can be identified as a hard nodule when palpated. Using a Tactile Sensing Instrument (TSI) developed at CSTAR, the distributed pressure profiles along the contacting surface can be measured during remote tissue palpation. The tumor can be detected by using a visualization software that creates a color contour map based on the magnitude of the pressure over the palpated area. The accuracy of this method depends on the uniformity of the force applied to the tissue. A haptics-enabled teleoperation system provides the surgeon with the opportunity to feel the interaction force between the instrument and tissue during Minimally Invasive Surgery (MIS). The objective of this research was to assess the feasibility of combining force feedback with tactile feedback in order to increase the overall performance of tumor localization. The teleoperation system used in this work consists of a Mitsubishi PA10 robot as the slave that is remotely controlled (over a dedicated network) through a 7 Degree-Of-Freedom (DOF) haptic interface. A two-channel architecture, along with hybrid impedance control was utilized to form a bilateral teleoperation system in which the master is under force control and the slave is under position control. The experimental results confirm the effectiveness of using force feedback in robot-assisted tactile sensing for tumor detection.

The limitations of minimally invasive surgery include the inability to sense forces exerted by the instruments on tissue and the limited visual cues available through the endoscope. A modular laparoscopic instrument capable of measuring force and position has been designed to address these limitations. Novel image-based position tracking software has been developed and integrated within a graphical user interface. This modular system is low cost, versatile, and could be used for training, localization of critical features or for guidance during surgical procedures.

Spherical joints have evolved into a critical component of many robotic systems, often used to provide dexterity at the wrist of a manipulator. In this work, a novel 3 degree of freedom spherical joint is proposed, actuated by tendons that run along the surface of the sphere. The joint is mechanically simple and avoids mechanical singularities. The kinematics and mechanics of the joint are modeled and used to develop both open and closed loop control systems. Simulated and experimental assessment of the joint performance demonstrates that it can be successfully controlled in 3 degrees of freedom. It is expected that the joint will be a useful option in the development of emerging robotic applications, particularly those requiring miniaturization.

Minimally invasive surgery is a technique that provides numerous benefits to the patient, but presents challenges to the surgeon in that dexterity, hand-eye coordination and haptic perception are compromised. Robot-assisted minimally invasive approaches have addressed the problems of dexterity and coordination; however, the lack of kinesthetic and tactile feedback remains a significant drawback. Despite many advances in this area, little is currently known about what level of feedback performance is adequate to allow the surgeon to palpate tissue to detect an underlying tumour. This paper describes experiments that were conducted on ex-vivo porcine lung, using artificial tumours, to elucidate one measure of sensor performance required to detect the presence of a tumour. The results indicate that a force- sensitive probe with a sensing range of 0 to 10 N and a resolution of 0.01 N would allow a tumour to be localized via palpation using kinesthetic feedback.

This paper describes the design and implementation of a set of building blocks for the construction of adaptive modular sensing systems. A hardware architecture is proposed which allows the development of a general set of modular components with embedded knowledge about their capabilities. These modular components are referred to as transducer interface modules (TIMs). A knowledge representation scheme enables individual TIMs to exchange information about their capabilities and form a collective identity. The overall system functionality is determined by the manner in which TIMs are connected. This functionality is supported by a distributed software architecture that allows configuration-specific algorithms to be automatically loaded into each module. These building blocks can be rapidly reconfigured to form modular systems that are expected to prove useful in many applications, including industrial control, inspection systems, mobile robotics, monitoring, and data acquisition.

In this paper, a software architecture and knowledge representation scheme that enables the combination and reconfiguration of modular sensor and actuator components is described. The proposed software architecture utilizes a realtime operating system with a pre-emptive kernel, which simplifies the implementation of the architecture itself through the modularization and concurrent execution of its various software components. A virtual machine-based middleware layer runs on top of the operating system, enabling platform-independent logical algorithms to be written once, and run on any module irrespective of its underlying hardware architecture. Logical algorithms govern the behaviour of a given set of heterogeneous modules, providing them with intelligence and enabling them to behave as a single entity known as a logical module.

In this paper, a novel reconfigurable surveillance system that incorporates multiple active-vision sensors is presented. The proposed system has been developed for visual-servoing and other similar applications, such as tracking and state estimation, which require accurate and reliable target surveillance data. In the specific implementation case discussed herein, the position and orientation of a single target are surveyed at predetermined time instants along its unknown trajectory. Dispatching is used to select an optimal subset of dynamic sensors, to be used in a data-fusion process, and maneuver them in response to the motion of the object. The goal is to provide information of increased quality for the task at hand, while ensuring adequate response to future object maneuvers. Our experimental system is composed of a static overhead camera to predict the object's gross motion and four mobile cameras to provide surveillance of a feature on the object (i.e., target). Object motion was simulated by placing it on an xy table and preprogramming a path that is unknown to the surveillance system. The selected cameras are independently and optimally positioned to estimate the target's pose (a circular marker in our case) at the desired time instant. The target data obtained from the cameras, together with their own position and bearing, are fed to a fusion algorithm, where the final assessment of the target's pose is determined. Experiments have shown that the use of dynamic sensors, together with a dispatching algorithm, tangibly improves the performance of a surveillance system

This paper introduces a modular, real-time, omnidirectional, active vision system, as well as a constructed prototype. By combining omnidirectional and active pan-tilt cameras, a robust vision system is created that builds on the strengths of each camera type. The system can be easily configured to provide nearly an entire spherical field of view and independently track several targets of interest within the environment. The novel design allows the camera modules to be stacked, creating a vertical sensor structure. This vertical arrangement also provides a simple solution to the epipolar geometry and triangulation for target localization. Applications for this modular system can range from simple mobile robot navigation to complex multi-target tracking and surveillance.

This paper describes a straightforward process for calibrating an active vision system containing both pinhole perspective and omnidirectional cameras. The perspective cameras can be easily calibrated using standard methods. Unfortunately, these methods are not suitable for omnidirectional cameras. Methods that rely on iterative least squares optimization, using a set of known image-world correspondences, are adopted for omnidirectional cameras. To ensure unbiased estimation of camera parameters, an omnidirectional calibration rig is employed so that nearly the entire field of view contains known calibration points. Measurement uncertainties collected from each stage of calibration are then combined to estimate the overall system uncertainty. This calibration process is evaluated experimentally by estimating the location of known points using triangulation, where the results achieved are comparable with the estimated system uncertainties.

This paper outlines an approach to the coordination of multiple mobile sensors for the surveillance of a single moving target. A real-time dispatching algorithm is used to select and position groups of sensors in response to the observed object motion. The aim is to provide robust, high-quality data while ensuring that the system can react to unexpected object manoeuvres. Sensors are assigned to collect data at specific points on the object trajectory. A dispatching strategy learned via reinforcement learning is used to control the sensor poses with respect to these points. In using the learned strategy, each sensor adopts an egocentric view of the system state to determine the most appropriate action. Simulations demonstrate the performance of the RL-based dispatcher, in comparison to similar static-sensor systems.

In this paper, the implementation of a novel surveillance system that incorporates multiple active-vision sensors controlled by a real-time dispatching algorithm is presented. The proposed system improves reliability and accuracy of target surveillance-tracking systems used for visual-servoing and other similar applications. Experiments using a dispatched system have shown that the use of dynamic sensors can improve the performance of a surveillance system, primarily, due to the following factors: (i) decrease in the uncertainty associated with the object's estimated pose, (ii) increase in robustness of the system due to its ability to cope with a wider range of a priori unknown object trajectories, and (iii) increase in reliability through sensory fault tolerance.

The paper presents a methodology for determining the initial configuration of a set of sensors for a surveillance task. It serves to complement a dynamic dispatching methodology, which selects and maneuvers subsets of sensors to achieve optimal data acquisition in real-time. Specifically, given a priori information about the expected object trajectory, the initial sensor poses are determined such that the sensing-system effectiveness is maximized. This is achieved using a constrained, nonlinear, direct search method in combination with simulations of the sensing-system performance (i.e., dynamic dispatching to adjust the sensor poses in response to the object motion).

Sensory data must be collected in real time for the majority of autonomous decision making tasks, such as target tracking, surveillance and navigation. The use of multiple sensors may significantly improve the quality and robustness of the data. Given an environment containing a set of mobile sensors, capable of altering their position and orientation, this work addresses the problem of selecting and maneuvering subsets of these sensors for optimal data acquisition in realtime. A heuristic approach to the dispatching problem suitable for on-line implementation is illustrated by a computer simulated example.