Magnetic resonance imaging (MRI)-guided focused ultrasound (MRg-FUS) is an effectivenoninvasiveintervention. However, when extending to mild hyperthermia treatment (mHT), accurate temperature control with uniform thermal distribution remains challenging for deep-seated targets in highly heterogeneous tissues. To this end, we propose a novel thermal modeling andclosed-loopcontrol scheme for robot-assisted MRg-FUS mHT. A discrete-time positive system is introduced for thermal dynamics modeling. By introducing an event-triggered model predictive controller, objective functions are formulated to optimize transducerphasesequences. Thermal feedback is leveraged to accommodate modeling uncertainties,closingthe control loop to generate constructive ultrasound interference. Our scheme enables simultaneous spatial and temporal heating control, maintaining target temperature within a narrow range ($41\,^\circ \mathrm{C}\text{--}43\,^\circ \mathrm{C}$). The thermal dose evaluated under unknown disturbances demonstrates the robustness of the proposed scheme against severe tissue heterogeneity. Overheating can be avoided, enhancing the potential for safe intervention. Results from mechanical transducer adjustment further support the feasibility of treating large target areas in robot-assisted mHT.

Endoluminal surgery offers a minimally invasive option for early-stage gastrointestinal and urinary tract, but is limited by basic surgical tools and a steep learning curve. Robotic systems, particularly continuum robots, provide flexible instruments that enable precise, intuitive tissue resection in confined spaces, potentially improving outcomes. This paper presents an integrated visual perception platform for a continuum robotic system in endoluminal surgery. Our objective is to leverage monocular endoscopic image-based perception algorithms to accurately identify the position and orientation of flexible instruments and measure their distances from surrounding tissues. This thorough understanding of continuum robots and surgical scenes enhances the robustness of robotic procedures. We introduce 2D and 3D learning-based perception algorithms and develop a physically-realistic simulator that models the dynamics of flexible instruments. This simulator features a pipeline for generating realistic endoluminal scenes, enabling control of flexible robots in a realistic environment and substantial data collection. Using a continuum robot prototype, we conducted extensive evaluations, including module assessments and system-level evaluation of the perception platform. Results demonstrate that our perception algorithms significantly improve control of flexible instruments, reducing manipulation time by over 70% for trajectory-following tasks and enhancing the understanding of complex surgical scenarios, leading to robust endoluminal surgeries.

This article is concerned with the problem of asynchronous control for Interval Type-2 (IT2) fuzzy nonhomogeneous Markov jump systems against successive denial-of-service (DoS) attacks. The system and the controller are assumed to be connected through a communication channel subject to malicious attacks. The maximum number and probability distribution of successive attacks are considered. Under the imperfect premise matching, a fuzzy asynchronous controller is constructed by the hidden Markov model. By means of the introduced transmission delay, a delay closed-loop system is constructed, where the stochastic description of the delay depends on the statistical characteristic of successive attacks. Then stability criteria together are derived in the form of linear matrix inequalities by the Lyapunov functional approach, as well as the condition on the existence of the fuzzy controller. Finally, the feasibility and effectiveness of the presented control scheme are demonstrated by simulation results.

The passivity-based asynchronous control is tackled for 2-D Roesser Markovian jump systems (MJSs) and stabilization is guaranteed when 2-D MJSs are susceptible to Denial-of-Service (DoS) attacks. A novel jump model is proposed in this article, where the switching law of subsystems is regulated by the sum of the horizontal and vertical coordinates' values. This differs from the conventional jump model, which presumes that the transition probabilities are identical in both directions. The proposed jump model can avoid the mode ambiguity problem. Given the openness and sharing nature of communication networks, they are susceptible to malicious cyber-attacks that impair system performance. The concept of global time is introduced to help characterize the jump law and construct DoS attack model. Besides, a hidden Markov model (HMM) is utilized to manage the inevitable mismatched mode problem induced by any delay or data dropouts. With the above considerations, several conditions are established for ensuring passivity performance of 2-D MJSs and stabilization when facing DoS attacks. Several equivalent solvable conditions are derived via decoupling strategy and matrix inequality technique. Finally, two simulation examples are provided to demonstrate the validity of the established theoretical results.

The secure consensus problem is addressed for multiagent systems (MASs) suffering from saturated control input and denial-of-service (DoS) attacks. The communication networks’ open setting and sharing nature give rise to security issues and impact the performance of MASs. Malicious DoS attacks attempt to disrupt the information exchange and undermine consensus by compromising the availability of transmitted data. Moreover, the control input can be saturated as a result of physical device limitations or safety concerns. To tackle these challenges, a state-prediction-based dynamic event-triggered mechanism (DETM) control protocol is designed to guarantee the secure consensus of MASs while reducing redundant communication, avoiding continuous monitoring of adjacent states, and ensuring effective utilization of limited bandwidth resources. Zeno behavior is eliminated by confirming the existence of a positive lower bound on interevent intervals. Sufficient conditions are established for the co-design of the DETM and controller to accomplish the desired goal. Finally, a simulation is conducted to substantiate the effectiveness and validity of the proposed control protocol.

Switching dynamics are prevalent in real-world systems, arising from either intrinsic changes or responses to external influences, which can be appropriately modeled by switched systems. Control synthesis for switched systems, especially integrating safety constraints, is recognized as a significant and challenging topic. This study focuses on devising a learning-based control strategy for switched nonlinear systems operating under arbitrary switching law. It aims to maintain stability and uphold safety constraints despite limited system data. To achieve these goals, we employ the control barrier function method and Lyapunov theory to synthesize a controller that delivers both safety and stability performance. To overcome the difficulties associated with constructing the specific control barrier and Lyapunov function and take advantage of switching characteristics, we create a neural control barrier function and a neural Lyapunov function separately for control policies through a state transition learning approach. These neural barrier and Lyapunov functions facilitate the design of the safe controller. The corresponding control policy is governed by learning from two components: policy loss and forward state estimation. The effectiveness of the developing scheme is verified through simulation examples.

This work addresses the output reachable set-based leader-following consensus problem, focusing on a group of positive agents over directed dwell-time switching networks. Two types of non-negative disturbances, namely, 1)$L_{1}$-norm bounded disturbances and 2)$L_{\infty,1}$-norm bounded disturbances are studied. Meanwhile, a class of directed dwell-time switching networks for modeling the communication protocol of positive agents is investigated. To deal with the presence of disturbances, an output-feedback control protocol is developed to achieve a robust consensus with positivity preserved based on the output reachable set. By exploiting the positive characteristics, switched linear copositive Lyapunov functions are adopted to establish output reachable set-based consensus conditions. These conditions can facilitate the control protocol design by solving a bilinear programming problem, and also generate hyperpyramidal regions to confine the output consensus error. A particle swarm optimization-based (PSO-based) algorithm is applied to compute the controller gains and optimize the volume of the hyperpyramids. The proposed methods are verified by the presented numerical case studies.

We present a design of an inductively coupled radio frequency (ICRF) marker for magnetic resonance (MR)-based positional tracking, enabling the robust increase of tracking signal at all scanning orientations in quadrature-excited closed MR imaging (MRI). The marker employs three curved resonant circuits fully covering a cylindrical surface that encloses the signal source. Each resonant circuit is a planar spiral inductor with parallel plate capacitors fabricated monolithically on flexible printed circuit board (FPC) and bent to achieve the curved structure. Size of the constructed marker is Ø3-mm ×5 -mm with quality factor > 22, and its tracking performance was validated with 1.5 T MRI scanner. As result, the marker remains as a high positive contrast spot under 360° rotations in 3 axes. The marker can be accurately localized with a maximum error of 0.56 mm under a displacement of 56 mm from the isocenter, along with an inherent standard deviation of 0.1-mm. Accrediting to the high image contrast, the presented marker enables automatic and real-time tracking in 3D without dependency on its orientation with respect to the MRI scanner receive coil. In combination with its small form-factor, the presented marker would facilitate robust and wireless MR-based tracking for intervention and clinical diagnosis. This method targets applications that can involve rotational changes in all axes (X-Y-Z).

state-feedback control problem for networked discrete-time systems with positivity constraint. This problem (for a single positive system), raised recently in the area of positive systems theory, is known to be challenging due to its inherent nonconvexity. In contrast to most works, which only provide sufficient synthesis conditions for a single positive system, we study this problem within a primal-dual scheme, in which necessary and sufficient synthesis conditions are proposed for networked positive systems. Based on the equivalent conditions, we develop a primal-dual iterative algorithm for solution, which helps prevent from converging to a local minimum. In the simulation, two illustrative examples are employed for verification of our proposed results.

This work addresses the problem of reachable set-based consensus for positive multiagent systems affected by typical classes of bounded disturbances. In the presence of disturbances, a reachable set-based consensus with positivity preservation is proposed to ensure that the state of the closed-loop system remains positive while enclosing the reachable set of the defined consensus error with an ellipsoidal bounding region. Sufficient conditions are established to achieve the reachable set-based positive consensus under the energy-bounded or peak-bounded disturbance input. Equivalent design conditions are provided, for which a heuristic algorithm is proposed for computing and optimizing the bounding region. Simulations are conducted to validate the obtained results.

The high cost of data labeling often results in node label shortage in real applications. To improve node classification accuracy, graph-based semi-supervised learning leverages the ample unlabeled nodes to train together with the scarce available labeled nodes. However, most existing methods require the information of all nodes, including those to be predicted, during model training, which is not practical for dynamic graphs with newly added nodes. To address this issue, an adversarially regularized graph attention model is proposed to classify newly added nodes in a partially labeled graph. An attention-based aggregator is designed to generate the representation of a node by aggregating information from its neighboring nodes, thus naturally generalizing to previously unseen nodes. In addition, adversarial training is employed to improve the model’s robustness and generalization ability by enforcing node representations to match a prior distribution. Experiments on real-world datasets demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art methods. The code is available at https://github.com/JiarenX/AGAIN.

This article investigates the positive consensus problem of a special kind of interconnected positive systems over directed graphs. They are composed of multiple fractional-order continuous-time positive linear systems. Unlike most existing works in the literature, we study this problem for the first time, in which the communication topology of agents is described by a directed graph containing a spanning tree. This is a more general and new scenario due to the interplay between the eigenvalues of the Laplacian matrix and the controller gains, which renders the positivity analysis fairly challenging. Based on the existing results in spectral graph theory, fractional-order systems (FOSs) theory, and positive systems theory, we derive several necessary and/or sufficient conditions on the positive consensus of fractional-order multiagent systems (PCFMAS). It is shown that the protocol, which is designed for a specific graph, can solve the positive consensus problem of agents over an additional set of directed graphs. Finally, a comprehensive comparison study of different approaches is carried out, which shows that the proposed approaches have advantages over the existing ones.

Image processing has significantly extended the practical value of the eye-in-hand camera, enabling and promoting its applications for quantitative measurement. However, fully vision-based pose estimation methods sometimes encounter difficulties in handling cases with deficient features. In this article, we fuse visual information with the sparse strain data collected from a single-core fiber inscribed with fiber Bragg gratings (FBGs) to facilitate continuum robot pose estimation. An improved extreme learning machine algorithm with selective training data updates is implemented to establish and refine the FBG-empowered (F-emp) pose estimatoronline. The integration of F-emp pose estimation can improve sensing robustness by reducing the number of times that visual tracking is lost given moving visual obstacles and varying lighting. In particular, this integration solves pose estimation failures under full occlusion of the tracked features or complete darkness. Utilizing the fused pose feedback, a hybrid controller incorporating kinematics and data-driven algorithms is proposed to accomplish fast convergence with high accuracy. The online-learning error compensator can improve the target tracking performance with a 52.3%–90.1% error reduction compared with constant-curvature model-based control, without requiring fine model-parameter tuning and prior data acquisition.

Soft robots have great potential in surgical applications due to their compliance and adaptability to their environment. However, their flexibility and nonlinearity bring challenges for precise modeling, sensing, and control, especially in constrained cavities. In this article, a simple, compact two-segment soft robot for flexible laser ablation is proposed. The proximal hydraulic-driven segment can offer omnidirectional bending so as to navigate toward lesions. The distal segment driven by tendons enables precise, fast steering of laser collimator for laser sweeping on lesion targets. The dynamics of such mechanical steering motion can be enhanced with a metal spring backbone integrated along the collimator, thus facilitating the control with certain linearity and responsiveness. A soft robot modeling and control scheme based on Koopman operators is proposed. We also design a disturbance observer so as to incorporate the controller feedback with real-time fiber optic shape sensing. Experimental validation is conducted on simulated orex-vivolaser ablation tasks, thus evaluating our control strategies in laser path following across various contours/patterns. As a result, such a simple compact laser manipulation can perform up to 6 Hz sweeping with precision of path following errors below 1 mm. Such modeling and control scheme could also be used on an endoscopic laser ablation robot with unsymmetric mechanism driven by two tendons.

Although the soft robot is gaining considerable popularity in dexterous and safe manipulation, accurate motion control is still an open problem to be explored. Recent investigations suggest that reinforcement learning (RL) is a promising solution but lacks efficient adaptability for Sim2Real transfer or environment variations. In this paper, we present a deep deterministic policy gradient (DDPG)-based control system for the continuous task-space manipulation of soft robots. Domain randomization is adopted in simulation for fast control-policy initialization, while an offline retraining strategy is utilized to update the controller parameters for incremental learning. The experiments demonstrate that the proposed RL controller can track a moving target accurately (with RMSE of 1.26 mm), and accommodate to external varying load effectively (with ~30% RMSE reduction after retraining). Comparisons among the proposed RL controller and other supervised-learning-based controllers in handling additional tip load were also conducted. The results support that our RL method is appropriate for automatic learning such that there is no need of manual interference for data processing, particularly in cases with external disturbances and actuation redundancy.

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry’s appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

In this article, the non-negative edge consensus problem is addressed for positive networked systems with undirected graphs using state-feedback protocols. In contrast to existing results, the major contributions of this work included: 1) significantly improved criteria of consequentiality and non-negativity, therefore leading to a linear programming approach and 2) necessary and sufficient criteria giving rise to a semidefinite programming approach. Specifically, an improved upper bound is given for the maximum eigenvalue of the Laplacian matrix and the (out-) in-degree of the degree matrix, and an improved consensuability and non-negativevity condition is obtained. The sufficient condition presented only requires the number of edges of a nodal network without the connection topology. Also, with the introduction of slack matrix variables, two equivalent conditions of consensuability and non-negativevity are obtained. In the conditions, the system matrices, controller gain, as well as Lyapunov matrices are separated, which is helpful for parameterization. Based on the results, a semidefinite programming algorithm for the controller is readily developed. Finally, a comprehensive analytical and numerical comparison of three illustrative examples is conducted to show that the proposed results are less conservative than the existing work.

This article addresses the distributed consensus problem for identical continuous-time positive linear systems with state-feedback control. Existing works of such a problem mainly focus on the case where the networked communication topologies are of either undirected and incomplete graphs or strongly connected directed graphs. On the other hand, in this work, the communication topologies of the networked system are described by directed graphs each containing a spanning tree, which is a more general and new scenario due to the interplay between the eigenvalues of the Laplacian matrix and the controller gains. Specifically, the problem involves complex eigenvalues, the Hurwitzness of complex matrices, and positivity constraints, which make analysis difficult in the Laplacian matrix. First, a necessary and sufficient condition for the consensus analysis of directed networked systems with positivity constraints is given, by using positive systems theory and graph theory. Unlike the general Riccati design methods that involve solving an algebraic Riccati equation (ARE), a condition represented by an algebraic Riccati inequality (ARI) is obtained for the existence of a solution. Subsequently, an equivalent condition, which corresponds to the consensus design condition, is derived, and a semidefinite programming algorithm is developed. It is shown that, when a protocol is solved by the algorithm for the networked system on a specific communication graph, there exists a set of graphs such that the positive consensus problem can be solved as well.

This article investigates the nonnegative consensus tracking problem for networked systems with a distributed static output-feedback (SOF) control protocol. The distributed SOF controller design for networked systems presents a more challenging issue compared with the distributed state-feedback controller design. The agents are described by multi-input multi-output (MIMO) positive dynamic systems which may contain uncertain parameters, and the interconnection among the followers is modeled using an undirected connected communication graph. By employing positive systems theory, a series of necessary and sufficient conditions governing the consensus of the nominal, as well as uncertain, networked positive systems, is developed. Semidefinite programming consensus design approaches are proposed for the convergence rate optimization of MIMO agents. In addition, by exploiting the positivity characteristic of the systems, a linear-programming-based design approach is also proposed for the convergence rate optimization of single-input multi-output (SIMO) agents. The proposed approaches and the corresponding theoretical results are validated by case studies.

Cardiac electrophysiology is an effective treatment for atrial fibrillation, in which a long, steerable catheter is inserted into the heart chamber to conduct radio frequency ablation. Magnetic resonance imaging (MRI) can provide enhanced intraoperative monitoring of the ablation progress as well as the localization of catheter position. However, accurate and real-time tracking of the catheter shape and its efficient manipulation under MRI remains challenging. In this article, we designed a shape tracking system that integrates a multicore fiber Bragg grating (FBG) fiber and tracking coils with a standard cardiac catheter. Both the shape and positional tracking of the bendable section could be achieved. A learning-based modeling method is developed for cardiac catheters, which uses FBG-reconstructed three-dimensional curvatures for model initialization. The proposed modeling method was implemented on an MRI-guided robotic platform to achieve feedback control of a cardiac catheter. The shape tracking performance was experimentally verified, demonstrating 2.33° average error for each sensing segment and 1.53 mm positional accuracy at the catheter tip. The feedback control performance was tested by autonomous targeting and path following (average deviation of 0.62 mm) tasks. The overall performance of the integrated robotic system was validated by a pulmonary vein isolation simulator withex-vivotissue ablation, which employed a left atrial phantom with pulsatile liquid flow. Catheter tracking and feedback control tests were conducted in an MRI scanner, demonstrating the capability of the proposed system under MRI.

This article is focused on the design of an output tracking control scheme for a class of continuous-time periodic piecewise time-varying systems (PPTVSs) with actuator saturation and nonlinear perturbations. The energy-to-peak tracking performance is studied based on an equivalent condition on the definiteness property of matrix polynomials. Considering the actuator saturation and nonlinear perturbation, matrix polynomial-based sufficient conditions are derived through the Lyapunov method using periodic matrix functions. From a perspective of subinterval segmentation aimed at PPTVSs, the proposed conditions can achieve less conservatism for tracking the output of a periodic time-varying reference system, while the controller gains can be computed using convex optimization. Moreover, a heuristic algorithm is constructed to simultaneously guarantee the closed-loop state convergence and the output tracking performance. The reduction in conservatism and the effectiveness of algorithm are demonstrated by illustrative case studies.

Lumbar injection is an image-guided procedure performed manually for diagnosis and treatment of lower back pain and leg pain. Previously, we have developed and verified an MR-Conditional robotic solution to assisting the needle insertion process. Drawing on our clinical experiences, a virtual remote center of motion (RCM) constraint is implemented to enable our robot to mimic a clinician’s hand motion to adjust the needle tip position in situ. Force and image data are collected to study the needle behavior in gel phantoms during this motion, and a mechanics-based needle-tissue interaction model is proposed and evaluated to further examine the underlying physics. This work extends the commonly-adopted notion of an RCM for flexible needles, and introduces new motion parameters to describe the needle behavior. The model parameters can be tuned to match the experimental result to sub-millimeter accuracy, and this proposed needle manipulation method presents a safer alternative to laterally translating the needle during in situ needle adjustments.

This paper investigates the problems of reachable set estimation and synthesis for periodic positive systems with two different exogenous disturbances. The lifting method and the pseudoperiodic Lyapunov function method are adopted for the estimation problem. The reachable set bounding conditions are proposed by employing Lyapunov-based inequalities and the S-procedure technique. Two optimization methods are used to minimize the bounding hyper-pyramids of the reachable set. In addition, the state-feedback controller design conditions that make the reachable set of closed-loop systems lie within a given hyper-pyramid are derived. Finally, numerical examples are presented to illustrate the validity of the obtained conditions.

This paper presents a novel proximity query (PQ) approach capable to detect the collision and calculate the minimal Euclidean distance between two non-convex objects in 3D, namely the robot and the environment. Such approaches are often considered as computationally demanding problems, but are of importance to many applications such as online simulation of haptic feedback and robot collision-free trajectory. Our approach enables to preserve the representation of unstructured environment in the form of triangular meshes. The proposed PQ algorithm is computationally parallel so that it can be effectively implemented on graphics processing units (GPUs). A GPU-based computation scheme is also developed and customized, which shows >200 times faster than an optimized CPU with single core. Comprehensive validation is also conducted on two simulated scenarios in order to demonstrate the practical values of its potential application in image-guided surgical robotics and humanoid robotic control.

Active constraints are high-level control algorithms providing software-generated force feedback from virtual environments. When applied to surgery, they can assist surgeons in performing complex tasks by guiding their navigation pathways along narrow, possibly convoluted, surgical trajectories. This paper presents a method to generate concave tubular constraints implicitly from pre- or intra-operative data. Patient-specific constraints may be generated efficiently with the proposed scheme and readily deployed in various surgical scenarios. Furthermore, a five degree-of-freedom active constraint framework is proposed, which accounts for the entire tool shaft rather than just the end-effector, and is applicable to both static and dynamic active constraint scenarios. Experimental results on simulated surgical tasks show that this framework can improve safety and accuracy as well as reduce the perceived workload during complex surgical tasks.

Proximity Query (PQ) is a process to calculate the relative placement of objects. It is a critical task for many applications such as robot motion planning, but it is often too computationally demanding for real-time applications, particularly those involving human-robot collaborative control. This paper derives a PQ formulation which can support non-convex objects represented by meshes or cloud points. We optimise the proposed PQ for reconfigurable hardware by function transformation and reduced precision, resulting in a novel data structure and memory architecture for data streaming while maintaining the accuracy of results. Run-time reconfiguration is adopted for dynamic precision optimisation. Experimental results show that our optimised PQ implementation on a reconfigurable platform with four FPGAs is 58 times faster than an optimised CPU implementation with 12 cores, 9 times faster than a GPU, and 3 times faster than a double precision implementation with four FPGAs.

This paper presents an Implicit Active Constraints control framework for robot-assisted minimally invasive surgery. It extends on current frameworks by prescribing the external constraints implicitly from the operator motion, forgoing the need for pre-operative imaging; the constraints are defined in situ so as to avoid the use of invasive fiducial markers. A hands-on cooperatively-controlled robotic platform, comprising of a surgical instrument and a compliant manipulator, has been designed for an arthroscopic procedure. The surgical platform is capable of constraining the pose of the instrument so as to ensure it passes through the incision point and does not cause trauma to the surrounding tissue. A flexible arthroscopic instrument is designed and its use is investigated to enlarge reachable and dexterous workspace, increasing the accessibility to the target anatomy. The behaviour of the flexible instrument is analysed. A detailed performance analysis is conducted on a group of subjects for validating the control framework, simulating a minimally invasive arthroscopic procedure. Results demonstrate a statistically significant enhancement in the control ergonomics as well as the accuracy and safety of the procedure.

This paper introduces a gaze contingent controlled robotic arm for laparoscopic surgery, based on gaze gestures. The method offers a natural and seamless communication channel between the surgeon and the robotic laparoscope. It offers several advantages in terms of reducing on-screen clutter and efficiently conveying visual intention. The proposed hands-free system enables the surgeon to be part of the robot control feedback loop, allowing user-friendly camera panning and zooming. The proposed platform avoids the limitations of using dwell-time camera control in previous gaze contingent camera control methods. The system represents a true hands-free setup without the need of obtrusive sensors mounted on the surgeon or the use of a foot pedal. Hidden Markov Models (HMMs) were used for real-time gaze gesture recognition. This method was evaluated with a cohort of 11 subjects by using the proposed system to complete a modified upper gastrointestinal staging laparoscopy and biopsy task on a phantom box trainer, with results demonstrating the potential clinical value of the proposed system.

This paper presents an ungrounded, hand-held surgical device that incorporates active constraints and force-feedback. Optical tracking of the device and embedded actuation allow for real-time motion compensation of a surgical tool as an active constraint is encountered. The active constraints can be made soft, so that the surgical tool tip motion is scaled, or rigid, so as to altogether prevent the penetration of the active constraint. Force-feedback is also provided to the operator so as to indicate penetration of the active constraint boundary by the surgical tool. The device has been evaluated in detailed bench tests to quantify its motion scaling and force-feedback capabilities. The combined effects of force-feedback and motion compensation are demonstrated during palpation of an active constraint with rigid and soft boundaries. A user study evaluated the combined effect of motion compensation and force-feedback in preventing penetration of a rigid active constraint. The results have shown the potential of the device operating in an ungrounded setup that incorporates active constraints with force-feedback.

This paper presents a real-time control framework for a snake robot with hyper-kinematic redundancy under dynamic active constraints for minimally invasive surgery. A proximity query (PQ) formulation is proposed to compute the deviation of the robot motion from predefined anatomical constraints. The proposed method is generic and can be applied to any snake robot represented as a set of control vertices. The proposed PQ formulation is implemented on a graphic processing unit, allowing for fast updates over 1 kHz. We also demonstrate that the robot joint space can be characterized into lower dimensional space for smooth articulation. A novel motion parameterization scheme in polar coordinates is proposed to describe the transition of motion, thus allowing for direct manual control of the robot using standard interface devices with limited degrees of freedom. Under the proposed framework, the correct alignment between the visual and motor axes is ensured, and haptic guidance is provided to prevent excessive force applied to the tissue by the robot body. A resistance force is further incorporated to enhance smooth pursuit movement matched to the dynamic response and actuation limit of the robot. To demonstrate the practical value of the proposed platform with enhanced ergonomic control, detailed quantitative performance evaluation was conducted on a group of subjects performing simulated intraluminal and intracavity endoscopic tasks.

Probe-based confocal laser endomicroscopy (pCLE) provides high resolution imaging of tissue in vivo. Maintaining a steady contact between target tissue and pCLE probe tip is important for image consistency. In this paper, a new prototype hand-held instrument for in vivo pCLE during Minimally Invasive Surgery (MIS) is presented. The proposed instrument incorporates adaptive force sensing and actuation, allowing improved image consistency and force control, thus minimizing tissue deformation and induced micro-structural variations. The performance and accuracy of the contact force control are evaluated in detailed laboratory settings and in vivo validation of the device during transanal microsurgery in a live porcine model further demonstrates the potential clinical value of the device.

This paper describes a multitasking robotic platform for Minimally Invasive Surgery (MIS). The device is designed to be introduced through a standard trocar port. Once the device is inserted to the desired surgical site, it can be reconfigured by lifting an articulated section, and protruding two tendon driven flexible arms. Each of the arms holds an interchangeable surgical instrument. The articulated section features a 2 Degrees-of-Freedom (DoF) universal joint followed by a single DoF yaw joint. It incorporates an on-board camera and LED light source at the distal end, leaving a Ø3mm channel for an additional instrument. The main shaft of the robot is largely hollow, leaving ample space for the insertion of two tendon driven flexible arms integrated with surgical instruments. The ex-vivo and in-vivo experiments demonstrate the potential clinical value of the device for performing surgical tasks through single incision or natural orifice transluminal procedures.

Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

This paper presents a human-robot interface with perceptual docking to allow for the control of multiple microbots. The aim is to demonstrate that real-time eye tracking can be used for empowering robots with human vision by using knowledge acquired in situ. Several micro-robots can be directly controlled through a combination of manual and eye control. The novel control environment is demonstrated on a virtual biopsy of gastric lesion through an endoluminal approach. Twenty-one subjects were recruited to test the control environment. Statistical analysis was conducted on the completion time of the task using the keyboard control and the proposed eye tracking framework. System integration with the concept of perceptual docking framework demonstrated statistically significant improvement of task execution.

This paper presents a novel brush stroke generation scheme based on Genetic Algorithms (GA) and pre-defined brush template models. The work is part of an endeavor to attempt imitating master works of famous past calligraphers. The concept is to parametrize, in some computational sense, the writing styles and techniques of certain calligraphers and then executes the results in a robot drawing platform developed in our laboratory. The present work describes the algorithmic development, simulation studies and experimentation of the GA-based stroke generation scheme upon given certain calligraphic characters. To further study the effectiveness of calligraphic writing with the robot platform, the Cross-Entropy method of the Traveling Salesman problem is incorporated to determine the sequence of stroke execution.

A robot drawing platform supporting four degrees of freedom (x, y, z and z-rotation) of a brush-pen motion for studying Chinese painting and calligraphy has been operational in our laboratory. This paper describes the real-time capturing and data analysis of the brush footprint using the new hardware and software capabilities in the platform. They include a transparent drawing plate and an underneath camera system, together with projective rectification and video segmentation algorithms. Preliminary result of the footprint analysis and nonparametric modeling, and their applications to well-known Chinese calligraphy are demonstrated