Mobility impairments, particularly those caused by stroke-induced hemiparesis, significantly impact independence and quality of life. Current smart walker controllers operate by using input forces from the user to control linear motion and input torques to dictate rotational movement; however, because they predominantly rely on user-applied torque exerted on the device handle as an indicator of user intent to turn, they fail to adequately accommodate users with unilateral upper limb impairments. This leads to increased physical strain and cognitive load. This paper introduces a novel smart walker equipped with a fuzzy control algorithm that leverages shoulder abduction angles to intuitively interpret user intentions using just one functional hand. By integrating a force sensor and stereo camera, the system enhances walker responsiveness and usability. Experimental evaluations with five participants showed that the fuzzy controller outperformed the traditional admittance controller, reducing wrist torque while using the right hand to operate the walker by 12.65 % for left turns, 80.36 % for straight paths, and 81.16 % for right turns. Additionally, average user comfort ratings on a Likert scale increased from 1 to 4. Results confirmed a strong correlation between shoulder abduction angles and directional intent, with users reporting decreased effort and enhanced ease of use. This study contributes to assistive robotics by providing an adaptable control mechanism for smart walkers, suggesting a pathway towards enhancing mobility and independence for individuals with mobility impairments. Project page: https://tbs-ualberta.github.io/fuzzy-sw/

This paper presents a three-dimensional (3D) control framework for bevel-tip steerable needles that combines model predictive control (MPC) with hierarchical supervisory logic. The MPC layer uses a reduced-order two-mode switching model to generate the desired control actions, while the supervisory logic adaptively prioritizes in-plane and out-of-plane corrections based on real-time error magnitudes. This hierarchical approach smoothly modulates the axial rotation commands to minimize abrupt needle flips, thereby reducing the so-called "drilling effect", a key source of tissue trauma. The simulation results show that the proposed approach reduces tissue trauma by more than 50% compared to conventional pulse-width-modulated sliding mode controllers while achieving mean absolute error and targeting errors in the submillimeter range.

A number of recent studies have focused on developing surgical simulation platforms to train machine learning (ML) agents or models with synthetic data for surgical assistance. While existing platforms excel at tasks such as rigid body manipulation and soft body deformation, they struggle to simulate more complex soft body behaviors like cutting and suturing. A key challenge lies in modeling soft body fracture and splitting using the finite-element method (FEM), which is the predominant approach in current platforms. Additionally, the two-way suture needle/thread contact inside a soft body is further complicated when using FEM. In this work, we use the material point method (MPM) for such challenging simulations and propose new rigid geometries and soft-rigid contact methods specifically designed for them. We introduce CRESSim-MPM, a GPU-accelerated MPM library that integrates multiple MPM solvers and incorporates surgical geometries for cutting and suturing, serving as a specialized physics engine for surgical applications. It is further integrated into Unity, requiring minimal modifications to existing projects for soft body simulation. We demonstrate the simulator’s capabilities in real-time simulation of cutting and suturing on soft tissue and provide an initial performance evaluation of different MPM solvers when simulating varying numbers of particles. The source code is available at https://github.com/yafei-ou/CRESSim-MPM.

The irrigation-suction process is a common procedure to rinse and clean up the surgical field in minimally invasive surgery (MIS). In this process, surgeons first irrigate liquid, typically saline, into the surgical scene for rinsing and diluting the contaminant, and then suction the liquid out of the surgical field. While recent advances have shown promising results in the application of reinforcement learning (RL) for automating surgical subtasks, fewer studies have explored the automation of fluid-related tasks. In this work, we explore the automation of both steps in the irrigation-suction procedure and train two vision-based RL agents to complete irrigation and suction autonomously. To achieve this, a platform is developed for creating simulated surgical robot learning environments and for training agents, and two simulated learning environments are built for irrigation and suction with visually plausible fluid rendering capabilities. With techniques such as domain randomization (DR) and imitation learning, two agents are trained in the simulator and transferred to the real world. Individual evaluations of both agents show satisfactory real-world results. With an initial amount of around 5 grams of contaminants, the irrigation agent ultimately achieved an average of 2.21 grams remaining after a manual suction. As a comparison, fully manual operation by a human results in 1.90 grams remaining. The suction agent achieved 2.64 and 2.24 grams of liquid remaining across two trial groups with more than 20 and 30 grams of initial liquid in the container. Fully autonomous irrigation-suction trials reduce the contaminant in the container from around 5 grams to an average of 2.42 grams, although yielding a higher total weight remaining (4.40) due to residual liquid not suctioned. Further information about the project is available at https://tbs-ualberta.github.io/CRESSim/.

Wheeled mobile manipulators (WMMs) involving a wheeled mobile platform and a serial manipulator are finding increasing applications in diverse fields, creating new challenges in performing high-precision operations in a spacious workspace. WMMs are challenging to control due to uncertainties in system parameters, coupled dynamics, and external disturbances, which make stability guarantees difficult. This paper proposes a virtual decomposition control (VDC)-based trajectory tracking controller for WMMs, addressing joint flexibility, external disturbances, etc. The proposed method uses a VDC-based iterative approach to manage the complex coupled dynamics and employs a separate adaptive controller to handle joint flexibility. The robotic system’s stability is validated using the specific features of VDC (proof of each subsystem’s virtual stability) according to the Lyapunov stability theory. The advantages and effectiveness of the proposed method are demonstrated through experiments.Note to Practitioners—This paper addresses the challenges faced in controlling WMMs, which are becoming increasingly common in various industrial and service applications due to their ability to perform tasks in large and dynamic environments. The coupling between the wheeled platform and the manipulator, as well as uncertainties in system parameters such as joint flexibility and external disturbances, make precise trajectory tracking difficult. To address these challenges, this paper presents a control approach based on VDC, which breaks down the complex system into manageable subsystems and ensures stability for each part individually. The control strategy also incorporates adaptive control to handle joint flexibility and unpredictable disturbances. The stability of the system is rigorously proven through Lyapunov theory, ensuring robust performance under real-world conditions. Practitioners working on autonomous mobile robots equipped with manipulators may find this approach useful for improving trajectory tracking performance in uncertain and dynamic environments. However, the practical implementation of this method will require careful tuning of controller parameters and real-time computational capabilities to ensure seamless operation in real applications.

Advancements in robotic systems aimed at improving mobility for individuals with disabilities have required more sophisticated control and navigation methods. Traditional control approaches often lack the complexity and adaptability needed for the high-dimensional nature of human activities. Consequently, reinforcement learning (RL) has emerged as a dynamic and effective framework for managing robotic actions in complex and unpredictable human environments. This article reviews the integration of RL in robotic systems for enhancing the mobility of individuals with disabilities, addressing the limitations of traditional control methods in complex and unpredictable environments. We critically analyze various RL algorithms, discussing their advantages and challenges in assistive and rehabilitation applications. The study highlights the ongoing development of these algorithms, presenting current research directions, future prospects, and key challenges to achieving higher autonomy in assistive robots. Our findings underscore the potential of RL to improve adaptability and effectiveness in robotic control and navigation, offering insights into advancing these technologies for practical implementations.

One of the most common cancers among women is breast cancer which can be treated surgically in the early stages with a lumpectomy technique. In the context of breast lumpectomy procedures, accurately tracking tumours presents a critical challenge worsened by various sources of anatomical deformations, including breathing, tissue cutting, and ultrasound probe pressure. To address this, we explore how a realistic tissue deformation simulator can enhance the precision of locating internal targets by accurately assessing the deformation applied to a preoperative model of the breast, considering the distinct mechanical properties of both the breast tissue and the tumour within it. Our method uses advanced artificial intelligence techniques by combining a generative variation autoencoder (GNN-VAE) and an updating method called ensemble smoother with multiple data assimilation (ES-MDA), creating a dynamic model based exclusively on surface node data to update all nodes within the tissue. By leveraging a realistic tissue deformation simulator, our approach uses breast surface tracking to infer full tissue deformations. This makes the method compatible with various simulation tools and suitable for tissues with complex properties. The results indicate that the trained network has an accuracy of 0.014 cm with training data, and 0.026 cm with the testing portion of data, demonstrating precision in tumour localization and significantly improving upon current methods. This innovation can potentially enhance patient outcomes by making breast cancer surgery safer, less invasive, and more efficient.

Gait asymmetry, a consequence of various neurological or physical conditions such as aging and stroke, detrimentally impacts bipedal locomotion, causing biomechanical alterations, increasing the risk of falls and reducing quality of life. Addressing this critical issue, this paper introduces a novel diagnostic method for gait symmetry analysis through the use of an assistive robotic Smart Walker equipped with an innovative asymmetry detection scheme. This method analyzes sensor measurements capturing the interaction torque between user and walker. By applying a seasonal-trend decomposition tool, we isolate gait-specific patterns within these data, allowing for the estimation of stride durations and calculation of a symmetry index. Through experiments involving 5 experimenters, we demonstrate the Smart Walker’s capability in detecting and quantifying gait asymmetry by achieving an accuracy of 84.9% in identifying asymmetric cases in a controlled testing environment. Further analysis explores the classification of these asymmetries based on their underlying causes, providing valuable insights for gait assessment. The results underscore the potential of the device as a precise, ready-to-use monitoring tool for personalized rehabilitation, facilitating targeted interventions for enhanced patient outcomes.

This paper introduces a novel hybrid torque allocation method for improving wearability and mobility in integrated functional electrical stimulation (FES) of the quadriceps muscles and powered exoskeleton systems. Our proposed approach leverages a hierarchical closed-loop controller for knee joint position tracking while addressing limitations of powered exoskeletons and FES systems by reducing power consumption and battery size and by mitigating FES-induced muscle fatigue, respectively. The core component is a model-free optimization algorithm that dynamically distributes torque between FES and the exoskeleton by considering tracking error, effort, and the prediction of muscle fatigue in the cost function, computing allocation gain in an online manner. The online optimization approach interactively changes the optimal allocation gain by taking into account the instantaneous value of error and effort and also penalizing FES-induced fatigue, a common challenge in long-duration experiments. The results demonstrate that this dynamic allocation significantly improves system wearability by reducing power consumption without increasing muscle fatigue during the extension phase of walking. This hybrid control approach contributes to improving exoskeleton wearability and rehabilitation outcomes for individuals with SCI and mobility impairments, enhancing assistive technology and quality of life.

This paper introduces intelligent central pattern generators (iCPGs) that can plan personalized walking trajectories for lower-limb exoskeletons. This can make walking more comfortable for the users by resolving one of the significant shortcomings of most commercially available exoskeletons, which is the use of pre-defined fixed trajectories for all users. The proposed method combines reinforcement learning (RL) with previously introduced adaptable central pattern generators (ACPGs) to learn a user's physical interaction behaviour and refine the exoskeleton's walking trajectories. The ACPG method embeds physical human-robot interaction (pHRI) in CPGs to make changing gait trajectories in real-time, possible. However, to effectively refine gait trajectories based on pHRIs, the parameters must be precisely identified and updated as a user interacts with the exoskeleton. Our proposed method uses RL to modify (amplify/attenuate) the pHRI energy based on a user's interaction behaviour, and form an effective energy value which can facilitate reaching desired gait pattern for users via iCPG dynamics. The proposed method can resolve the aforementioned challenges with ACPGs and personalized trajectory generation. The simulation and experimental results provide evidence that the proposed method can effectively adapt to the user's behaviour in different walking scenarios with the Indego lower-limb exoskeleton.

Learning from demonstrations is the paradigm where robots acquire new skills demonstrated by an expert and alleviate the physical burden on experts to perform repetitive tasks. Ultrasound scanning is one of the ways to view the anatomical structures of soft tissues, but it is repetitive for some tissue scanning tasks. In this study, an autonomous ultrasound scanning towards a standard plane framework is proposed. Interaction probabilistic movement primitives (iProMP) was proposed for the collaborative tasks for human and robot movement. Inspired by the interval type-2 fuzzy system, an interval iProMP is proposed to learn the ultrasound scanning navigation strategy from scanning demonstrations and the collaborative agents are the robot movement and ultrasound image information. The proposed interval iProMP improves the capacity of dealing with uncertainties due to insufficient observations during reproduction. U-Net is applied to recognize the desired ultrasound image shown during demonstrations and a confidence map is used to evaluate the ultrasound image quality. Breast seroma scanning is chosen as the ultrasound scanning task to validate the performance of the proposed autonomous ultrasound scanning framework. Ultrasound navigation is to realize autonomous ultrasound scanning for localizing the breast seroma. The simulation comparison result shows the better performance of the proposed interval iProMP under insufficient observation, compared to traditional iProMP. The experiment result validates the feasibility and generality of the proposed autonomous ultrasound scanning framework using interval iProMP with a higher success rate than that with traditional iProMP.

This article proposes the novel concepts of the high-order discrete-time control barrier function (CBF) and adaptive discrete-time CBF. The high-order discrete-time CBF is used to guarantee forward invariance of a safe set for discrete-time systems of high relative degree. An optimization problem is then established unifying high-order discrete-time CBFs with discrete-time control Lyapunov functions to yield a safe controller. To improve the feasibility of such optimization problems, the adaptive discrete-time CBF is designed, which can relax constraints on system control input through time-varying penalty functions. The effectiveness of the proposed methods in dealing with high relative degree constraints and improving feasibility is verified on the discrete-time system of a three-link manipulator.

Robot-assisted arthroscopic surgery is transforming the tradition in orthopaedic surgery. Compliance and stability are essential features that a surgical robot must have for safe physical human-robot interaction (PHRI). Surgical tools attached at the robot end-effector and human-robot interaction will affect the robot dynamics inevitably. This could undermine the utility and stability of the robotic system if the varying robot dynamics are not identified and updated in the robot control law. In this paper, an integrated frame-work for robot impedance control and nonlinear disturbance observer (NDOB)-based compensation of uncertain dynamics is proposed, where the former ensures compliant robot behavior and the latter compensates for dynamic uncertainties when necessary. The combination of impedance controller and NDOB is analyzed theoretically in three scenarios. A complete simulation and experimental studies involving three common conditions are then conducted to evaluate the theoretical analyses. A preliminary$p$HRI application on arthroscopic surgery is designed to implement the proposed framework on a robotic surgeonassist system and evaluate its effectiveness experimentally. By integrating impedance controller with NDOB, the proposed framework allows an accurate impedance control when dynamic model inaccuracy and external disturbance exist.

Ultrasound (US) imaging is a common but physically demanding task in the medical field, and sonographers may need to put in considerable physical effort for producing high-quality US images. During physical human-robot interaction on US imaging, robot compliance is a critical feature that can ensure human user safety while automatic force regulation ability can help to improve task performance. However, higher robot compliance may mean lower force regulation accuracy, and vice versa. Especially, the contact/non-contact status transition can largely affect the control system stability. In this paper, a novel electromyography (EMG)-based hybrid impedance-force control system is developed for US imaging task. The proposed control system incorporates the robot compliance and force regulation ability via a hybrid controller while the EMG channel enables the user to online modulate the trade-off between the two features as necessary. Two experiments are conducted to examine the hybrid controller and show the necessity of involving an EMG-based modulator. A proof-of-concept study on US imaging is performed with implementing the proposed EMG-based control system, and the effectiveness is demonstrated. The proposed control system is promising to ensure robot's stability and patient's safety, thus obtain high-quality US images, while monitoring and reducing sonographer's fatigue. Furthermore, it can be easily adapted to other physically demanding tasks in the field of medicine.

This paper proposes a variable impedance control architecture to facilitate eye surgery training in a dual-user haptic system. In this system, an expert surgeon (the trainer) and a novice surgeon (the trainee) collaborate on a surgical procedure using their own haptic devices. The mechanical impedance parameters of the trainer's haptic device remain constant during the operation, whereas those of the trainee vary with his/her proficiency level. The trainee's relative proficiency might be objectively quantified in real-time based on position error between the trainer and the trainee. The proposed architecture enables the trainer to intervene in the training process as needed to ensure the trainee is following the right course of action and to avoid the trainee's from potential tissue injuries. The stability of the overall nonlinear closed-loop system has been investigated using the input-to-state stability (ISS) criterion. High-gain observer with unknown inputs is considered in this work to estimate the interaction forces. Simulation and experimental results under different scenarios confirm the effectiveness of the proposed control methods.

This article presents an overview of the current state of research and application of haptic (primarily kinesthetic) feedback and force-based teleoperation in the context of surgical robotics. Telerobotic surgery provides an approach for transferring the sensorimotor skills of a surgeon through a robotic platform to perform surgical intervention inside a patient’s body. Integration of advanced sensing and haptic technologies in telerobotic surgery can help to enhance the sensory awareness and motor accuracy of the surgeon, thereby leading to improved surgical procedures and outcomes for patients. The primary mode of sensory feedback has been through 3-D visual observation using stereo endoscopes. However, until recently, the sense of touch, i.e., haptics, has been missing in the commercial telesurgery robots approved for use in the operating room despite over two decades of research and development in the field of haptics for teleoperated systems (“telehaptics”). Research has shown that high-fidelity force feedback can enhance the performance of telesurgery and potential outcomes by enabling the surgeon to have a more natural feel of interaction between surgical tools and tissue as normally experienced during open surgery. Interaction forces, such as those generated during palpation of tissue, insertion of a needle, unintentional (and potentially unsafe) exertion of force by a tool, suture breakage, needle slippage, or tool interaction, are replaced by indirect (virtual) sensations, termed visual haptics, which provides an alternative to sensory compensation. Although there is a significant amount of literature supporting this benefit, there are still several important technical challenges in introducing haptics in telesurgery, including instrumentation, fidelity (transparency), stability, and modalities for force reflection, e.g., direct or indirect. This article examines these challenges and discusses recent work on haptics-based teleoperated surgical robotic systems.

In this survey, various concepts and methodologies developed over the past two decades for varying and learning the impedance or admittance of robotic systems that physically interact with humans are explored. For this purpose, the assumptions and mathematical formulations for the online adjustment of impedance models and controllers for physical human-robot interaction (HRI) are categorized and compared. In this systematic review, studies on: 1) variation and 2) learning of appropriate impedance elements are taken into account. These strategies are classified and described in terms of their objectives, points of view (approaches), and signal requirements (including position, HRI force, and electromyography activity). Different methods involving linear/nonlinear analyses (e.g., optimal control design and nonlinear Lyapunov-based stability guarantee) and the Gaussian approximation algorithms (e.g., Gaussian mixture model-based and dynamic movement primitives-based strategies) are reviewed. Current challenges and research trends in physical HRI are finally discussed.

With the increasing applications of wheeled mobile manipulators (WMMs), consisting of a mobile platform (MP) and a manipulator, in diverse fields, new challenges have arisen in achieving multiple tasks such as obstacle avoidance in a constrained environment during the end-effector (EE) operation. A WMM is usually redundant due to the combination of the MP and the manipulator, making multitask control possible via employing its null space. Dual-user/two-handed teleoperation of a WMM is desirable for tasks where it is important to simultaneously control the poses of both the MP and the EE. The existing teleoperation approaches for WMMs are mostly executed at the kinematic level, without considering the nonlinear rigid-body dynamics of the WMMs. In this article, a task-priority-based dual-user teleoperation framework for a WMM is implemented to perform tasks in a constrained environment. It can simultaneously manipulate the MP and the EE, the overground obstacles are avoided by telecontrolling the MP using the WMM’s null space. Any residual redundancy can be further employed for other tasks such as singularity avoidance. The stability of the entire teleoperation design is rigorously proved even with arbitrary time delays. Experiments with a dual-user teleoperation system, consisting of two local robots and an omnidirectional WMM, are conducted to verify the proposed approach’s feasibility and effectiveness.

During the COVID-19 pandemic, due to the unprecedented workload and cross-infection hazard, the health-care workers’ lives are under a significant threat. However, minimizing the duration and frequency of close clinician-to-patient contacts using simple technologies that enable physical distancing could reduce the risk of spreading the disease. In this context, this paper presents the conceptual design and preliminary assessment of a low-cost and intrinsically safe remote service delivery platform that can assist clinicians in doing various tasks at a safe distance from patients. This mechanism is capable of manipulating objects in three-dimensional Cartesian space and can be adapted to handling a wide variety of medical devices. Moreover, its passive weight-compensating design provides the mechanism with high maneuverability, enhanced dynamic manipulability, and better force feedback quality. The advantages and effectiveness of the proposed mechanism are demonstrated through experiments. In the experiment, an ultrasound probe is mounted at the end effector of the device to perform an imaging task from a safe distance. Due to the existence of the force feedback, the user could remotely manipulate the ultrasound probe for having a successful vertical and pivot scanning to get high-quality images with a low physical and mental demand.

In this paper, we propose a robotic ultrasound imaging method that scans the breast in two separate phases to acquire high-quality ultrasound images. Our proposed system controls five Degrees of Freedom (DoFs) of the robot that hold an ultrasound probe to perform precise scanning. This system finds the desired trajectory based on geometrical analysis of the target inside the breast in a pre-scan phase and uses this information to control the probe in a post-scan phase. The proposed method updates the desired values of rotational and translational movement of the probe in the post-scan by calculating the center of mass of segmented target in each acquired frame and the average of image confidence map. The proposed method has been tested experimentally on a plastisol phantom. Given a specific trajectory, the position and orientation of the probe have been controlled at each point of the trajectory. The experiments’ result shows us that our proposed visual servoing algorithm successfully controls the probe to look at target tissue and is fast enough for use in a robotic control loop.

Due to the high sensitivity and complexity of robotic surgery tasks, acquiring appropriate skill levels by trainee surgeons through an effective training process is very important and affects the patient’s safety and the quality of surgical outcomes. With the advanced deep learning technology and the recent availability of surgical procedures data, intelligent methods can be deployed to assess and transfer the skills of an experienced surgeon (mentor) to a novice surgeon (trainee). In this paper, we introduce a novel deep-learning-based skill transfer scheme consisting of a deep convolutional model, SkillNet, and a skill transfer algorithm for robotic surgery training. The proposed SkillNet extracts skill-related features of the mentor from different layers of the network. Then, trainee’s maneuver is enhanced by the proposed skill transfer algorithm while minimizing deviations from the trainee’s original intended trajectory. For validation, the JIGSAWS dataset and also our own experimental data were used to prove the generalizability of SkillNet in capturing skill-related features. The capability of the skill transfer algorithm in enhancing trainee trajectories in terms of predictability, hand tremor reduction, and noise cancellation were investigated separately. The obtained results indicate that this approach can be used as a high-performance filter that makes minor corrections to the input trajectory and improves the skill level of the trainee’s trajectory in practice.

Human-robot collaboration (HRC) significantly extends robotic systems’ applications when working in spaces like houses, hospitals, or laboratories. However, new challenges appear during a close collaboration between humans and robots and imitating the movement of humans by robots. Learning from demonstration (LfD), or kinesthetic teaching, is a popular approach to help teach a robot human behavior by demonstrations without the need to explicitly reprogram the robot for different procedures. In this paper, we propose a method for object manipulation, including lifting, carrying, and lowering the object through a collaboration of a human with a wheeled mobile manipulator (WMM). The WMM is first trained with the help of a human demonstrator to collaborate with the user to execute the task. Then, the WMM will independently cooperate with the user by reproducing the learned skills to perform the same task. The redundancy of the WMM will also be employed to enhance its force exertion capability in the vertical direction to offset the object’s weight. The advantages and effectiveness of the proposed method are investigated through experiments.

In this paper, a new remote center of motion (RCM) mechanism is presented whose end-effector is able to move through an entire hemisphere. In general, minimally invasive surgery (MIS) applications, an elliptic cone workspace with vertex angles of 60° and 90° gives the surgeon enough freedom to operate. Therefore, the majority of the developed RCM mechanisms have such a cone as the workspace. However, there are still situations in which a larger workspace is required, like the breast ultrasound scanning application in which the RCM mechanisms should be able to move over a hemisphere to do the breast scanning. The proposed RCM mechanism is developed based upon a spherical scissor linkage and benefits from the high stiffness characteristics of parallel structures while eliminating the common problem of linkage collision in parallel structures. It has two rotational degrees of freedom that are decoupled from each other. The Jacobian and the stiffness of the mechanism while considering the bending of the links is calculated through the virtual joints method (VJM). The kinemato-static equations and the methodology for calculating stiffness are described in detail. The optimal arc angle of the mechanism's links is found using a multi-objective genetic algorithm optimization. A prototype of the mechanism is built and forward kinematic of the proposed mechanism is examined experimentally. The experiments indicate that the proposed mechanism is able to provide a hemisphere as its workspace while the RCM point of the mechanism is fixed in the space.

This paper presents a "Learning from Demonstration" method to perform robot movement trajectories that can be defined as you go. This way unstructured tasks can be performed, without the need to know exactly all the tasks and start and end positions beforehand. The long-term goal is for children with disabilities to be able to control a robot to manipulate toys in a play environment, and for a helper to demonstrate the desired trajectories as the play tasks change. A relatively inexpensive 3-DOF haptic device made by Novint is used to perform tasks where trajectories of the end-effector are demonstrated and reproduced. Under the condition where the end-effector carries different loads, conventional control systems possess the potential issue that they cannot compensate for the load variation effect. Adaptive tracking control can handle the above issue. Using the Lyapunov stability theory, a set of update laws are derived to give closed-loop stability with proper tracking performance.

The increasing application of wheeled mobile robots (WMR) in many fields has brought new challenges on its control and teleoperation, two of which are induced by contact slippage phenomenon between wheel and terrain as well as time delays in the master-slave communication channel. In the WMR bilateral tele-driving system, in this paper, the linear velocity of the slave mobile robot follows the position command from the haptic master robot while the slippage-induced velocity error is fed back as a haptic force felt by the human operator. To cope with the slippage-induced nonpassivity and constant time delays, this paper proposes three methods to design the WMR bilateral teleoperation system's controller. An experiment system is set up with Phantom Premium 1.5A haptic device as the master robot and a simulation platform of WMR as the slave robot. Experiments with the proposed methods demonstrate that they can result in a stable WMR bilateral tele-driving system under wheel's slippage and constant time-delays.

Due to the limitations of therapists time and healthcare resources to cover the increasing demand for rehabilitation services, robot-assisted rehabilitation is becoming an appealing, powerful and economical solution. In our previous research, a solution that combines Learning from Demonstration (LfD) and robotic rehabilitation to save the therapists time and reduce the therapy costs was proposed. In this paper we compare two modalities, Robot-and Telerobotic-Mediated Kinesthetic Teaching (RMKT and TMKT), for implementing LfD in robotic rehabilitation. Our results show that behaviors demonstrated in both modalities are able to be imitated accurately, but demonstrations in TMKT have less repeatability.

Robotic-assisted needle steering can enhance the accuracy of needle-based interventions. Application of current needle steering techniques are restricted by the limited deflection curvature of needles. Here, a novel steerable needle with improved curvature is developed and used with an online motion planner to steer the needle along curved paths inside tissue. The needle is developed by carving series of small notches on the shaft of a standard needle. The notches decrease the needle flexural stiffness, allowing the needle to follow tightly curved paths with small radius of curvature. In this paper, first, a finite element model of the notched needle deflection in tissue is presented. Next, the model is used to estimate the optimal location for the notches on needle's shaft for achieving a desired curvature. Finally, an ultrasound-guided motion planner for needle steering inside tissue is developed and used to demonstrate the capability of the notched needle in achieving high curvature and maneuvering around obstacles in tissue. We simulated a clinical scenario in brachytherapy, where the target is obstructed by the pubic bone and cannot be reached using regular needles. Experimental results show that the target can be reached using the notched needle with a mean accuracy of 1.2 mm. Thus, the proposed needle enables future research on needle steering toward deeper or more difficult-to-reach targets.

The inclusion of robots in therapy is becoming common due to robots' power, repetitive motion ability, reprogramming capacity and adaptability to new tasks. In recent years, the demand for rehabilitation services has increased due to the rising number of patients with disability. In this paper, we propose a solution to the rising demand for therapists' services by combining Learning from Demonstration (LfD) and robotic rehabilitation. The goal of the paper is to implement LfD to model and learn the therapist's behavior (be it a trajectory or force) as a nonlinear dynamic system using a method called Stable Estimator of Dynamical Systems (SEDS) to later reproduce the learned behavior in the absence of the therapist using a robot. This method allows the therapists to first train a robot to learn his/her behavior such that, later when the therapist is no longer involved and the patient works alone with the robot, the robotic system determines whether and how to interact with the patient the same way the therapist would have interacted.

Telerobotic manipulation allows patients living with upper limb impairments to interact with a variety of environments and accomplish through teleoperation daily activities such as playing, feeding, self-care, and leisure, that would otherwise be difficult to perform. In this paper, we propose a nonlinear mapping between the patient's range of motion and the workspace of an environment being manipulated. The objective is to identify the patient's workspace and span it to that of the environment or an object, thus optimizing the scaling factor while soliciting the entire patient's range of motion. The boundaries of each workspace are obtained from scattered measurements of the master and slave robots end-effector position. The nonlinear mapping is then achieved through thin plate spline interpolation that describes deformation between two surfaces by scattered point-to-point preponderances. Experimental results reported in three different scenarios confirm the suitability of the nonlinear transformation to map diverse workspace volumes.

The process through which children learn about the world and develop perceptual, cognitive and motor skills relies heavily on object exploration in their physical world. New types of assistive technology that enable children with impairments to interact with their environment have emerged in recent years, and they could be beneficial for children's cognitive and perceptual skills development. Many studies have reported on brain computer interface (BCI) research. However, a conventional electroencephalography (EEG) system is generally bulky and expensive. It also requires special equipment and technical expertise to operate successfully. In this study, a compact low-cost EEG system was used to detect signals related to movement intention and control a mobile robot control. EEG signals of three non-disabled adults were acquired by the BCI system and the movement intention was classified during physical movement and motor imagery. The average classification accuracies achieved during testing were 56.4% for the motor imagery and 72.7% for the physical movement. The results show moderate classification accuracy for the motor imagery; however, the classification accuracy for the physical movement was high for all the subjects. Even though further improvement of the system is still needed, the experimental results demonstrated the feasibility of a BCI-based robotic system that is affordable and accessible for many people including children with disabilities.

The problem of synchronization of bilateral teleoperators in the presence of stochastic disturbances in control inputs is considered in this paper. It is clear that the mechanical systems are often subjected to random disturbances and it can influence the performance of the control system in an uncertain manner. To cope with this, the new adaptive controller is proposed. This technique uses the exponential practical stability concept which guarantees that the tracking error and its derivative converge to an arbitrarily small neighborhood of zero by appropriate tuning of the controller's parameters. It is noteworthy that, the proposed method does not need information, such as the physical parameters of the master and slave robots. Finally, the simulation results are given to show the effectiveness of proposed technique.

Despite recent advances to improve transparency of teleoperation systems, certain tasks remain difficult and time-consuming when performed via teleoperation. Operators are often required to perform tasks involving multiple degrees-of-freedom (DOFs) requiring great dexterity. To be fully adopted, the speed and ease of teleoperated task performance must be increased. A possibility is to use cooperative manipulation, namely two-handed teleoperation, to allow the two hands of the user to manipulate two master haptic devices in order to control a slave robot with multiple DOFs; the total DOFs of the two masters are equal to the DOFs of the slave. We present the results of a user study that evaluates the performance of a bimanual teleoperation system involving two 3-DOF haptic master interfaces to control a 6-DOF slave manipulator. The two master's motions are complementary. Then, we compare this performance to a single-master/single-slave teleoperation system using 6-DOF master and slave manipulators. In order to compare the users' performance in the two systems, a 6-DOF task experiment is considered. The task performed resembles typical tasks carried out in surgery. The results of our study suggest that DOF decomposition leads to significant improvements regarding task completion time and trajectory tracking for a task which involves following a pattern while maintaining a desired depth.

In this paper, we propose an automated method to reconstruct the three-dimensional (3-D) needle shape during needle insertion procedures using only 2-D transverse ultrasound (US) images. Using a set of transverse US images, image processing and random sample consensus are used to locate the needle within each image and estimate the needle shape. The method is validated with an in vitro needle insertion setup and a transparent tissue phantom, where two orthogonal cameras are used to capture the true 3-D needle shape for verification. Results showed that the use of at least three images obtained at 75% of the maximum insertion depth or greater allows for maximum needle shape estimation errors of less than 2 mm. In addition, the needle shape can be calculated consistently as long as the needle can be identified in 30% of the transverse US images obtained. Application to permanent prostate brachytherapy is also presented, where the estimated needle shape is compared to manual segmentation and sagittal US images. Our method is intended to help to assess needle placement during manual or robot-assisted needle insertion procedures after the needle has been inserted.

We propose a method to estimate the entire shape of a long flexible needle, suitable for a needle insertion assistant robot. This method bases its prediction on only a small segment of a needle, imaged via ultrasound, after insertion. An algorithm is developed that can segment a needle observed partially in ultrasound images and fully in camera images, returning a polynomial representation of the needle shape after RANSAC processing. The polynomial corresponding to the partial needle observation in ultrasound images is used as the input to a needle-tissue interaction model that predicts the entire needle shape. The needle shape predicted by the model is compared to the segmented needle shape based on camera images to validate the proposed approach. The results show that the entire needle shape can be accurately predicted in tissues of varying stiffness based on observation of parts of the needle in an ultrasound image.

In needle-based medical procedures, beveled-tip flexible needles are steered inside soft tissue with the aim of reaching pre-defined target locations. The efficiency of needle-based interventions depends on accurate control of the needle tip. This paper presents a comprehensive mechanics-based model for simulation of planar needle insertion in soft tissue. The proposed model for needle deflection is based on beam theory, works in real-time, and accepts the insertion velocity as an input that can later be used as a control command for needle steering. The model takes into account the effects of tissue deformation, needle-tissue friction, tissue cutting force, and needle bevel angle on needle deflection. Using a robot that inserts a flexible needle into a phantom tissue, various experiments are conducted to separately identify different subsets of the model parameters. The validity of the proposed model is verified by comparing the simulation results to the empirical data. The results demonstrate the accuracy of the proposed model in predicting the needle tip deflection for different insertion velocities.

A tissue-independent model to estimate needle deflection during insertion in soft tissue is presented in this paper. A force/torque sensor is connected to the needle base in order to measure forces and moments during insertion due to needle deflection. A static mechanical model, which is based on the Euler-Bernoulli beam equation and the balance of forces applied by the tissue onto the needle takes these force and moment measurements as input. The needle tip deflection can then be calculated based on the beam model undergoing these forces. Three different needle-tissue interaction models are presented. Their estimation performance is evaluated and experimentally compared by carrying out insertion experiments into phantom tissue. The experimental results show a precise estimate of needle tip deflection for a novel virtual sensor introduced in this work. The main advantage of this virtual sensor approach is that measurements obtained from the force/torque sensor are the only necessary model inputs. Furthermore, the approach does not rely on ultrasound or other image-based needle observation techniques. This makes the virtual sensor suitable for real-time feedback of needle tip deflection.

This paper presents a Therapist-in-the-Loop (TIL) framework for robotics-assisted mirror rehabilitation therapy integrated with adaptive Assist-as-Needed (ANN) training, to be adjusted based on the impairment and disability level of the patient's affected limb. Closed-loop system stability has been investigated using a combination of the Circle Criterion and the Small-Gain Theorem to account both for time-delay and the time-varying adaptive ANN training. Experiments to investigate the performance of the proposed framework are reported.

This paper introduces an automatic method to visualize 3D needle shapes for reliable assessment of needle placement during needle insertion procedures. Based on partial observations of the needle within a small sample of 2D transverse ultrasound images, the 3D shape of the entire needle is reconstructed. An intensity thresholding technique is used to identify points representing possible needle locations within each 2D ultrasound image. Then, a Random Sample and Consensus (RANSAC) algorithm is used to filter out false positives and fit the remaining points to a polynomial model. To test this method, a set of 21 transverse ultrasound images of a brachytherapy needle embedded within a transparent tissue phantom are obtained and used to reconstruct the needle shape. Results are validated using camera images which capture the true needle shape. For this experimental data, obtaining at least three images from an insertion depth of 50 mm or greater allows the entire needle shape to be calculated with an average error of 0.5 mm with respect to the measured needle curve obtained from the camera image. Future work and application to robotics is also discussed.

In this paper, a new passivity-based technique is proposed to analyze and guarantee the stability of haptics-enabled telerobotic rehabilitation systems where there is a possibility of having more sources of non-passivity than communication delays. In practice, the difficulty of therapeutic exercises should be tuned taking into account the stage of physical disability. However, tuning the difficulty and intensity should not violate the stability of patient-robot interaction. This usually puts conservative prefixed limits on the allowable exercise intensity. In this paper, patient-robot interaction safety is studied in the context of Strong Passivity Theory (SPT). Our goal is to ultimately relax the limitation on the allowable robotic therapies while preserving system stability. The proposed stabilizing scheme does not try to make the entire non-passive component passive. This allows the therapist to have freedom in injecting energy into the system for assistive therapies while ensuring safe patient-robot interaction. In this paper, the case of telerobotic rehabilitation is considered. Experimental implementation and evaluation are presented to support the proposed theory.

This paper represents an extension to the kinematic bicycle model for beveled-tip needle motion in soft tissue, which accounts for non-constant curvature paths for the needle tip. For a tissue that is not stiff relative to the needle, the tissue deformation caused by needle insertion deviates the needle tip position from a constant curvature path. The proposed model is obtained by replacing the bicycle wheels with omnidirectional wheels that move in two orthogonal directions independently. Such wheels can move sideways, providing a means for modeling the deviations of the needle tip from a constant curvature path by incorporating new parameters in the model. Using an experimental setup, the needle is inserted into soft phantom tissue at different constant velocities and model parameters are fitted to experimental data. The model is verified by comparing the results from the model to empirical data.

This paper presents a comparison of three-dimensional (3-D) segmentations of the prostate, based on two-dimensional (2-D) manually segmented contours, obtained using ultrasound (US) and magnetic resonance (MR) imaging data collected from 40 patients diagnosed with localized prostate cancer and scheduled to receive brachytherapy treatment. The approach we propose here for 3-D prostate segmentation first uses radial basis function interpolation to construct a 3-D point distribution model for each prostate. Next, a modified principal axis transformation is utilized for rigid registration of the US and MR images of the same prostate in preparation for the following shape comparison. Then, statistical shape models are used to capture the segmented 3-D prostate geometries for the subsequent cross-modality comparison. Our study includes not only cross-modality geometric comparisons in terms of prostate volumes and dimensions, but also an investigation of interchangeability of the two imaging modalities in terms of automatic contour segmentation at the pre-implant planning stage of prostate brachytherapy treatment. By developing a new scheme to compare the two imaging modalities in terms of the segmented 3-D shapes, we have taken a first step necessary for building coupled US-MR segmentation strategies for prostate brachytherapy pre-implant planning, which at present is predominantly informed by US images only.

Beating-heart surgery is not currently possible for most surgical procedures as it requires superhuman skill to manually track the heart's motion while performing a surgical task. However, if a surgical tool could track the motion of the point of interest (POI) on the heart, then, with respect to the surgical tool tip the POI would appear stationary. Such a system can be created with a teleoperated surgical robot that is controlled to track the combination of the heart's and the surgeon's motion, as input through a separate user console. To develop such a system, the motion of the heart is found in ultrasound images where the image acquisition introduces delays of approximately 40 ms and image processing further increases this delay. Directly using this delayed position measurement in the feedback control loop can lead to instability and poor tracking. The generalized predictive controller used in this work compensates for this time delay despite large disturbances with velocities up to 210 mm/s and accelerations up to 3800 mm/s2caused by the moving heart.

A pneumatic actuator with solenoid valves is a discontinuous-input system because each valve can be either in on or off state. For such an actuator, this paper proposes a sliding-mode control scheme based on an averaged continuous-input equivalent model for the open-loop system. The averaged model is obtained from the nonlinear dynamics of the open-loop discontinuous-input system undergoing pulse-width-modulation (PWM) at the input (i.e., valve open/close action). The PWM duty cycle will be regarded as a continuous input to the proposed averaged model, and thus generated by the proposed sliding-mode controller. By adjusting the PWM duty cycle, the controller switches between seven modes of operation of the open-loop system in order to select the ones with necessary and sufficient amounts of drive energy to achieve position tracking. We will show that this results in reduced position error and valve switching activity for the actuator. The proposed control scheme is experimentally used in the position control of a pneumatic actuator and the results are presented.

Performing surgery on fast-moving heart structures while the heart is freely beating is next to impossible. Nevertheless, the ability to do this would greatly benefit patients. By controlling a teleoperated robot to continuously follow the heart's motion, the heart can be made to appear stationary. The surgeon will then be able to operate on a seemingly stationary heart when in reality it is freely beating. The heart's motion is measured from ultrasound images and thus involves a non-negligible delay due to image acquisition and processing, estimated to be 150 ms that, if not compensated for, can cause the teleoperated robot's end-effector (i.e., the surgical tool) to collide with and puncture the heart. This research proposes the use of a Smith predictor to compensate for this time delay in calculating the reference position for the teleoperated robot. The results suggest that heart motion tracking is improved as the introduction of the Smith predictor significantly decreases the mean absolute error, which is the error in making the distance between the robot's end-effector and the heart follow the surgeon's motion, and the mean integrated square error.

Conventional approaches for stability analysis of bilateral teleoperation systems assume that the human operator does not inject energy into the system and behaves in a passive manner. Does this assumption hold for various tasks the human operator may execute in a teleoperation context? To answer this question, in this paper we measure the endpoint impedance (inertia, viscosity, and stiffness) of the human arm during two tasks: (1) relaxed grasping of a haptic device while the device imposes position perturbations, and (2) rigid grasping of a haptic device (posture maintenance) while the device imposes force perturbations. The human arm impedance is identified as a 2 × 2 transfer function matrix and assessed for passivity over the frequency range characteristic of human motion. Our results agree with previous findings that the relaxed human arm behaves as a passive system. However, whether the rigid arm behaves as an active or passive system is found to depend on the magnitude of the force perturbations. We discuss why the passivity of the human operator is task dependent.

In this paper, a novel control scheme is proposed to guarantee global asymptotic stability of bilateral teleoperation systems that are subject to time-varying time delays in their communication channel and “sandwich linearity” in their actuators. This extends prior art concerning control of nonlinear bilateral teleoperation systems under time-varying time delays to the case where the local and the remote robots' control signals pass through saturation or similar nonlinearities that belong to a class of systems we name sandwich linear systems. We call the proposed method nonlinear proportional plus damping (nP+ D). The asymptotic stability of the closed-loop system is established using a Lyapunov-Krasovskii functional under conditions on the controller parameters, the actuator saturation characteristics, and the maximum values of the time-varying time delays. The controller is experimentally validated on a pair of 3-DOF PHANToM Premium 1.5A robots, which have limited actuation capacity, that form a teleoperation system with a varying-delay communication channel.

A bilateral teleoperation system includes a human operator and an environment, which make the system stability analysis complicated due to their unknown, time-varying and nonlinear nature. Unable to have exact models for the human operator and the environment, it is typically assumed that they are passive but otherwise arbitrary. In this paper, through a set of experiments, first we show that a human operator's relaxed arm is strictly passive while voluntary motions of the human operator's arm involve non-passive characteristics. Then, we adjust the passivity assumption of the human operator's arm (by tightening it for an input-strictly-passive arm and relaxing it for a non-passive arm) in order to enable a more precise stability analysis of the teleoperation system. Inspired by Llewellyn's absolute stability criterion, a powerful stability analysis approach is developed to investigate the stability of a two-port network when it is coupled to an input-strictly-passive or a non-passive termination. Although this new stability criterion is applicable to any two-port network system, we apply it to a position-error-based bilateral teleoperation system as a case study.

This paper presents a criterion for absolute stability of a general class of three-port networks. Trilateral haptic systems, which have recently found many interesting applications, can be modeled as three-port networks. Traditionally, existing criteria (Llewellyn‘s criterion) have facilitated the stability analysis of bilateral haptic systems modeled as two-port networks. If the same criteria were to be used for stability analysis of a three-port network, its third port would need to be assumed known for it to reduce to a two-port network. However, this is restrictive because, according to the definition of absolute stability, all three terminations of the three-port network must be allowed to be arbitrary (while passive). In this paper, extending Llewellyn's criterion, we present closed-form necessary and sufficient conditions for absolute stability of a general class of three-port networks — the three terminations need to be passive but are otherwise arbitrary. To this end, we first find a symmetrization condition under which a general asymmetric impedance (or admittance) matrix Z3×3has an equivalent symmetric counterpart Zeq; this Zeq models a reciprocal three-port network with the same stability characterization as the general nonreciprocal three-port network modeled by Z. Then, based on the equivalence of passivity and absolute stability for the equivalent reciprocal network, an absolute stability condition for the original nonreciprocal network is derived. To show how the resulting absolute stability criterion can be utilized at the system design stage, we have applied it to the problem of designing controllers for triple-user collaborative haptic virtual environment systems. The validity of the resulting absolute stability conditions have been verified via simulations.

Trilateral haptic systems can be modeled as three-port networks. Analysis of coupled stability of a three-port network can be accomplished in either the passivity or the absolute stability frameworks assuming all three ports are connected to passive but otherwise unknown terminations. This paper first reviews our recent results in terms of passivity and absolute stability criteria for general three-port networks - both criteria are founded on the properties of a positive-real Hermitian matrix. Next, we show that the absolute stability criterion is less conservative than the passivity criterion and that the two criteria become the same when the trilateral system is represented by a reciprocal immitance matrix. Then, to show how the two criteria may be utilized at the system design stage, we apply them to the problem of designing controllers for a dual-user haptic teleoperation system. Using the two criteria, controllers are then designed and compared in terms of conservatism and performance in simulations.

This paper studies the stability of a delayed 4-channel bilateral teleoperation system based on the passivity framework. Assuming that the operator and the environment are passive systems, the stability of the teleoperation system is reduced to ensuring the passivity of a master control unit (MCU), a slave control unit (SCU), and the time-delayed communication channel. Each of these three blocks is modeled as a 2 × 2 transfer function matrix and passified using our proposed approach in a multi-loop feedback (MLF) structure. We report conditions on the controllers of the 4-channel architecture that are sufficient for passivity of MCU and SCU. Simulation results confirm the validity of these conditions for the stability of the teleoperation system.

Surgery on a freely beating-heart is extremely difficult as the surgeon must perform the procedure while following the heart's fast motion. However, by controlling a teleoperated robot to continuously follow the heart's motion, the surgeon can operate on a seemingly stationary heart. The heart's motion is calculated from ultrasound images and thus involves a non-negligible delay estimated to be 100 ms that, if not compensated for, can cause the robot end-effector (i.e., the surgical tool) to collide with and puncture the heart. This research proposes the use of a Smith predictor to compensate for this time delay. The results suggest that this improves heart motion tracking as the mean absolute error, the difference between the surgeon's motion and the distance between the heart and surgical tool, and the mean integrated square error decreased.

In this paper, a new adaptive control design method for nonlinear telerobotic systems with varying asymmetric time delays is presented. Using the proposed controller, it is possible to synchronize the state behavior of the local and the remote robots. While prior art on adaptive teleoperation has addressed stability in such systems only for constant delays, we guarantee asymptotic stability in the presence of delays that may be time-varying and/or unequal in the forward and backward directions. Using the proposed controller, asymptotic stability of the bilateral teleoperation system subject to any bounded varying delay with a bounded rate of variation can be guaranteed. The proposed controller also has the ability to cope with parameter variations in the dynamics of the local and the remote robots. To study the transparency of the closed-loop teleoperation system, we prove that the position and velocity errors between the local and the remote manipulators converge to zero asymptotically. To show the efficiency of the proposed controller, simulation results on a pair of two-degree-of-freedom manipulators with varying time delays in the communication channel are presented.

For a pneumatic teleoperation system, sliding-mode control laws ensuring both transparency and low switching (open/close) activity of the valves are developed. Each pneumatic actuator has four on/off valves, thus sixteen possible combinations (“operating modes”) for the valves exist but only seven of them are both functional and unique. While previous works have focused on three-mode sliding-based position control of one pneumatic actuator, this paper develops seven-mode sliding-based bilateral control of a teleoperation system comprising a pair of pneumatic actuators. The proposed bilateral sliding control scheme is experimentally validated on a pair of actuators arranged in a force-position teleoperation architecture. The results demonstrate that leveraging the additional modes of operation leads to more efficient and smooth control of the system.

Stability of a haptic teleoperation system is influenced by the typically uncertain, time-varying and/or unknown dynamics of the operator and the environment. For a stability analysis that is independent of the operator and the environment dynamics, Llewellyn's absolute stability criterion proposes certain conditions on the two-port network representing the teleoperator (comprising the master, the controller and communication channel, and the slave) assuming that the terminations (i.e., the operator and the environment) are passive. These are less-than-accurate assumptions. It is desirable to extend Llewellyn's result to the cases where the operator or the environment is non-passive. This paper revisits Llewellyn's criterion and relaxes the assumption of passivity for one of the terminations. The possibly non-passive termination is realistically assumed to have a complex impedance with an upper or lower bound on its amplitude or real part, respectively. Although the proposed stability criteria are useful for any application of two-port network systems, we specifically apply them on bilateral teleoperation systems and find the stability conditions when the operator or the environment is not passive; this is a result that Llewellyn's absolute stability criterion cannot afford.

Instability and poor performance are two wellknown problems encountered in bilateral teleoperation over a communication channel with variable time delays, where force feedback from the slave side is provided to the master side. When unknown disturbances or external forces act on the master and/or the slave manipulators, the teleoperation system will be even more prone to stability and performance degradation. By adopting a Lyapunov approach, we present a novel nonlinear disturbance observer based control scheme for teleoperation systems that are subject to variable time delays and disturbances. Lumping the effects of dynamic uncertainties, unknown forces/torques exerted by the human operator and the remote environment, and external disturbances into a single disturbance term enables us to use a disturbance observer and suppress these disturbances in order to alleviate their adverse effects on the teleoperation system stability and performance. The proposed disturbance observer based control laws guarantee asymptotic disturbance tracking, asymptotic position tracking, and stability of teleoperation system in both constrained and free motions. Experimental results are presented to verify the effectiveness of the proposed approach.

In a bilateral teleoperation system, conditions involving open-loop model parameters and controller parameters for ensuring teleoperator passivity are useful as control design guidelines to attain maximum teleoperation transparency (due to passivity/transparency tradeoffs). By teleoperator, we mean the teleoperation system excluding the human operator and the remote environment. The rationale behind considering teleoperator passivity instead of teleoperation system stability is that, unlike the former, the latter is influenced by the dynamics of the human operator and the remote environment, which are typically uncertain, time-varying, and/or nonlinear. In this paper, a condition for the passivity of a teleoperator is found when the teleoperation controllers are implemented in the discrete-time domain. Such as new passivity analysis is necessary because discretization causes energy leaks and does not necessarily preserve passivity. We show that the passivity criterion for the sampled-data teleoperator imposes a lower bound on the robot damping and upper bounds on the control gains and the sampling time. The criterion has been verified through computer simulations as well as experimental tests involving a bilateral teleoperation system consisting of a pair of Phantom Omni robots.

This paper presents a novel bilateral control scheme for pneumatic teleoperation systems that are actuated by low cost solenoid valves. A sliding mode control is incorporated into a two-channel, bilateral teleoperation architecture involving position-position, force-force, or force-position schemes. An analysis of stability and transparency of the closed-loop teleoperation system is carried out. The proposed control design is verified on a single-degree-of-freedom pneumatic teleoperation system with four on/off solenoid valves. Moreover, simulation results demonstrate high accuracies in terms of position and force tracking in the teleoperation system.

Inverse dynamics controllers deal with nonlinear terms in the robot dynamics in a way that, in the ideal case, the closed-loop system becomes linear and decoupled. Consequently, the performance of the closed-loop systems will be easy to study. Due to such an advantage, inverse dynamics-based adaptive control has been applied to motion control of an uncertain robot in free motion in the literature. However, so far there has been no attempt at simultaneous motion and force control in a master-slave haptic teleoperation system using an adaptive inverse dynamics approach. In this paper, for multi-degree-of-freedom teleoperation systems with nonlinear and uncertain dynamics, adaptive inverse dynamics controllers are incorporated into Lawrence's 4-channel bilateral teleoperation control framework. The resulting high-fidelity control system does not need exact knowledge of the dynamics of the master or the slave. A Lyapunov function is presented to analyze the transparency of the teleoperation system. A simulation study is included to demonstrate the effectiveness of the proposed control method.

Existing work concerning adaptive control of uncertain teleoperation systems can only deal with linearly parameterized (LP) dynamic uncertainties. Typical teleoperation system dynamics, however, posses terms with nonlinearly parameterized (NLP) structures. Stribeck friction is an example of NLP terms in robot dynamics. If not compensated for in the control scheme, uncertainties in the NLP dynamic terms may lead to significant tracking errors. In this paper, for a bilateral teleoperation system, adaptive controllers are designed for the master and slave robots with both LP and NLP dynamic uncertainties. Next, these controllers are incorporated into the 4-channel bilateral teleoperation framework. Then, transparency of the overall teleoperation is studied via a Lyapunov function analysis. A simulation study demonstrates the effectiveness of the proposed adaptive scheme.

This paper proposes a sliding mode law for precise position control with minimal switching activity for a robotic system that uses on/off (solenoid) pneumatic actuators, For a two-chamber pneumatic actuator with four binary solenoid valves, there is a total of sixteen possible input combinations defined directly from the state of the four on/off solenoid valves present in the system. However, only seven of these discrete operating modes are considered both functional and unique. Accordingly, we use a seven-mode sliding controller that minimizes the position error using modes that have both the necessary and sufficient amounts of drive energy and, thus, involve reduced switching activity. An analysis of the closed-loop system stability is carried out. The performance of the proposed control design is experimentally verified on a single pneumatic actuator comprising of two chambers driven by four on/off solenoid valves.

This paper presents an enhanced sliding mode control for pneumatic master-slave teleoperation systems that are actuated by low-cost solenoid valves. A five-mode sliding control is incorporated into position-position, force-force, and force-position teleoperation architectures. While on/off valve pneumatic actuators have previously been modeled as having three discrete operating modes, an extension to five discrete control levels as proposed here helps to improve the actuator dynamic performance and reduce the switching activities of the valves. Stability and transparency analyses of the closed-loop teleoperation system are carried out. The proposed control design is experimentally tested on a single-degree-of-freedom pneumatic teleoperation system. Experimental results demonstrate high accuracies in terms of position and force tracking in the teleoperation system.

This paper presents a new predictive hybrid control law for a pneumatic teleoperation system using solenoid valves. Based on a predictive model of the mass flow rate of the valves, this method is used within a four-channel (4CH) bilateral control architecture for haptic teleoperation. An analysis of the controller parameters is carried out in order to achieve acceptable performances. The results show that a good accuracy in position and force tracking of the teleoperation system is obtained.

Research so far on adaptive bilateral control of master-slave teleoperation systems considers dynamic uncertainties but stops short of considering kinematic uncertainties. However, when picking up objects of unknown lengths, orientations and gripping points, the overall kinematics of a robot in the teleoperation system becomes uncertain. Therefore, new controllers are required that can guarantee the stability and motion tracking performance of the system in the presence of both dynamic and kinematic uncertainties in the master and the slave robots. In this paper, first the uncertain dynamics of the human operator and the environment are incorporated into the dynamics of the master and the slave, respectively. Then, for a teleoperation system with uncertain dynamics and kinematics, nonlinear adaptive controllers are designed for both the master and the slave. The controllers do not need exact knowledge of the dynamics of the master, the slave, the operator, or the environment, or of the kinematics of the master or the slave. The stability and position tracking convergence of the entire teleoperation system are studied. The validity of the theoretical results is verified by simulations.

There is ample research on the stabilization of haptic teleoperation systems under communication time delay. Little attention, however, has so far been paid to the usefulness of delayed haptic feedback on task performance. While the usefulness of haptic feedback in no-delay teleoperation has been previously established, this paper investigates whether haptic feedback helps to improve task performance in the presence of delay. We consider peg-in-the-hole insertion, which is a dexterous manipulation task requiring high force levels at certain points during the task execution. Through a user study involving unilateral and bilateral teleoperation experiments under different delays, it is observed that in both unilateral and bilateral teleoperation, the task completion time increases as delay increases. It is also seen that haptic feedback helps reduce the amount and rate of energy transfer to the environment and the occurrence of larger robot/environment interaction forces. However, with the users mindful of minimizing contact forces, haptic feedback causes the task to take more time compared to no haptic feedback regardless of the time delay. Thus, for tasks where low completion times are crucial given a tolerance for larger forces, unilateral feedback may be sufficient.

In this paper, based on a passivity framework, admittance-type and hybrid-type delay-compensated communication channel models are introduced, which warrant different bilateral control architectures for wave-based teleoperation under time delay. We utilize wave transforms and signal filtering for passivating the delayed-communication channel and passivity/stability conditions are derived using scattering theory based on an end-to-end model of the teleoperation system rather than the communication channel alone. Contrary to a commonly held view, it is proven that the teleoperation system can remain stable when force measurement data of the master and the slave manipulators interactions with the operator and the remote environment are used. Experimental results on a soft-tissue task for a hybrid-type architecture and for round-trip delays of 60 msec and 600 msec show that using slave-side force measurements considerably enhances teleoperation transparency.

Discretization of a stabilizing continuous-time bilateral teleoperation controller for digital implementation may not necessarily lead to stable teleoperation. This paper addresses the stability of master-slave teleoperation under discrete-time bilateral control. Stability regions are determined in the form of conditions involving the sampling period, control gains including the damping introduced by the controller, and environment stiffness. Due to the tradeoff between stability and transparency in bilateral teleoperation, such stability boundaries are of particular importance when the teleoperation system has good transparency.

In applications such as space and surgical robotics, the use of thin, lightweight manipulators and cable-driven end- effectors results in flexibility of the manipulator. In bilateral teleoperation, however, any flexibility in a link or joint of the robot reduces the transparency of teleoperation. In this paper, we analyze master-slave teleoperation transparency under slave robot joint elasticity and evaluate the added benefits of using extra sensors at the end-effector of the elastic-joint robot. It is shown that velocity (or position) feedback from the output shaft of the elastic joint improves free-space position tracking performance, which in the absence of such feedback is hampered by the joint's anti-resonance. Also, when the interaction forces with an environment are measured by a force sensor and fed back to the user, end-effector velocity feedback improves hard-contact force tracking performance. If the operating trajectories correspond to low frequencies, both free-space position tracking and hard-contact force tracking are satisfactory regardless of end-effector feedback, yet the elasticity in the joint will be transmitted to the user during a hard contact task unless end-effector velocity feedback is used.

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.

In robot-assisted interventions, providing a surgeon with haptic information regarding contacts made between surgical instruments and tissue can improve task performance and reliability. In this paper, a force-reflective user interface is used with a sensorized surgical instrument to form a master-slave test-bed for studying haptic interaction in a soft-tissue endoscopic surgery environment. After modeling and parametric identification of the master and the slave, bilateral controllers are designed and teleoperation experiments involving a single degree of freedom surgical task on soft tissue (palpation) are conducted. The transparency of the teleoperator in terms of transmitting the critical task-related information to the user in the context of soft-tissue surgical applications is investigated

Minimally invasive surgery (MIS) is an alternative to open surgery where special instruments are inserted into the body cavity through tiny incisions in order to perform surgical procedures. In this work, some design issues in a master-slave robotic system for use in MIS are discussed. First, we discuss the design of a user interface that can be used to incorporate haptic interaction in robot-assisted MIS. Then we discuss the design of a laparoscopic end effector that meets MlS requirements and is instrumented for haptic feedback.

Minimally invasive surgery involves inserting special instruments into the body cavity through tiny incisions in order to perform surgical procedures. In this paper, the design of a robotic master-slave system for use in minimally invasive surgery is discussed. This system is capable of providing haptic feedback to the surgeon in all available degrees of freedom. System design as well as master and slave bilateral control and communication issues are discussed.