In this paper, a novel semi-autonomous control framework is presented for enabling probe-based confocal laser endomicroscopy (pCLE) scan of the retinal tissue. With pCLE, retinal layers such as nerve fiber layer (NFL) and retinal ganglion cell (RGC) can be scanned and characterized in real-time for an improved diagnosis and surgical outcome prediction. However, the limited field of view of the pCLE system and the micron-scale optimal focus distance of the probe, which are in the order of physiological hand tremor, act as barriers to successful manual scan of retinal tissue. Therefore, a novel sensorless framework is proposed for real-time semi-autonomous endomicroscopy scanning during retinal surgery. The framework consists of the Steady-Hand Eye Robot (SHER) integrated with a pCLE system, where the motion of the probe is controlled semi-autonomously. Through a hybrid motion control strategy, the system autonomously controls the confocal probe to optimize the sharpness and quality of the pCLE images, while providing the surgeon with the ability to scan the tissue in a tremor-free manner. Effectiveness of the proposed architecture is validated through experimental evaluations as well as a user study involving 9 participants. It is shown through statistical analyses that the proposed framework can reduce the work load experienced by the users in a statistically-significant manner, while also enhancing their performance in retaining pCLE images with optimized quality.

This paper presents a novel multimodal training platform integrated with hand-over-hand (HOH) haptic guidance for dual-console surgical robotic systems such as the da Vinci Si system. The expert-in-the-loop (EIL) framework incorporates a fuzzy interface system in order to provide a trainee with adaptive authority over the procedure as well as hand-over-hand haptic guidance adjusted in real time based on the proficiency level of the trainee. The EIL expertise-oriented framework enables performance of a surgical procedure by an expert surgeon on a patient, while simultaneously providing a trainee at any stage of the motor-skills development with multimodal training without jeopardizing patient safety. Closed-loop stability of the system is investigated using the circle criterion and it is shown that the proposed architecture is unconditionally stable. Experimental evaluations are presented in support of the proposed platform through the implementation of a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA, USA) and the dV-Trainer master console (Mimic Technology, Inc., Seattle, WA, USA). To the best of our knowledge, the implemented setup is the first research platform for dual-console studies involving the classic da Vinci surgical system.

This paper presents a Therapist-in-the-Loop (TIL) framework for robotics-assisted mirror rehabilitation therapy integrated with adaptive Assist-as-Needed (ANN) training, to be adjusted based on the impairment and disability level of the patient's affected limb. Closed-loop system stability has been investigated using a combination of the Circle Criterion and the Small-Gain Theorem to account both for time-delay and the time-varying adaptive ANN training. Experiments to investigate the performance of the proposed framework are reported.

In this paper, a new passivity-based technique is proposed to analyze and guarantee the stability of haptics-enabled telerobotic rehabilitation systems where there is a possibility of having more sources of non-passivity than communication delays. In practice, the difficulty of therapeutic exercises should be tuned taking into account the stage of physical disability. However, tuning the difficulty and intensity should not violate the stability of patient-robot interaction. This usually puts conservative prefixed limits on the allowable exercise intensity. In this paper, patient-robot interaction safety is studied in the context of Strong Passivity Theory (SPT). Our goal is to ultimately relax the limitation on the allowable robotic therapies while preserving system stability. The proposed stabilizing scheme does not try to make the entire non-passive component passive. This allows the therapist to have freedom in injecting energy into the system for assistive therapies while ensuring safe patient-robot interaction. In this paper, the case of telerobotic rehabilitation is considered. Experimental implementation and evaluation are presented to support the proposed theory.

This paper proposes an expertise-oriented training platform for robotics-assisted minimally invasive surgery. The framework builds on previous work of the authors and makes use of dual-user teleoperation scenario, allowing the presence of an expert in the training loop. A Fuzzy-Logic (FL) methodology is proposed, which specifies the level/mode of the training required for the trainee according to his/her level of proficiency over the task. A major advantage of the proposed FL approach is that, having the expert in the loop, it can specify the trainee's proficiency level relative to that of the expert in real-time. Moreover, based on the relative skills assessment, the proposed FL approach decides if or to what extent the trainee should receive a haptic guidance force based on Virtual Fixtures or the environment force from the interaction between the surgical instrument and tissue at the slave side. In addition to the level/mode of the haptics-enabled training required for the trainee, the proposed FL framework specifies the authority level of the trainees over the operation in real-time, according to their proficiency levels over the task. Stability of the overall closed-loop teleoperated system is also investigated using the small-gain theorem, resulting in a sufficient condition to guarantee stability in the presence of constant communication delays. Finally, experimental results are given to evaluate the design and feasibility of the proposed framework.

In this paper, a safety concern arising from pathological tremors in patients interacting with haptics-enabled rehabilitation robots is analyzed and the issue of tremor amplification for assistive/coordinative robotic rehabilitation is investigated. In order to deal with this issue, a control architecture is proposed to dissipate the extra energy of the system and guarantee its stability and safety of the patient. For this purpose, (a) first, a multilayer adaptive filter is proposed to estimate high-frequency components of hand motions (corresponding to involuntary movements); (b) then a resistive force field is generated and applied by the robot to attenuate the tremor; and (c) simultaneously the residual low-frequency voluntary actions are amplified/coordinated to deliver appropriate therapy. Stability analysis and a stabilization scheme are developed to guarantee safe interaction regardless of variations in the patient's dynamics and tremor kinematics. The ultimate goal is to make it possible for patients with pathological tremors to take advantage of non-passive robotic assistive/coordinative therapy. This would not be possible using conventional systems due to the possibility of tremor amplification. Experimental results are presented.

In this paper, a novel robotics-assisted rehabilitation framework is proposed for bilateral mirror-image therapy. For this purpose, a customized dual-user teleoperation architecture is designed incorporating Guidance Virtual Fixtures (GVFs) to deliver the appropriate therapeutic movements to the patient's impaired limb by providing an assist-as-needed treatment strategy. In addition, the therapist is provided with informative haptic feedback that is generated based on the patient's movements, allowing the therapist to decide in real-time on the level and format of the therapy required for the patient. Stability of the closed-loop system is also investigated using the small gain theorem, in the presence of communication time delays, facilitating the case of remote tele-rehabilitation. Experimental results are given to validate the performance of the proposed platform.

In this paper, a robotic solution is proposed to deal with the challenges caused by lung motion during needle insertion. To accomplish this goal, a macro-micro robotic tool is designed to compensate for tissue motion using the macro part, while performing the needle insertion independently with the micro part. The main application of this work is for robotics-assisted lung tumor biopsy, where the combined motions of respiration and heartbeat may compromise success. An impedance-based controller keeps the macro reference coordinate in contact with the moving soft tissue using measurements from small pressure sensors mounted at the tip of the macro shaft. The micro part, mounted at the end of the macro robot, manipulates the needle in the harmonized reference coordinate system. Preoperative identification of ex vivo soft tissue is performed to estimate the dynamic behavior of the tissue. The controller is then synthesized based on the identified model. The effects of identification error and high frequency uncertainty are addressed in the control design. A prototype was built to evaluate the proposed approach using: 1) two Mitsubishi PA-10 robots, one for manipulating the macro part and the other for mimicking tissue motion, 2) one motorized linear stage to handle the micro part, and 3) a Phantom Omni haptic device for remote manipulation. Experimental results demonstrate the performance of the motion compensation system.

This paper proposes a teleoperated dual-user system incorporating Virtual Fixtures (VFs) that allows concurrent performance of a robotic surgical task by an expert and a trainee. In order to guide the trainee through the procedure, an adaptive VF is created in the trainee's workspace according to the motion generated by the expert who is performing the surgery at the same time. The VF gets adaptively adjusted based on the level of expertise the trainee shows during the surgery. In addition, the trainee's level of expertise is used to adaptively adjust the dual-user dominance factor in an online fashion, which gives the trainee some authority over the task based on his/her skill level. To quantify the trainee's expertise level, a performance measure is proposed, based on the force generated by the VF. Three performance measures from the literature are also used. To satisfy the desired objectives of the proposed system, an impedance-based control methodology is adopted. Stability of the closed-loop system is investigated using the small-gain theorem. A sufficient stability condition is derived that guarantees stability in the presence of time-varying communication delay. Experimental results are given to validate the performance of the system.

This paper focuses on control challenges caused by the slave-arm flexibility in master-slave telerobotic systems. Apart from time delay, the nonlinear non-minimum phase deflection of flexible-link slave manipulators creates extra challenges for control of telerobotic systems. These challenges have forced most of the prior research to use basic simplifications. In this paper, we try to remove most of the simplifications using a more realistic model for slave deflections. The degrading effects of flexibility on stability and performance of conventional telerobotics architectures are analyzed. Then, the standard Extended Lawrence Four-Channel (ELFC) control architecture is modified to obtain the stability condition and to enhance performance. Finally, the input-to-output stability (IOS) Small-Gain Theorem is used to generalize the stability criteria for varying delay and to remove the restrictive assumption of deflection-linearity. The proposed architecture consists of a local Partial Feedback Linearization scheme embedded into a modified ELFC architecture. This study is motivated by the use of light-weight cable driven tools in telerobotic surgery.

In this paper, a novel decentralized multilateral structure is proposed for the dual user systems in the presence of communication delay. The proposed structure utilizes adaptive impedance control approach in order to overcome the destructive effect of the time-delay on system desired-objectives, which is a disregarded issue in the previous studies on dual user system. The proposed control strategy, which utilizes three desired impedance surfaces defined in the paper, satisfactorily brings the system hybrid matrix close to the ideal one that guarantees the system stability and transparency. The controller is designed in a way that eliminates the necessity of the delay estimation as one of its outstanding characteristics; consequently, the unknown communication time-delay can be handled via this structure while previous studies have disregarded the issue of time delay in dual user system. Furthermore, the adaptive structure of the controller promises to overcome the uncertainties on robot's dynamics. In addition, the efficiency of the controller in guaranteeing the system stability in the presence of unknown communication delay is investigated through passivity theory and the presented analysis illustrates complete independency of the closed-loop system stability on time delay value applying the proposed controller. Experimental results performed on a delayed dual user system demonstrate validity of the proposed scheme.