Advances in optical imaging, and probe-based Confocal Laser Endomicroscopy (pCLE) in particular, offer real-time cellular level information for in-vivo tissue characterization. However for large area coverage, the limited field-of-view necessitates the use of a technique known as mosaicking to generate usable information from the incoming image stream. Mosaicking also needs a continuous stream of good quality images, but this is challenging as the probe needs to be maintained within an optimal working range and the contact force controlled to minimize tissue deformation. Robotic manipulation presents a potential solution to these challenges, but the lack of haptic feedback in current surgical robot systems hinders the technology's clinical adoption. This paper proposes a sensorless alternative based on processing the incoming image stream and deriving a quantitative measure representative of the image quality. This measure is then used by a controller, designed using model-free reinforcement learning techniques, to maintain optimal contact autonomously. The developed controller has shown near real-time performance in overcoming typical loss-of-contact and excess-deformation scenarios experienced during endomicroscopy scanning procedures.

Whilst current minimally invasive surgical robots offer many advantages to the surgeon, most of them are still controlled using the traditional master-slave approach, without fully exploiting the complementary strengths of both the human user and the robot. This paper proposes a framework that provides a cooperative control approach to human-robot interaction. Typical teleoperation is enhanced by incorporating haptic guidance-based feedback for surgical tasks, which are demonstrated to and learned by the robot. Safety in the surgical scene is maintained during reproduction of the learned tasks by including the surgeon in the guided execution of the learned task at all times. Continuous Hidden Markov Models are used for task learning, real-time learned task recognition and generating setpoint trajectories for haptic guidance. Two different surgical training tasks were demonstrated and encoded by the system, and the framework was evaluated using the Raven II surgical robot research platform. The results indicate an improvement in user task performance with the haptic guidance in comparison to unguided teleoperation.

In recent years, robotic systems have been playing an increasingly important role in physiotherapy. The aim of these platforms is to aid the recovery process from strokes or muscular damage by assisting patients to perform a number of controlled tasks, thus effectively complementing the role of the physiotherapist. In this paper, we present a novel learning from demonstration framework for cooperative control in robotic-assisted physiotherapy. Unlike other approaches, the aim of the proposed system is to guide the patients to optimally execute a task based on previously learned demonstrations. This allows the generation of patient-specific gestures under the supervision of the expert physiotherapist. The guidance is performed through stiffness control of a compliant manipulator, where the stiffness profile of the generalized trajectory is determined according to the relative importance of each section of the task. In contrast with the traditional learning approach, where the execution of the generalized trajectory by the robot is automated, this cooperative control architecture allows the patients to perform the task at their own pace, while ensuring the movements are executed correctly. Increased performance of the learning framework is accomplished through a novel fast, low-cost multi-demonstration dynamic time warping algorithm used to build the model. Experimental validation of the framework is carried out using an interactive setup designed to provide further guidance through additional visual and sensory feedback based on the task model. The results demonstrate the potential of the proposed framework, showing a significant improvement in the performance of guided tasks compared to unguided ones.

This paper introduces the CYCLOPS, a novel robotic tool for single-access and natural-orifice endoscopic surgery. Based on the concept of tendon-driven parallel robots, this highly original design gives the system some of its unique capabilities. Just to name a few, unparalleled force exertion capabilities of up to 65N, large and adjustable workspace, bimanual instrument triangulation. Due to the simplicity and nature of the design, the system could be adapted to an existing laparoscope or flexible endoscope. This promises a more immediate and accelerated route to clinical translation not only through endearing low-cost and adaptive features, but also by directly addressing several major barriers of existing designs.

Precise and repetitive positional control of surgical robots is important to reduce time and risks of surgical procedures. These factors become particularly important when deploying the surgical system through a flexible path to areas with a tight workspace such as the stomach or oesophagus where high dexterity, flexibility, accuracy and stability are required. This paper presents a flexible access robot combining articulated joints and continuum flexible section for both transluminal and endoluminal surgeries. Kinematic model and control strategy for the flexible robot are described in the paper. The experiment simulating a transoral gastric procedure demonstrates great flexibility and dexterity of the device. The results show that good accuracy and repetitive control of the device are achieved, which demonstrate the potential application of the device for transluminal or endoluminal surgery.

Recent technological advances in surgery have permitted cellular and molecular imaging to be carried out intra-operatively. Although optical biopsy techniques such as probe-based confocal laser endomicroscopy (pCLE) have enabled real-time diagnosis and tissue characterisation in vivo, the flexibility of the probe introduces significant challenges under manual control. Examination of large tissue areas is particularly challenging due to micron-scale resolution of the probe and the need for maintaining consistent probe orientation and force contact with the tissue to avoid cellular deformation or damage. The use of a robotic manipulator to perform surface scanning automatically introduces great benefits in terms of positioning repeatability and accuracy. However, pre-programming of such complex task is not realistic due to patient-specific anatomy and constant changes in tissue morphology during the operation. To overcome this problem, a cooperative, in situ microscopic scanning and simultaneous tissue surface reconstruction technique is proposed. The system provides a hands-on, learning-based framework for optimal trajectory coverage from surgeon-demonstrated motions intraoperatively. The position and force information acquired during the scanning are also used to simultaneously reconstruct the surface morphology and combined with the pCLE images to generate a 3D functional map of the tissue.

We propose a learning-based controller to enable autonomous execution of the eFAST scanning by a lightweight robotic manipulator according to expert demonstrations. The benefits of this approach are two-fold. Firstly, the automatically acquired USS images can be sent to the expert radiologist from a remote location without the need for complex robotic tele-operation. Secondly, the application of learning by demonstration alleviates the complexity of robotic programming and allows extracting operator-specific knowledge in situ in a natural and intuitive way. The provision of incorporating force information can further improve the versatility of the system, allowing easy adaptation to different dynamic environments.

This paper presents a real-time control framework for a snake robot with hyper-kinematic redundancy under dynamic active constraints for minimally invasive surgery. A proximity query (PQ) formulation is proposed to compute the deviation of the robot motion from predefined anatomical constraints. The proposed method is generic and can be applied to any snake robot represented as a set of control vertices. The proposed PQ formulation is implemented on a graphic processing unit, allowing for fast updates over 1 kHz. We also demonstrate that the robot joint space can be characterized into lower dimensional space for smooth articulation. A novel motion parameterization scheme in polar coordinates is proposed to describe the transition of motion, thus allowing for direct manual control of the robot using standard interface devices with limited degrees of freedom. Under the proposed framework, the correct alignment between the visual and motor axes is ensured, and haptic guidance is provided to prevent excessive force applied to the tissue by the robot body. A resistance force is further incorporated to enhance smooth pursuit movement matched to the dynamic response and actuation limit of the robot. To demonstrate the practical value of the proposed platform with enhanced ergonomic control, detailed quantitative performance evaluation was conducted on a group of subjects performing simulated intraluminal and intracavity endoscopic tasks.

The increasing number of degrees-of-freedom involved in new generations of surgical robotics and the need for incorporating active constraints and haptic feedback, require more intuitive and effective ways of robot control. This paper presents a novel manipulator that allows for ergonomic bimanual joint-level control of an anthropomorphic surgical robot. Through the combined use of bidirectional compressed airflow, the manipulator can operate on nearly zero friction and simulate a range of frictional forces. As a generic platform, the system can withstand large payloads and is able to accommodate a wide range of existing haptic manipulators. The performance of the proposed platform is evaluated with detailed experimental tests and proven to provide negligible friction even at high loads. Its dynamic friction is controllable and positional locking can be flexibly applied. Detailed experimental results demonstrate the practical value of the system.

This paper introduces a modular, mechatronic joint design for a flexible access minimally invasive surgical platform. The design features a hybrid tendon-micromotor actuation scheme coupled via an internal “ring gear”. This configuration transmits the rotary motion of the embedded micromotor to joint rotation in an efficient manner while also providing a hollow lumen through the centre of the joint unit. Detailed analyses of the joint design including tendon path length, positional accuracy and force/torque transmission characteristics are presented. The design has been incorporated into a dexterous, seven degree-of-freedom flexible access platform and its clinical efficacy for performing a Natural Orifice Translumenal Endoscopic Surgery (NOTES) diagnostic peritoneoscopy in an in-vivo environment is presented.

This paper presents a human-robot interface with perceptual docking to allow for the control of multiple microbots. The aim is to demonstrate that real-time eye tracking can be used for empowering robots with human vision by using knowledge acquired in situ. Several micro-robots can be directly controlled through a combination of manual and eye control. The novel control environment is demonstrated on a virtual biopsy of gastric lesion through an endoluminal approach. Twenty-one subjects were recruited to test the control environment. Statistical analysis was conducted on the completion time of the task using the keyboard control and the proposed eye tracking framework. System integration with the concept of perceptual docking framework demonstrated statistically significant improvement of task execution.