Fetoscopy is a technically challenging surgery, due to the dynamic environment and low diameter endoscopes often resulting in a limited field of view. In this paper, we report on the design and operation of a robotic multimodal endoscope with optical ultrasound and white light stereo camera. The manufacture and control of the endoscope is presented, along with large area (80 mm ×80 mm) surface visualisations of a placenta phantom using the optical ultrasound sensor. The repeatability of the surface visualisations was found to be 0. 446 ± 0.139 mm and 0. 267 ± 0.017 mm for a raster and spiral scan, respectively.

During laparoscopic surgery, tissues frequently need to be retracted and mobilized for manipulation or visualisation. State-of-the-art robotic platforms for minimally invasive surgery (MIS) typically rely on rigid tools to interact with soft tissues. Such tools offer a very narrow contact surface thus applying relatively large forces that can lead to tissue damage, posing a risk for the success of the procedure and ultimately for the patient. In this paper, we show how the use of Pneumatically Attachable Flexible (PAF) rail, a vacuum-based soft attachment for laparoscopic applications, can reduce such risk by offering a larger contact surface between the tool and the tissue. Ex vivo experiments are presented investigating the short- and long-term effects of different levels of vacuum pressure on the tissues surface. These experiments aim at evaluating the best trade-off between applied pressure, potential damage, task duration and connection stability. A hybrid control system has been developed to perform and investigate the organ repositioning task using the proposed system. The task is only partially automated allowing the surgeon to be part of the control loop. A gradient-based planning algorithm is integrated with learning from teleoperation algorithm which allows the robot to improve the learned trajectory. The use of Similar Smooth Path Repositioning (SSPR) algorithm is proposed to improve a demonstrated trajectory based on a known cost function. The results obtained show that a smoother trajectory allows to decrease the minimum level of pressure needed to guarantee active suction during PAF positioning and placement.

Robot-assisted laparoscopic prostatectomy (RALP) is a treatment for prostate cancer that involves complete or nerve sparing removal prostate tissue that contains cancer. After removal the bladder neck is successively sutured directly with the urethra. The procedure is called urethrovesical anastomosis and is one of the most dexterity demanding tasks during RALP. Two suturing instruments and a pair of needles are used in combination to perform a running stitch during urethrovesical anastomosis. While robotic instruments provide enhanced dexterity to perform the anastomosis, it is still highly challenging and difficult to learn. In this paper, we presents a vision-guided needle grasping method for automatically grasping the needle that has been inserted into the patient prior to anastomosis. We aim to automatically grasp the suturing needle in a position that avoids hand-offs and immediately enables the start of suturing. The full grasping process can be broken down into: a needle detection algorithm; an approach phase where the surgical tool moves closer to the needle based on visual feedback; and a grasping phase through path planning based on observed surgical practice. Our experimental results show examples of successful autonomous grasping that has the potential to simplify and decrease the operational time in RALP by assisting a small component of urethrovesical anastomosis.

In this paper, we present a Mobile Agile Printer (MAP) construction robot; a highly agile, 4-legged, omnidirectional robot capable of 3D printing large structures. To overcome dynamic challenges when operating within an outdoors construction site, MAP incorporates a high-DoF 3D printing system connected to a mobile platform with novel features designed to enable disturbance rejection and live adaption to the robot's pose. In doing so, we demonstrate the benefits of designing construction robots with a focus on agility, a compact working volume and ability to operate within a potentially unlimited workspace. Performance tests were conducted showing smooth omni-directional motion as a key requirement for maintaining low 3D printing trajectory deviations over a large volume. In doing so, we show that MAP has the ability to construct in new ways more sensitive to its environment, context and concurrent on-site operations.

Recovering 3D scene geometry from underwater images involves the Refractive Structure-from-Motion (RSfM) problem, where the image distortions caused by light refraction at the interface between different propagation media invalidates the single view point assumption. Direct use of the pinhole camera model in RSfM leads to inaccurate camera pose estimation and consequently drift. RSfM methods have been thoroughly studied for the case of a thick glass interface that assumes two refractive interfaces between the camera and the viewed scene. On the other hand, when the camera lens is in direct contact with the water, there is only one refractive interface. By explicitly considering a refractive interface, we develop a succinct derivation of the refractive fundamental matrix in the form of the generalised epipolar constraint for an axial camera. We use the refractive fundamental matrix to refine initial pose estimates obtained by assuming the pinhole model. This strategy allows us to robustly estimate underwater camera poses, where other methods suffer from poor noise-sensitivity. We also formulate a new four view constraint enforcing camera pose consistency along a video which leads us to a novel RSfM framework. For validation we use synthetic data to show the numerical properties of our method and we provide results on real data to demonstrate performance within laboratory settings and for applications in endoscopy.

Transanal Endoscopic Microsurgery (TEM) is a minimally invasive oncological resection procedure that utilises a natural orifice approach rather than the traditional abdominal or open approach. However, TEM has a significant recurrence rate due to incomplete excisions, which can possibly be attributed to the absence of intraoperative image guidance. The use of real-time histological data could allow the surgeons to assess the surgical margins intraoperatively and adjust the procedure accordingly. This paper presents the integration of endomicroscopy and ultrasound imaging through a robotically actuated instrument. Endomicroscopy can provide high resolution images at a surface level while ultrasound provides depth resolved information at a macroscopic level. Endomicroscopy scanning is achieved with a novel scanning approach featuring a passive force adaptive mechanism. The instrument is manipulated across the surgical workspace through an articulated flexible shaft. This results in the ability to perform large area mosaics coupled with ultrasound scanning. In addition, the use of endoscopic tracking is demonstrated, allowing three-dimensional reconstruction of the ultrasound data displayed onto the endoscopic view. An ex vivo study on porcine colon tissue has been performed, demonstrating the clinical applicability of the instrument.