Image processing has significantly extended the practical value of the eye-in-hand camera, enabling and promoting its applications for quantitative measurement. However, fully vision-based pose estimation methods sometimes encounter difficulties in handling cases with deficient features. In this article, we fuse visual information with the sparse strain data collected from a single-core fiber inscribed with fiber Bragg gratings (FBGs) to facilitate continuum robot pose estimation. An improved extreme learning machine algorithm with selective training data updates is implemented to establish and refine the FBG-empowered (F-emp) pose estimatoronline. The integration of F-emp pose estimation can improve sensing robustness by reducing the number of times that visual tracking is lost given moving visual obstacles and varying lighting. In particular, this integration solves pose estimation failures under full occlusion of the tracked features or complete darkness. Utilizing the fused pose feedback, a hybrid controller incorporating kinematics and data-driven algorithms is proposed to accomplish fast convergence with high accuracy. The online-learning error compensator can improve the target tracking performance with a 52.3%–90.1% error reduction compared with constant-curvature model-based control, without requiring fine model-parameter tuning and prior data acquisition.

Soft robots have great potential in surgical applications due to their compliance and adaptability to their environment. However, their flexibility and nonlinearity bring challenges for precise modeling, sensing, and control, especially in constrained cavities. In this article, a simple, compact two-segment soft robot for flexible laser ablation is proposed. The proximal hydraulic-driven segment can offer omnidirectional bending so as to navigate toward lesions. The distal segment driven by tendons enables precise, fast steering of laser collimator for laser sweeping on lesion targets. The dynamics of such mechanical steering motion can be enhanced with a metal spring backbone integrated along the collimator, thus facilitating the control with certain linearity and responsiveness. A soft robot modeling and control scheme based on Koopman operators is proposed. We also design a disturbance observer so as to incorporate the controller feedback with real-time fiber optic shape sensing. Experimental validation is conducted on simulated orex-vivolaser ablation tasks, thus evaluating our control strategies in laser path following across various contours/patterns. As a result, such a simple compact laser manipulation can perform up to 6 Hz sweeping with precision of path following errors below 1 mm. Such modeling and control scheme could also be used on an endoscopic laser ablation robot with unsymmetric mechanism driven by two tendons.