Invasive flexible neural electrodes are becoming increasingly prevalent in monitoring and modulating brain neural activity, necessitating the precise and minimally invasive implantation of these electrodes to a depth of a few millimeters beneath the cerebral surface. Although Neuralink has pioneered robot-assisted neural electrode implantation guided by microscopy, it currently lacks the ability to detect non-cerebral surface microvessels that are invisible under the white-light microscope, leading to inaccurate implantation planning and a high risk of trauma. To address this limitation, we introduce a vascular-enhanced strategy that fuses intraoperative white-light microscopy and preoperative photoacoustic microscopy and applies the fusion results to our established microsurgical robotic system for brain electrode implantation. Specifically, a multi-modality data preprocessing pipeline is devised to extract representative features, and a 2.5D fusion network that incorporates a depth encoding mechanism is proposed to predict cross-modality correspondence. The enhanced fusion results are utilized for implantation planning and intraoperative guidance during in vivo surgical procedures. Both quantitative and qualitative results are presented to demonstrate the effectiveness of our proposed cross-modality fusion methods. Furthermore, in vivo surgical implementations on mice underscore the potential of the proposed approach for achieving more precise and minimally invasive brain electrode implantation.

In teleoperated surgery, the motion scaling factor directly influences both the operator’s control precision of surgical instruments and operational comfort. Previous studies have revealed that the master manipulator state and operator’s gaze information can reflect the complexity of surgical operations and the operator’s intention to some extent. Although enabling real-time adjustment of scaling factors, they were limited by the narrow range of core parameters and the results were significantly influenced by subjective factors. To tackle these challenges, this paper presents a multi-dimensional adaptive motion scaling strategy based on the Bayesian optimization. The prediction of operator’s intention and attention is achieved by integrating multiple dimensional parameters, including master-slave manipulator states, gaze information, as well as pupillary data, all of which have been experimentally validated. Specifically, there exists a significant temporal synchronization between the Index of Pupillary Activity (IPA) and teleoperation tasks, which aligns with research on the correlation between IPA and attention levels. Furthermore, to evaluate the proposed adaptive scaling strategy, we combine subjective questionnaire surveys with objective metric assessments, effectively reducing the excessive influence of operators’ personal conditions and proficiency levels on optimization results.

Photoacoustic Microscopy (PAM) is a relatively new imaging modality in biomedicine. However, point-by-point raster scanning in PAM suffers from low imaging speed. Sparse sampling has been studied in recent years and with the development of deep learning algorithms, extensive efforts have been devoted to sparse image reconstruction while little attention has been paid to sparse sampling trajectory design required for actual implementation. The use of real-time adaptive robotically controlled sampling with micro-scale accuracy with due consideration of physical constraints can pave the way for using PAM for robot-assisted microsurgery. This work proposes a fast PAM scheme with robot-controlled microtrajectory optimization. The proposed method is adaptive to imaging details of different regions of interest (ROI) and detailed experiments have been conducted on both simulation and in-vivo settings. Results show that our proposed method can achieve faster scanning speed than traditional raster scanning and improved image quality in ROI than the standard spiral trajectory, which demonstrates the effectiveness of our proposed method and its potential to be deployed in other point-by-point scanning systems.