The self-supervised monocular depth estimation framework is well-suited for medical images that lack ground-truth depth, such as those from digestive endoscopes, facilitating navigation and 3D reconstruction in the gastrointestinal tract. However, this framework faces several limitations, including poor performance in low-texture environments, limited generalisation to real-world datasets, and unclear applicability in downstream tasks like visual servoing. To tackle these challenges, we propose MonoLoT, a self-supervised monocular depth estimation framework featuring two key innovations: point matching loss and batch image shuffle. Extensive ablation studies on two publicly available datasets, namely C3VD and SimCol, have shown that methods enabled by MonoLoT achieve substantial improvements, with accuracies of 0.944 on C3VD and 0.959 on SimCol, surpassing both depth-supervised and self-supervised baselines on C3VD. Qualitative evaluations on real-world endoscopic data underscore the generalisation capabilities of our methods, outperforming both depth-supervised and self-supervised baselines. To demonstrate the feasibility of using monocular depth estimation for visual servoing, we have successfully integrated our method into a proof-of-concept robotic platform, enabling real-time automatic intervention and control in digestive endoscopy. In summary, our method represents a significant advancement in monocular depth estimation for digestive endoscopy, overcoming key challenges and opening promising avenues for medical applications.

Due to the limited field of view (FOV), the probe-based confocal laser endomicroscopy (pCLE) imaging system remains challenging to be widely used in clinic. Existing video mosaicking approaches are usually troubled by poor real-time capability and sensitivity to tissue deformations and intensity fluctuations. In this paper, a novel pCLE mosaicking algorithm that simultaneously implements rigid probe motion tracking and inter-frame tissue deformation correction is proposed. The sum of conditional variance (SCV) metric with low calculational complexity and intensity variation invariance is embedded into the reconstruction of pCLE mosaics. We also deduce the derivatives of the SCV metric with respect to transformation variables to enhance the robustness of numerical solutions. Moreover, a parallel acceleration mechanism that maximally exploits the resources of graphics processing unit (GPU) is designed to boost mosaicking efficiency. Experiments on lens tissue paper and swine tissue highlight the excellent mosaicking accuracy of the proposed method in this field. It is also verified that the mosaic-king rate of the proposed method exceeds 14 frames per second (fps), which can keep up with the acquisition rate of most pCLE imaging systems (8 fps~12 fps). Benefiting from finer mosaics and faster speed, the proposed algorithm has high potential to achieve large and accurate mosaicking for intraoperative endomicroscopy.

Driver fatigue evaluation is of great importance for traffic safety and many intricate factors would exacerbate the difficulty. In this paper, based on the spatial-temporal structure of multichannel electroencephalogram (EEG) signals, we develop a novel EEG-based spatial-temporal convolutional neural network (ESTCNN) to detect driver fatigue. First, we introduce the core block to extract temporal dependencies from EEG signals. Then, we employ dense layers to fuse spatial features and realize classification. The developed network could automatically learn valid features from EEG signals, which outperforms the classical two-step machine learning algorithms. Importantly, we carry out fatigue driving experiments to collect EEG signals from eight subjects being alert and fatigue states. Using 2800 samples under within-subject splitting, we compare the effectiveness of ESTCNN with eight competitive methods. The results indicate that ESTCNN fulfills a better classification accuracy of 97.37% than these compared methods. Furthermore, the spatial-temporal structure of this framework advantages in computational efficiency and reference time, which allows further implementations in the brain-computer interface online systems.

Confocal endomicroscopy can be used for identification of early mucosal dysplasia in various gastrointestinal conditions, and has particular potential in the monitoring of Barrett's oesophagus and the early stages of oesophageal cancer. However, it can be difficult to systematically scan a significant area of the oesophagus because of the small field-of-view and limited flexibility of the probe. Tissue deformation and inconsistent probe-tissue contact also make it difficult to form large mosaics. A mechanical scanning device is therefore desirable for controlled, large area surface scanning and mosaicing of the oesophagus. This paper proposes a robotic catheter encapsulated in an inflatable balloon, providing stable scanning over the oesophageal surface. It has an outer diameter of 3 mm, making it suitable for deployment through an endoscope working channel, and uses a custom endomicroscopy probe based on a leached flexible fibre bundle and an external confocal laser scanning system. Detailed mechanical performance and image quality evaluations were performed to assess the clinical potential of the device. In ex vivo studies using swine oesophagus, long helical scans were obtained, demonstrating that the device is able to scan the lumen stably and maintain good probe-tissue contact. The experimental results demonstrate the potential of the robotic catheter for systematic high-resolution imaging of the oesophageal mucosa, potentially reducing or even eliminating the need for physical biopsy.

Recent work on probe-based confocal endomicroscopy has demonstrated its potential role for real-time assessment of tumour margins during breast conserving surgery. However, endomicroscope probes tend to have a very small field-of-view, making surveillance of large areas of tissue difficult, and limiting practical clinical deployment. In this paper, a new robotic device for controlled, large area scanning based on a fibre bundle endomicroscope probe is proposed. The prototype uses a 2-DOF mechanism (−90 to +90 degrees bending on one axis, 360 degrees of rotation on a second axis) as well as a passive linear structure to conform to undulating surfaces. Both axes are driven by brushless DC servo motors with computer control, thus facilitating large field-of-view mosaicing. Experimental results have shown good repeatability and low hysteresis of the device, which is able to scan different surface trajectories (e.g. a spiral pattern over a hemi-spherical surface) with consistent tissue contact. Ex vivo human breast tissue results are demonstrated, illustrating a viable scanning approach for breast endomicroscopy.