In Fourier-domain optical coherence tomography (FD-OCT) imaging, saturation of the data acquisition chain leads to loss of information by clipping the spectral interferograms. When reconstructed, affected depth profiles are degraded by bright axial lines or periodical patterns which mask the underlying information and thereby affect clinical utility. In this work, we propose an approach to faithfully recover the distorted information from saturated FD-OCT scans based on physical insights. For this purpose, genesis and appearance of saturation artifacts are analyzed, revealing a wavelength-dependency within individual interferograms. We propose to use this knowledge in a dual manner: Firstly, we present a simulation model that can dynamically generate realistically saturated interferograms from clean counterparts. This physics-based model allows training neural networks for artifact removal solely on synthesized image pairs. Secondly, to make the wavelength-dependency of artifacts directly accessible, we propose a multi-input, single-output (MISO) network framework. In addition to full B-scans, MISO receives images reconstructed from various spectral sub-windows. Our experiments confirm successful generalization of networks trained on simulated training data to real-world artifact removal in ophthalmic anterior segment imaging. This includes reflexes of various origin in diagnostic or intra-surgical imaging, such as different tissue structures or surgical instruments. Furthermore, a comparison of the proposed multi-input network to single-input baselines reveals consistent performance and reliability gains on both swept-source and spectral domain datasets.

Four-dimensional microscope-integrated optical coherence tomography enables volumetric imaging of tissue structures and tool-tissue interactions in ophthalmic surgery at interactive update rates. This enables surgeons to undertake particular surgical steps under four-dimensional optical coherence tomography (4D OCT) guidance. However, current 4D OCT systems are limited by their field of view and signal quality. Both are attributable to the emphasis on high volume acquisition rates, which is critical for smooth visual perception by the surgeon. Existing 3D volume mosaicing methods are developed in the context of diagnostic imaging and do not take dynamic surgical interactions and real-time processing into account. In this paper, we propose a novel volume mosaicing and visualization methodology that not only aims at leveraging the temporal information to overcome some of the current limitations and imaging artifacts of 4D OCT, but also is aware of the surgical context and dynamic instrument motion implicitly during registration and explicitly for visualization. We propose a rapid 4-degrees of freedom volume registration, integrating an innovative approach for volume mosaicing that takes temporal recency and semantic information into account for enhanced surgical visualization. Our experiments on 4D OCT datasets demonstrate high registration accuracy and illustrate the benefits for visualization by reducing imaging artifacts and dynamically expanding the surgical view.

The injection of therapeutic agents into the sub-retinal space might allow improved treatment of age-related macular degeneration. Various robotic systems have been developed to achieve the required precision and, in combination with intraoperative Optical Coherence Tomography (iOCT) imaging, methods for autonomous robotic guidance have been proposed. In such systems, the robot’s cognition is often governed by machine learning algorithms, such as convolutional neural networks (CNNs), which provide semantic scene information from iOCT images. Although the robot performs a surgical task autonomously, human supervision is critical to monitor the robot’s execution and, if necessary, stop the robot or take control to avoid trauma to the patient. In this paper, we propose a novel visualization concept for improved human supervision of autonomous robotic subretinal injection that integrates uncertainty information of the data provided to the robot. We design a focus and context visualization that renders an automatically identified instrument-aligned B-scan in the context of the 3D OCT volume. Our visualization is enriched by augmenting the uncertainty information on the instrument-aligned B-scan. To dynamically model task-specific uncertainty, we introduce a weighting scheme to assign an importance factor to each pair of classes, controlling the impact of their confusion on the overall uncertainty. We demonstrate our visualization concept on iOCT volumes acquired at different stages during subretinal injection on ex-vivo porcine eyes. We show that our processing pipeline achieves sufficient update rates for surgical display and discuss the impact of our visualization concept on the acceptance of robotic task autonomy for subretinal injection procedures.

Subretinal injection is a delicate and complex microsurgery, which requires surgeons to inject the therapeutic substance in a pre-operatively defined and intra-operatively updated subretinal target area. Due to the lack of subretinal visual feedback, it is hard to sense the insertion depth during the procedure, thus affecting the results of surgical outcome and hindering the widespread use of this treatment. This paper presents a novel approach to estimate the 3D position of the needle under the retina using the information from microscope-integrated Intraoperative Optical Coherence Tomography (iOCT). We evaluated our approach on both tissue phantom and ex-vivo porcine eyes. Evaluation results show that the average error in distance measurement is 4.7 μm (maximum of 16.5 μm). We furthermore, verified the feasibility of the proposed method to track the insertion depth of needle in robot-assisted subretinal injection.

Sonic interaction as a technique for conveying information has advantages over conventional visual augmented reality methods specially when augmenting the visual field with extra information brings distraction. Sonification of knowledge extracted by applying computational methods to sensory data is a well-established concept. However, some aspects of sonic interaction design such as aesthetics, the cognitive effort required for perceiving information, and avoiding alarm fatigue are not well studied in literature. In this work, we present a sonification scheme based on employment of physical modeling sound synthesis which targets focus demanding tasks requiring extreme precision. Proposed mapping techniques are designed to require minimum training for users to adapt to and minimum mental effort to interpret the conveyed information. Two experiments are conducted to assess the feasibility of the proposed method and compare it against visual augmented reality in high precision tasks. The observed quantitative results suggest that utilizing sound patches generated by physical modeling achieve the desired goal of improving the user experience and general task performance with minimal training.

Augmented Reality (AR) in microscopic surgery has been subject of several studies in the past two decades. Nevertheless, AR has not found its way into everyday microsurgical workflows. The introduction of new surgical microscopes equipped with Optical Coherence Tomography (OCT) enables the surgeons to perform multimodal (optical and OCT) imaging in the operating room. Taking full advantage of such elaborate source of information requires sophisticated intraoperative image fusion, information extraction, guidance and visualization methods. Medical AR is a unique approach to facilitate utilization of multimodal medical imaging devices. Here we propose a novel medical AR solution to the long-known problem of determining the distance between the surgical instrument tip and the underlying tissue in ophthalmic surgery to further pave the way of AR into the surgical theater. Our method brings augmented reality to OCT for the first time by augmenting the surgeon's view of the OCT images with an estimated instrument cross-section shape and distance to the retinal surface using only information from the shadow of the instrument in intraoperative OCT images. We demonstrate the applicability of our method in retinal surgery using a phantom eye and evaluate the accuracy of the augmented information using a micromanipulator.