Robotic bronchoscopic intervention requires detailed 3D airway maps for both localisation and enhanced visualisation, especially at peripheral airways. Patient-specific airway maps can be generated from preoperative chest CT scans. Due to pathological abnormalities and anatomical variations, automatically delineating the airway tree with distal branches is a challenging task. In the paper, we propose a cascaded 2D+3D model that has been tailored for airway segmentation from pathological CT scans. A novel 2D neural network is developed to generate the initial predictions where the peripheral airways are refined by a 3D adversarial training model. A sampling strategy based on a sequence of morphological operations is employed for the concatenation of the 2D and 3D models. The method has been validated on 20 pathological CT scans with results demonstrating improved segmentation accuracy and consistency, especially in peripheral airways.

This paper presents an image-based method for virtual bronchoscope with photo-realistic rendering. The technique is based on recovering bidirectional reflectance distribution function (BRDF) parameters in an environment where the choice of viewing positions, directions, and illumination conditions are restricted. Video images of bronchoscopy examinations are combined with patient-specific three-dimensional (3-D) computed tomography data through two-dimensional (2-D)/3-D registration and shading model parameters are then recovered by exploiting the restricted lighting configurations imposed by the bronchoscope. With the proposed technique, the recovered BRDF is used to predict the expected shading intensity, allowing a texture map independent of lighting conditions to be extracted from each video frame. To correct for disocclusion artefacts, statistical texture synthesis was used to recreate the missing areas. New views not present in the original bronchoscopy video are rendered by evaluating the BRDF with different viewing and illumination parameters. This allows free navigation of the acquired 3-D model with enhanced photo-realism. To assess the practical value of the proposed technique, a detailed visual scoring that involves both real and rendered bronchoscope images is conducted.

Photo-realistic rendering combined with vision techniques is an important trend in developing next generation surgical simulation devices. Training with simulator is generally low in cost and more efficient than traditional methods that involve supervised learning on actual patients. Incorporating genuine patient data in the simulation can significantly improve the efficacy of training and skills assessment. In this paper, a photo-realistic simulation architecture is described that utilises patient-specific models for training in minimally invasive surgery. The datasets are constructed by combining computer tomographic images with bronchoscopy video of the same patient so that the three dimensional structures and visual appearance are accurately matched. Using simulators enriched by a library of datasets with sufficient patient variability, trainees can experience a wide range of realistic scenarios, including rare pathologies, with correct visual information. In this paper, the matching of CT and video data is accomplished by using a newly developed 2D/3D registration method that exploits a shape from shading similarity measure. Additionally, a method has been devised to allow shading parameter estimation by modelling the bidirectional reflectance distribution function (BRDF) of the visible surfaces. The derived BRDF is then used to predict the expected shading intensity such that a texture map independent of lighting conditions can be extracted. Thus new views can be generated that were not captured in the original bronchoscopy video, thus allowing free navigation of the acquired 3D model with enhanced photo-realism.