Abstract The aim of this research is to propose an extended mean value cloning algorithm with automatic trimap generation and accurate alpha matting. This implementation improves the visualization accuracy of the merged video by reducing the discolored and smudging artefacts of the remote surgeon’s boundary. It also makes the merge robust for the illumination changes by taking less processing time in real time surgery. The proposed system uses automatic trimap generation from the source video for accurate foreground extraction. Extended mean value cloning with gradient mixing is then applied for the cloning with optimized alpha matting for accurate and realistic video composition. The proposed system improved the visualization accuracy by providing almost 99.7% visibility of the pixels compared to the state-of-the-art solution, which provides 99.1% visibility of pixels. The overlay error was reduced from 0.93 mm to 0.63 mm. The processing time was also reduced. The proposed solution processed 8 frames per second, which is less time than the state-of-the-art solution, which processed 5 frames per second. The extended mean value cloning smooths the differences that presented in the target and source frames for seamless and realistic blending of pixels. The automatic trimap generation reduced the risk of false foreground selection and the generated optimal trimaps improved the alpha matte quality, which is optimized to reduce the smudging artefacts completely and to produce accurate visualization of the final merged image.

Abstract Over time, Augmented Reality (AR) based technology becomes not being properly to implement with oral and maxillofacial surgery to visualise the narrow area spot in jaw surgery as blood vassals and root canals in these types of surgeries. Image registration is considered the major limitation of using the AR in these types of surgeries and reduces the accuracy of visualising the narrow areas. In this research, we propose a Correntropy based scale ICP algorithm as a solution to improve the image registration during jaw surgery. Correntropy is considered here to minimise the error metric of the ICP algorithm instead of the Euclidean distance measurement compared to the state-of-the-art solution. This led to decrease the registration error, increase the video accuracy and reduce the processing time simultaneously. The proposed system consists of Enhanced Tracking Learning Detection (TLD), which is used as an occlusion removal featured algorithm in the intra-operative stage of the AR-based jaw surgery system. In this research, a Modified Correntropy-based enhanced ICP (MCbeICP) algorithm is proposed for the system’s pose-refinement phase. Moreover, this proposed algorithm (MCbeICP) has a new function to process the point set registration with great noises and outliers. It eliminates the poor performance of the ICP algorithm of the noisy point set. Furthermore, the ICP algorithm considers the scale factor to register the point with different scales of the real-time video and the sample models. Additionally, this method improves the result of the pose refinement stage in terms of registration accuracy and processing time. By this method, the pose refinement stage gives an improved result in terms of registration accuracy and processing time. The samples, which were taken from the upper (maxillary) and the lower (mandible) jaw bone show that the proposed algorithm provides a significant accuracy improvement in alignment to 0.21- 0.29 mm from 0.23 to 0.35 mm and an increment in processing time from 8 to 12 frames per second (fs/s) to 10-14 fs/s compared to the result provided by state of the art. The proposed augmented reality (AR) system is focused on the overlay accuracy and processing time. Finally, this study addressed the limitation of Image registration with AR using modified Correntropy-based enhanced ICP algorithm to implement oral and maxillofacial surgery successfully.