Opportunities for medical students to gain practical experience in vaginal births are increasingly constrained by shortened clinical rotations, patient reluctance, and the unpredictable nature of labour. To alleviate clinicians' instructional burden and enhance trainees' learning efficiency, we introduce a mixed reality (MR) system for childbirth training that combines virtual guidance with tactile manikin interaction, thereby preserving authentic haptic feedback while enabling independent practice without continuous on-site expert supervision. The system extends the passthrough capability of commercial head-mounted displays (HMDs) by spatially calibrating an external RGB-D camera, allowing real-time visual integration of physical training objects. Building on this capability, we implement a coarse-to-fine localization pipeline that first aligns the maternal manikin with fiducial markers to define a delivery region and then registers the pre-scanned neonatal head within this area. This process enables spatially accurate overlay of virtual guiding hands near the manikin, allowing trainees to follow expert trajectories reinforced by haptic interaction. Experimental evaluations demonstrate that the system achieves accurate and stable manikin localization on a standalone headset, ensuring practical deployment without external computing resources. A large-scale user study involving 83 fourth-year medical students was subsequently conducted to compare MR-based and virtual reality (VR)-based childbirth training. Four senior obstetricians independently assessed performance using standardized criteria. Results showed that MR training achieved significantly higher scores in delivery, post-delivery, and overall task performance, and was consistently preferred by trainees over VR training. Although validated only in the context of obstetric delivery, the system demonstrates strong potential for broader manikin-based procedural training and other healthcare education scenarios.

Immersive virtual reality (VR) applications demand accurate, temporally coherent full-body pose tracking. Recent head-mounted camera-based approaches show promise in egocentric pose estimation, but encounter challenges when applied to VR head-mounted displays (HMDs), including temporal instability, inaccurate lower-body estimation, and the lack of real-time inference. To address these limitations, we present EgoPoseVR, an end-to-end framework for accurate egocentric full-body pose estimation in VR that integrates headset motion cues with egocentric RGB-D observations through a dual-modality fusion pipeline. A spatiotemporal encoder extracts frame- and joint-level representations, which are fused via cross-attention to fully exploit complementary motion cues across modalities. A kinematic optimization module then imposes constraints from HMD signals, enhancing the accuracy and stability of pose estimation. To facilitate training and evaluation, we introduce a large-scale synthetic dataset of over 1.8 million temporally aligned HMD and RGB-D frames across diverse VR scenarios. Experimental results show that EgoPoseVR outperforms state-of-the-art egocentric pose estimation models. A user study in real-world scenes further shows that EgoPoseVR achieved significantly higher subjective ratings in accuracy, stability, embodiment, and intention for future use compared to baseline methods. These results show that EgoPoseVR enables robust full-body pose tracking, offering a practical solution for accurate VR embodiment without requiring additional body-worn sensors or room-scale tracking systems.

State-of-the-art image matching methods have shown strong generalization across natural image datasets, but their effectiveness in complex surgical environments remains underexplored. Surgical scenes introduce unique challenges, including homogeneous tissue textures, variable lighting, and frequent occlusions, which can degrade the reliability of keypoint correspondences essential for downstream vision tasks such as camera pose estimation and structure-from-motion. In this study, we systematically evaluate leading image matching methods within laparoscopic surgical settings, emphasizing performance under resource-constrained conditions. We present an optimized evaluation pipeline that incorporates robust estimators to enhance correspondence filtering and assess their impact on pose estimation accuracy. Our approach also examines the influence of fine-tuning individual pipeline components, particularly robust estimators, on overall system performance. Mean Reprojection error is refined by thresholding the nearest ground truth projections, enabling a more precise characterization of matching accuracy. Across five robust estimators, FM_8PTS consistently demonstrates superior resilience to outliers. Our results establish RoMa as the leading model for balancing pose estimation accuracy, reprojection performance, and computational efficiency, making it suitable for real-time surgical applications. By providing the first systematic benchmark and actionable insights for optimizing image matching pipelines in surgical domains, this work sets a new standard and paves the way for more reliable, efficient, and clinically applicable image-guided tools in minimally invasive surgery.

Direct Volume Rendering (DVR) plays an important role in scientific data visualization. To generate photo-realistic DVR results, the physical light transport throughout the volume is simulated by applying the Monte Carlo-based volumetric path tracing (VPT) approach. For real-time applications, due to the time constraint for rendering each frame, only a limited number of samples shall be taken for the computation per pixel. This can result in a significant amount of noise in the rendering results. This paper describes our optimized VPT sampling algorithm and a novel denoising technique to generate consistently high-quality realistic DVR results in real time. We develop a new shading model that can reduce estimation variance to enhance the quality of DVR results. Additionally, a hybrid acceleration structure is created by integrating both octree and macrocell to improve sampling efficiency. This allows the acquisition of sufficiently more shading samples while maintaining the desired interactive frame rate. To further eliminate remaining noise and improve temporal stability of DVR results, we develop a novel spatiotemporal denoising framework. Our denoiser decouples the estimated radiance into high-detail low-noise and low-detail high-noise components. Different denoising algorithms are separately applied to these components to reduce noise without introducing blurring artifacts. Our DVR system can consistently offer high rendering quality and good temporal stability across DVR result frames in real time. During fast user interactions and with rapid alterations of the illumination condition, our rendering method can still provide good visual comfort and representation accuracy without visible latency.

A high dynamic range (HDR) image is commonly used to reveal stereo illumination, which is crucial for generating high-quality realistic rendering effects. Compared to the high-cost HDR imaging technique, low dynamic range (LDR) imaging provides a low-cost alternative and is preferable for interactive graphics applications. However, the limited LDR pixel bit depth significantly bothers accurate illumination estimation using LDR images. The conflict between the realism and promptness of illumination estimation for realistic rendering is yet to be resolved. In this paper, an efficient method that accurately infers illuminations of real-world scenes using LDR panoramic images is proposed. It estimates multiple lighting parameters, including locations, types and intensities of light sources. In our approach, a new algorithm that extracts illuminant characteristics during the exposure attenuation process is developed to locate light sources and outline their boundaries. To better predict realistic illuminations, a new deep learning model is designed to efficiently parse complex LDR panoramas and classify detected light sources. Finally, realistic illumination intensities are calculated by recovering the inverse camera response function and extending the dynamic range of pixel values based on previously estimated parameters of light sources. The reconstructed radiance map can be used to compute high-quality image-based lighting of virtual models. Experimental results demonstrate that the proposed method is capable of efficiently and accurately computing comprehensive illuminations using LDR images. Our method can be used to produce better realistic rendering results than existing approaches.

Abstract Monoscopic panorama provides the display of omnidirectional contents surrounding the viewer. An increasingly popular way to reconstruct a panorama is to stitch a collection of fisheye images. However, such non‐planar views may result in problems such as distortions and boundary irregularities. In most cases, the computational expense for stitching non‐planar images is also too high to satisfy real‐time applications. In this paper, a novel monoscopic panorama reconstruction pipeline that produces better quad‐fisheye image stitching results for omnidirectional environment viewing is proposed. The main idea is to apply mesh deformation for image alignment. To optimize inter‐lens parallaxes, unwarped images are firstly cropped and reshuffled to facilitate the circular environment scene composition by the seamless ring‐connection of the panorama borders. Several mesh constraints are then adopted to ensure a high alignment accuracy. After alignment, the boundary of the result is rectified to be rectangular to prevent gapping artefacts. We further extend our approach to video stitching. The temporal smoothness model is added to prevent unexpected artefacts in the panoramic videos. To support interactive applications, our stitching algorithm is programmed using CUDA. The camera motion and average gradient per video frame are further calculated to accelerate for synchronous real‐life panoramic scene reconstruction and visualization. Experimental results demonstrate that our method has advantages in respects of alignment accuracy, adaptability and image quality of the stitching result.