Addressing the intricate challenge of modeling and re-rendering dynamic scenes, most recent approaches have sought to simplify these complexities using plane-based explicit representations, overcoming the slow training time issues associated with methods like Neural Radiance Fields (NeRF) and implicit representations. However, the straight-forward decomposition of 4D dynamic scenes into multiple 2D plane-based representations proves insufficient for re-rendering high-fidelity scenes with complex motions. In response, we present a novel direction-aware representation (DaRe) approach that captures scene dynamics from six different directions. This learned representation under-goes an inverse dual-tree complex wavelet transformation (DTCWT) to recover plane-based information. DaReNeRF computes features for each space-time point by fusing vectors from these recovered planes. Combining DaReNeRF with a tiny MLP for color regression and leveraging volume rendering in training yield state-of-the-art performance in novel view synthesis for complex dynamic scenes. Notably, to address redundancy introduced by the six real and six imag-inary direction-aware wavelet coefficients, we introduce a trainable masking approach, mitigating storage issues without significant performance decline. Moreover, DaReNeRF maintains a 2 × reduction in training time compared to prior art while delivering superior performance.

A common problem with segmentation of medical images using neural networks is the difficulty to obtain a significant number of pixel-level annotated data for training. To address this issue, we proposed a semi-supervised segmentation network based on contrastive learning. In contrast to the previous state-of-the-art, we introduce Min-Max Similarity (MMS), a contrastive learning form of dual-view training by employing classifiers and projectors to build all-negative, and positive and negative feature pairs, respectively, to formulate the learning as solving a MMS problem. The all-negative pairs are used to supervise the networks learning from different views and to capture general features, and the consistency of unlabeled predictions is measured by pixel-wise contrastive loss between positive and negative pairs. To quantitatively and qualitatively evaluate our proposed method, we test it on four public endoscopy surgical tool segmentation datasets and one cochlear implant surgery dataset, which we manually annotated. Results indicate that our proposed method consistently outperforms state-of-the-art semi-supervised and fully supervised segmentation algorithms. And our semi-supervised segmentation algorithm can successfully recognize unknown surgical tools and provide good predictions. Also, our MMS approach could achieve inference speeds of about 40 frames per second (fps) and is suitable to deal with the real-time video segmentation.

The anatomy of the ear and the bones surrounding it are intricate yet critical for medical professionals to know. Current best practices teach ear anatomy through two-dimensional representations, which poorly characterize the three-dimensional (3D), spatial nature of the anatomy and make it difficult to learn and visualize. In this work, we describe an immersive, stereoscopic visualization tool for the anatomy of the ear based on real patient data. We describe the interface and its construction. And we compare how well medical students learn ear anatomy in the simulation compared with more traditional learning methods. Our preliminary results suggest that virtual reality may be an effective tool for anatomy education in this context.

Multi-channel cochlear implants (CI) leverage frequency based cochlear tonotopic mapping to map acoustic information to the cochlear place of stimulation which is primarily determined by electrode locations. Despite the fact that electrode locations within the cochlea are unique to each patient, the acoustic frequencies assigned to the electrodes by the CI processor are determined generically, resulting in a mismatch between intended and actual pitch perception. This is known to be a limiting factor for hearing outcomes with CIs. In this study, we propose a novel, image-guided CI processor programming strategy to select more optimal, patient-customized frequency assignments. The performance of the proposed strategy was evaluated using vocoder-based simulations with ten normal hearing listeners. In our simulations, our strategy results in significantly better speech recognition scores than the standard clinical strategy.