The Gaussian relighting is critical for appearance editing and physical-based rendering, which rely on the decomposition and estimation of light and materials. However, existing methods treat both the environmental lighting and material features as unknowns and estimate them jointly, which results in inaccurate material decomposition and estimation due to the inherent variation of environmental illumination across datasets. In this study, we introduce the realistic and stable 3D Gaussian relighting (RS3DGR) that is enabled by the semantic-guided stable material estimation through object-oriented differentiable Gaussian path tracing. First, the target object is segmented from environment via semantic 3D Gaussian segmentation and the semantic label is designed as the roughness regularization term in optimization. Then, the object is illuminated by the environmental light sampled from the radiance field of environmental Gaussians or other types of radiance caches. The object-oriented path tracing rendering is differentiable and efficient. Under the alpha blending of primary rays penetrating through object Gaussians, the secondary rays are uniformly emitted from those primary-hit Gaussians to directly sample the environmental radiance field to model the reflection on object surface. As a result, the influence of unstable environmental lighting is removed, and the semantic prior successfully promotes the convergence on material estimation. Experimental results demonstrate significant improvements in stability and accuracy for material reconstruction on both synthetic and self-constructed datasets.

Three-dimensional (3D) light field displays (LFDs) provide immersive visual experiences and have attracted increasing attention. However, visual fatigue remains an important concern when users watch 3D LFDs which limits their development and application. In this paper, we propose a comprehensive methodology that integrates subjective and objective data to establish a robust dataset and employs eye movement data for systematically investigating visual fatigue in 3D LFDs. Firstly, a multimodal dataset is constructed by integrating subjective fatigue scores and objective eye movement data collection. Then, we propose the Deep Correlation Data Analysis Model (DCDAM), which uses Spearman's rank correlation coefficient to analyze correlations between key objective metrics and subjective fatigue curves, validating the effectiveness of these metrics. Furthermore, to comprehensively assess visual fatigue, we develop a specialized model, the Temporo-Spatial Synergy Network (TSSNet), which uses temporal and spatial eye movement features to predict subjective fatigue curves. Through validation across diverse videos, the model achieves R² > 0.98 (±0.005) and RMSE of 0.02 (±0.05) between actual and predicted values, demonstrating high precision and valid generalization across different video content. The proposed model provides a foundational framework for future research on visual fatigue assessment tasks of 3D LFDs.

Multi-view online hashing has received much at-tention due to its huge potential in the area of large-scale multimedia retrieval. However, there are still some issues, e.g., how to alleviate the catastrophic forgetting, how to adequately extract high-level semantic information of multi-view streaming data and improve the discrimination of hash models, and how to effectively optimize the binary constraint problem. In this paper, we propose a novel Semantic Consistency based Dual-asymmetric Discrete Online Hashing method, SC-DDOH for short. It adopts a dual-asymmetric distance-based similarity supervision to retain similarities of new data chunk and database. To extract efficient high-level semantic information, an online semantic consistent supervision to mine the semantic related information from word embedding labels. Moreover, an efficient discrete iterative optimization algorithm is introduced to directly learn hash code in the Hamming space. Experiment results on three large-scale multi-view datasets demonstrate the superiority of SC-DDOH over the state-of-the-art baselines.

Online hashing has gained significant interest for its tremendous promise in handling large-scale streaming image data. However, there are still several problems to be solved. Firstly, many existing methods struggle to fully utilize previous knowledge and fail to effectively mitigate catastrophic forgetting. Secondly, current online hashing methods lack targeted guidance for hash function learning, resulting in weak discriminative ability. Thirdly, many current methods adopt ineffective optimization methods to learn hash code. This paper proposes a Discriminative Visual-semantic Collaborative Online Decomposition Hashing method, abbreviated as DVsCODH. It includes two steps: hash code learning and hash function learning. In the first step, DVsCODH adopts an online pairwise similarity supervision to guide hash code learning, and simultaneously introduces a visual-semantic collaborative online decomposition strategy to adaptively fuse visual and semantic information. Through incremental online learning, DVsCODH is capable of extracting complete similarity, visual and high-level semantic information of the database. In the second step, DVsCODH leverages database-wide similarity to supervise hash function online learning. Furthermore, we propose an effective online discrete optimization algorithm that directly generate hash code, thereby enhancing model training efficiency. Experimental results on three public image datasets demonstrate the excellent retrieval performance of DVsCODH.

Holographic displays are ideal display technologies for virtual and augmented reality because all visual cues are provided. However, real-time high-quality holographic displays are difficult to achieve because the generation of high-quality computer-generated hologram (CGH) is inefficient in existing algorithms. Here, complex-valued convolutional neural network (CCNN) is proposed for phase-only CGH generation. The CCNN-CGH architecture is effective with a simple network structure based on the character design of complex amplitude. A holographic display prototype is set up for optical reconstruction. Experiments verify that state-of-the-art performance is achieved in terms of quality and generation speed in existing end-to-end neural holography methods using the ideal wave propagation model. The generation speed is three times faster than HoloNet and one-sixth faster than Holo-encoder, and the Peak Signal to Noise Ratio (PSNR) is increased by 3 dB and 9 dB, respectively. Real-time high-quality CGHs are generated in 1920 × 1072 and 3840 × 2160 resolutions for dynamic holographic displays.

Neural Radiance Fields (NeRF) can implicitly represent 3D-consistent RGB images and geometric by optimizing an underlying continuous volumetric scene function using a sparse set of input views, which has greatly benefited view synthesis tasks. However, NeRF fails to estimate correct geometry when given fewer views, resulting in failure to synthesize novel views. Existing works rely on introducing depth images or adding depth estimation networks to resolve the problem of poor synthetic view in NeRF with fewer views. However, due to the lack of spatial consistency of the single-depth image and the poor performance of depth estimation with fewer views, the existing methods still have challenges in addressing this problem. So this paper proposes Bidirectional Optical Flow NeRF(BOF-NeRF), which addresses this problem by mining optical flow information between 2D images. Our key insight is that utilizing 2D optical flow images to design a loss can effectively guide NeRF to learn the correct geometry and synthesize the right novel view. We also propose a view-enhanced fusion method based on geometry and color consistency to solve the problem of novel view details loss in NeRF. We conduct extensive experiments on the NeRF-LLFF and DTU MVS benchmarks for novel view synthesis tasks with fewer images in different complex real scenes. We further demonstrate the robustness of BOF-NeRF under different baseline distances on the Middlebury dataset. In all cases, BOF-NeRF outperforms current state-of-the-art baselines for novel view synthesis and scene geometry estimation.