Abstract Realistically inserting virtual 3D objects into real‐world images requires perceptually coherent shadowing of the object and background scene. Achieving this in single‐view indoor scenes with sunlight is challenging due to complex, partially visible occluders and indirect lighting. Environment maps alone cannot produce realistic shadows on virtual objects, and any representation (scene parameters) used for rendering must be practically estimable. We introduce ProjectiveShading, the first automatic method for inverse‐ and re‐rendering that handles bi‐directional shadow interactions for realistic object composition. Our key innovation is the sunlight map, a 2D image encoding direct sunlight and arbitrary occlusions. It is generated from single‐view estimations using off‐the‐shelf models and is compatible with standard rendering engines. We also propose algorithms to estimate sunlight direction and to blend virtual and real shadows while preserving background textures. Experiments on synthetic and in‐the‐wild images show our method outperforms previous approaches.

Rendering high-resolution photorealistic images in real time is challenging for video games and emerging virtual reality headsets. Thus, fovea sampled image reconstruction and super-resolution technologies become more and more crucial. However, most current methods process foveated reconstruction and super-resolution separately, which is slow. To address this issue, we propose a novel multi-scale spatiotemporal kernel prediction network for real-time foveated rendering that can perform sparse peripheral region reconstruction and supersampling simultaneously, resulting in a substantial reduction in rendering computation without visually noticeable quality degradation. Thanks to the multiscale kernel prediction architecture, different levels of details can be effectively preserved. Furthermore, we introduce an effective motion vector mask to explicitly identify occluded regions, which can help to use historical information more effectively. Our network runs in real time and achieves superior image quality and better inter-frame stability than existing methods.

We propose a complete pipeline for modeling and rendering realistic bird feathers from a single photograph, achieving both high visual fidelity and practical efficiency. Given a single input image of a feather, our approach extracts the feather's shaft curve, outline, and albedo, then reconstructs a compact hierarchical representation in a planar/curve (UV) domain. This representation encodes fine barb and barbule details procedurally, enabling continuous multiscale rendering with correct self-shadowing and masking. We analyze the appearance phenomena of different feathers and propose a new feather scattering model for non-iridescent feathers (e.g., parrot feathers), while introducing an additional sheen lobe to capture the distinctive fluffy rim-lighting effect. Our pipeline produces consistent, realistic results under arbitrary lighting and viewing conditions, and achieves real-time performance with a minimal memory footprint (0.02% of explicit-fiber geometry models), making it a practical solution for digital feather rendering without compromising realism.

In this paper, we propose an efficient method for hair and fur rendering that approximates multiple scattering as volumetric light transport, achieving near path-traced visual fidelity at a drastically reduced cost. Multiple scattering among hair fibers is crucial for a realistic appearance, especially for light-colored hair, but it is extremely expensive to simulate directly. Prior approximations, such as dual scattering and fur BSSRDF, improve performance but often fail for dense hair/fur, yielding an overly dry or blurred appearance. Unlike previous volumetric hair rendering solutions, we treat dense hair/fur as a highly anisotropic participating medium to capture soft volumetric illumination, while still using explicit fiber geometry for direct lighting to preserve fine details. We accumulate hair fiber coverage in screen space and stochastically sample a single-scattering event to approximate higher-order scattering. Our method reproduces the rich appearance of hair and fur resulting from multiple scattering, while running about 7–10× faster than path tracing, making it suitable for use in production. We also demonstrate that our approach is robust under a variety of lighting conditions.

Data-parallel ray tracing is a crucial technique for rendering large-scale scenes that exceed the memory capacity of a single compute node. It partitions scene data across multiple nodes and accesses remote data through inter-node communication. However, the resulting communication overhead remains a significant bottleneck for practical performance. Existing approaches mitigate this bottleneck by enhancing data locality through dynamic scheduling during rendering, typically employing spatial partitioning to enable access prediction. Although effective in some scenarios, these methods incur significant redundancy in base geometry when applied to large-scale instanced scenes. In this paper, we introduce the first object-space-based dynamic scheduling algorithm, which uses object groups as the scheduling units to eliminate redundant storage of base data in instanced scenes. Additionally, we propose two data access frequency prediction methods to guide asynchronous data prefetching, enhancing rendering efficiency. Compared to the state-of-the-art method, our approach achieves an average rendering speedup of 77.6%, with a maximum improvement of up to 146.1%, while incurring only a 5% increase in scene memory consumption.

Recovering high-fidelity spatially varying bidirectional reflectance distribution function (SVBRDF) maps from a single image remains an ill-posed and challenging problem, especially in the presence of saturated highlights. Existing methods often fail to reconstruct the underlying texture in regions overwhelmed by intense specular reflections. This kind of bake-in artifacts caused by highlight corruption can be greatly alleviated by providing a series of material images under different lighting conditions. To this end, our key insight is to leverage the strong priors of diffusion models to generate images of the same material under varying lighting conditions. These generated images are then used to aid a multi-image SVBRDF estimator in recovering highlight-free reflectance maps. However, strong highlights in the input image lead to inconsistencies across the relighting results. Moreover, texture reconstruction becomes unstable in saturated regions, with variations in background structure, specular shape, and overall material color. These artifacts degrade the quality of SVBRDF recovery. To address this issue, we propose a shuffle-based background consistency module that extracts stable background features and implicitly identifies saturated regions. This guides the diffusion model to generate coherent content while preserving material structures and details. Furthermore, to stabilize the appearance of generated highlights, we introduce a lightweight specular prior encoder that estimates highlight features and then performs grid-based latent feature translation, injecting consistent specular contour priors while preserving material color fidelity. Both quantitative analysis and qualitative visualization demonstrate that our method enables stable neural relighting from a single image and can be seamlessly integrated into multi-input SVBRDF networks to estimate highlight-free reflectance maps.

Abstract Realistic hair rendering remains a significant challenge in computer graphics due to the intricate microstructure of hair fibers and their anisotropic scattering properties, which make them highly sensitive to noise. Although recent advancements in image‐space and 3D‐space denoising and antialiasing techniques have facilitated real‐time rendering in simple scenes, existing methods still struggle with excessive blurring and artifacts, particularly in fine hair details such as flyaway strands. These issues arise because current techniques often fail to preserve sub‐pixel continuity and lack directional sensitivity in the filtering process. To address these limitations, we introduce a novel real‐time hair filtering technique that effectively reconstructs fine fiber details while suppressing noise. Our method improves visual quality by maintaining strand‐level details and ensuring computational efficiency, making it well‐suited for real‐time applications in video games and virtual reality (VR) and augmented reality (AR) environments.

Abstract We propose a new image‐based representation for real‐time distant objects rendering: Reshadable Impostors with Level‐of‐Detail (RiLoD). By storing compact geometric and material information captured from a few reference views, RiLoD enables reliable forward mapping to generate target views under dynamic lighting and edited material attributes. In addition, it supports seamless transitions across different levels of detail. To support reshading and LoD simultaneously while maintaining a minimal memory footprint and bandwidth requirement, our key design is a compact yet efficient representation that encodes and compresses the necessary material and geometric information in each reference view. To further improve the visual fidelity, we use a reliable forward mapping technique combined with a hole‐filling filtering strategy to ensure geometric completeness and shading consistency. We demonstrate the practicality of RiLoD by integrating it into a modern real‐time renderer. RiLoD delivers fast performance across a variety of test scenes, supports smooth transitions between levels of detail as the camera moves closer or farther, and avoids the typical artifacts of impostor techniques that result from neglecting the underlying geometry.

Soft shadows play a crucial role in enhancing visual realism in real-time rendering. Although traditional shadow mapping techniques offer high efficiency, they often suffer from artifacts and limited quality. In contrast, ray tracing can produce high-fidelity soft shadows but incurs substantial computational cost. In this paper, we propose a general-purpose, real-time soft shadow generation method based on neural networks. To encode shadow geometry, we employ the hard shadows via shadow mapping as input to our network, which effectively captures the spatial layout of shadow positions and contours. A lightweight U-Net architecture then refines this input to synthesize high-quality soft shadows in real time. The generated shadows closely approximate ray-traced references in visual fidelity. Compared to existing learning-based methods, our approach produces higher-quality soft shadows and offers improved generalization across diverse scenes. Furthermore, it requires no scene-specific precomputation, making it directly applicable to practical real-time rendering scenarios.

With the proliferation of mobile hardware-accelerated ray tracing, visual quality at low sampling rates (1spp) significantly deteriorates due to high-frequency noise and temporal artifacts introduced by Monte Carlo path tracing. Traditional spatiotemporal denoising methods, such as Spatiotemporal Variance-Guided Filtering (SVGF), effectively suppress noise by fusing multi-frame information and using geometry buffer (G-buffer) guided filters. However, their reliance on per-frame variance computation and global filtering imposes prohibitive overhead for mobile devices. This paper proposes an edge-aware, data-driven real-time denoising architecture within the SVGF framework, tailored explicitly for mobile computational constraints. Our method introduces two key innovations that eliminate variance estimation overhead: (1) an adaptive filtering kernel sizing mechanism, which dynamically adjusts filtering scope based on local complexity analysis of the G-buffer; and (2) a data-driven weight table construction strategy, converting traditional computational processes into efficient real-time lookup operations. These innovations significantly enhance processing efficiency while preserving edge accuracy. Experimental results on the Qualcomm Snapdragon 768G platform demonstrate that our method achieves 55 FPS with 1spp input. This frame rate is 67.42% higher than mobile-optimized SVGF, provides better visual quality , and reduces power consumption by 16.80% . Our solution offers a practical and efficient denoising framework suitable for real-time ray tracing in mobile gaming and AR/VR applications.

Many real-life materials have sparkling appearances. Some small flakes on the surface of an object can make a considerable contribution by reflecting or refracting light at a particular angle, eventually causing a sparkling appearance. Most existing approaches have focused on the glinty effects on reflective surfaces. However, transparent glint rendering has not been well studied, even though there are many natural phenomena (e.g., frost) in the real world. Recent studies have proposed the simulation of transparent glints under specific constraints (e.g., limited to the Beckmann distribution and V-groove shadowing-masking function). In this study, we propose a more general transparent glint model by performing a four-dimensional hierarchical search to count the particles located in the pixel footprint and cone around the refracted ray. Our method can produce transparent glint appearances for arbitrary normal distribution functions (e.g., GGX or Beckmann) and converge to a smooth microfacet model with a large particle count.

Foveated rendering exploits the non-uniform acuity of human vision to allocate computational resources more efficiently by reducing image fidelity in the peripheral field of view. While existing perceptual models for foveated rendering focus primarily on spatial resolution and contrast sensitivity, they overlook the perceptual asymmetry between direct and indirect illumination. In this work, we introduce a novel perceptual model that incorporates illuminance demodulation to account for this distinction. Our model adaptively modulates the foveation rate based on the relative contributions of direct and indirect illumination. Building on this model, we develop a practical rendering framework that separately applies tailored foveation strategies to direct and indirect illumination effects. Quantitative metrics and user studies confirm that our method maintains perceptual equivalence to full-resolution rendering. The sparse rendering stage achieves a $2.18\times$2.18× to $7.10\times$7.10× speedup, contributing to an overall acceleration of $1.71\times$1.71× to $3.26\times$3.26×.

With technological advancements, virtual reality (VR), once limited to high-end professional applications, is rapidly expanding into entertainment and broader consumer domains. However, the inherent contradiction between mobile hardware computing power and the demand for high-resolution, high-refresh-rate rendering has intensified, leading to critical bottlenecks, including frame latency and power overload, which constrain large-scale applications of VR systems. This study systematically analyzes four key technologies for efficient VR rendering: (1) foveated rendering, which dynamically reduces rendering precision in peripheral regions based on the physiological characteristics of the human visual system (HVS), thereby significantly decreasing graphics computation load; (2) stereo rendering, optimized through consistent stereo rendering acceleration algorithms; (3) cloud rendering, utilizing object-based decomposition and illumination-based decomposition for distributed resource scheduling; and (4) low-power rendering, integrating parameter-optimized rendering, super-resolution technology, and frame-generation technology to enhance mobile energy efficiency. Through a systematic review of the core principles and optimization approaches of these technologies, this study establishes research benchmarks for developing efficient VR systems that achieve high fidelity and low latency while providing further theoretical support for the engineering implementation and industrial advancement of VR rendering technologies.

It is time-consuming to render high-resolution images in applications such as video games and virtual reality, and thus super-resolution technologies become increasingly popular for real-time rendering. However, it is challenging to preserve sharp texture details, keep the temporal stability and avoid the ghosting artifacts in real-time super-resolution rendering. To address this issue, we introduce radiance demodulation to separate the rendered image or radiance into a lighting component and a material component, considering the fact that the light component is smoother than the rendered image so that the high-resolution material component with detailed textures can be easily obtained. We perform the super-resolution on the lighting component only and re-modulate it with the high-resolution material component to obtain the final super-resolution image with more texture details. A reliable warping module is proposed by explicitly marking the occluded regions to avoid the ghosting artifacts. To further enhance the temporal stability, we design a frame-recurrent neural network and a temporal loss to aggregate the previous and current frames, which can better capture the spatial-temporal consistency among reconstructed frames. As a result, our method is able to produce temporally stable results in real-time rendering with high-quality details, even in the challenging 4 × 4 super-resolution scenarios. Code is available at: https://github.com/Riga2/NSRD.

Abstract Screen Space Reflection (SSR) can reliably achieve highly efficient reflective effects, significantly enhancing users' sense of realism in real‐time applications. However, when directly applied to stereo rendering, popular SSR algorithms lead to inconsistencies due to the differing information between the left and right eyes. This inconsistency, invisible to human vision, results in visual discomfort. This paper analyzes and demonstrates how screen‐space geometries, fade boundaries, and reflection samples introduce inconsistent cues. Considering the complementary nature of screen information, we introduce a stereo‐aware SSR method to alleviate visual discomfort caused by screen space disparities. By contrasting our stereo‐aware SSR with conventional SSR and ray‐traced results, we showcase the effectiveness of our approach in mitigating the inconsistencies stemming from screen space differences while introducing affordable performance overhead for real‐time rendering.

Global illumination (GI) plays a crucial role in rendering realistic results for virtual exhibitions, such as virtual car exhibitions. These scenarios usually include all-frequency bidirectional reflectance distribution functions (BRDFs), although their geometries and light configurations may be static. Rendering all-frequency BRDFs in real time remains challenging due to the complex light transport. Existing approaches, including precomputed radiance transfer, light probes, and the most recent path-tracing-based approaches (ReSTIR PT), cannot satisfy both quality and performance requirements simultaneously. Herein, we propose a practical hybrid global illumination approach that combines ray tracing and cached GI by caching the incoming radiance with wavelets. Our approach can produce results close to those of offline renderers at the cost of only approximately 17 ms at runtime and is robust over all-frequency BRDFs. Our approach is designed for applications involving static lighting and geometries, such as virtual exhibitions.

Reconstructing objects with realistic materials from multi-view images is problematic, since it is highly ill-posed. Although the neural reconstruction approaches have exhibited impressive reconstruction ability, they are designed for objects with specific materials (e.g., diffuse or specular materials). To this end, we propose a novel framework for robust geometry and material reconstruction, where the geometry is expressed with the implicit signed distance field (SDF) encoded by a tensorial representation, namely TensoSDF. At the core of our method is the roughness-aware incorporation of the radiance and reflectance fields, which enables a robust reconstruction of objects with arbitrary reflective materials. Furthermore, the tensorial representation enhances geometry details in the reconstructed surface and reduces the training time. Finally, we estimate the materials using an explicit mesh for efficient intersection computation and an implicit SDF for accurate representation. Consequently, our method can achieve more robust geometry reconstruction, outperform the previous works in terms of relighting quality, and reduce 50% training times and 70% inference time. Codes and datasets are available at https://github.com/Riga2/TensoSDF.

Distributed ray tracing algorithms are widely used when rendering massive scenes, where data utilization and load balancing are the keys to improving performance. One essential observation is that rays are temporally coherent, which indicates that temporal information can be used to improve computational efficiency. In this paper, we use temporal coherence to optimize the performance of distributed ray tracing. First, we propose a temporal coherence-based scheduling algorithm to guide the task/data assignment and scheduling. Then, we propose a virtual portal structure to predict the radiance of rays based on the previous frame, and send the rays with low radiance to a precomputed simplified model for further tracing, which can dramatically reduce the traversal complexity and the overhead of network data transmission. The approach was validated on scenes of sizes up to 355 GB. Our algorithm can achieve a speedup of up to 81% compared to previous algorithms, with a very small mean squared error.

Abstract Caustics are complex optical effects caused by the light being concentrated in a small area due to reflection or refraction on surfaces with low roughness, typically under a sharp light source. Rendering caustic effects is challenging for Monte Carlo‐based approaches, due to the difficulties of sampling the specular paths. One effective solution is using the specular manifold to locate these valid specular paths. Unfortunately, it needs many iterations to find these paths, leading to a long rendering time. To address this issue, our key insight is that the specular paths tend to be similar for neighboring shading points. To this end, we propose to reuse the specular paths spatially. More specifically, we generate some specular path samples with a low sample rate and then reuse these specular path samples as the initialization for specular manifold walk among neighboring shading points. In this way, much fewer specular path‐searching iterations are performed, due to the efficient initialization close to the final solution. Furthermore, this reuse strategy can be extended for dynamic scenes in a temporal manner, such as light moving or specular geometry deformation. Our method outperforms current state‐of‐the‐art methods and can handle multiple bounces of light and various scenes.

Abstract We present a new software ray tracing solution that efficiently computes visibilities in dynamic scenes. We first introduce a novel scene representation: ray‐aligned occupancy map array (ROMA) that is generated by rasterizing the dynamic scene once per frame. Our key contribution is a fast and low‐divergence tracing method computing visibilities in constant time, without constructing and traversing the traditional intersection acceleration data structures such as BVH. To further improve accuracy and alleviate aliasing, we use a spatiotemporal scheme to stochastically distribute the candidate ray samples. We demonstrate the practicality of our method by integrating it into a modern real‐time renderer and showing better performance compared to existing techniques based on distance fields (DFs). Our method is free of the typical artifacts caused by incomplete scene information, and is about 2.5×–10× faster than generating and tracing DFs at the same resolution and equal storage.

Various temporal denoising methods have been proposed to clean up the noise for real-time ray tracing (RTRT). These methods rely on the temporal correspondences of pixels between the current and previous frames, i.e. per-pixel screen-space motion vectors. However, the state-of-the-art temporal reuse methods with traditional motion vectors cause artifacts in motion occlusions. We accordingly propose a novel neural temporal denoising method for indirect illumination of Monte Carlo (MC) ray tracing at 1 sample per pixel. Based on end-to-end multi-scale kernel-based reconstruction, we apply temporally reliable dual motion vectors to facilitate better reconstruction of the occlusions, and also introduce additional motion occlusion loss to reduce ghosting artifacts. Experiments show that our method significantly reduces the over-blurring and ghosting artifacts while generating high-quality images at real-time rates.

Basis functions provide both the abilities for compact representation and the properties for efficient computation. Therefore, they are pervasively used in rendering to perform all-frequency shading. However, common basis functions, including spherical harmonics (SH), wavelets, and spherical Gaussians (SG) all have their own limitations, such as low-frequency for SH, not rotationally invariant for wavelets, and no multiple product support for SG. In this paper, we present neural basis functions, an implicit and data-driven set of basis functions that circumvents the limitations with all desired properties. We first introduce a representation neural network that takes any general 2D spherical function (e.g. environment lighting, BRDF, and visibility) as input and projects it onto the latent space as coefficients of our neural basis functions. Then, we design several lightweight neural networks that perform different types of computation, giving our basis functions different computational properties such as double/triple product integrals and rotations. We demonstrate the practicality of our neural basis functions by integrating them into all-frequency shading applications, showing that our method not only achieves a compression rate of and 10 × -40 × better performance than wavelets at equal quality, but also renders all-frequency lighting effects in real-time without the aforementioned limitations from classic basis functions.

Abstract Normal map‐based microstructure rendering can generate both glint and scratch appearance accurately. However, the extra high‐resolution normal map that defines every microfacet normal may incur high storage and computation costs. We present an example‐based real‐time rendering method for arbitrary microstructure materials, which significantly reduces the required storage space. Our method takes a small‐size normal map sample as input. We implicitly synthesize a high‐resolution normal map from the normal map sample and construct MIP‐mapped 4D position‐normal Gaussian lobes. Based on the above MIP‐mapped 4D lobes and a LUT (lookup table) data structure for the synthesized high‐resolution normal map, an efficient Gaussian query method is presented to evaluate ‐NDFs (position‐normal distribution functions) for shading. We can render complex scenes with glint and scratch surfaces in real time ( 30 fps) with a full high‐definition resolution, and the space required for each microstructure material is decreased to 30 MB.

The interaction between light and materials is key to physically-based realistic rendering. However, it is also complex to analyze, especially when the materials contain a large number of details and thus exhibit “glinty” visual effects. Recent methods of producing glinty appearance are expected to be important in next-generation computer graphics. We provide here a comprehensive survey on recent glinty appearance rendering. We start with a definition of glinty appearance based on microfacet theory, and then summarize research works in terms of representation and practical rendering. We have implemented typical methods using our unified platform and compare them in terms of visual effects, rendering speed, and memory consumption. Finally, we briefly discuss limitations and future research directions. We hope our analysis, implementations, and comparisons will provide insight for readers hoping to choose suitable methods for applications, or carry out research.

Fur appearance rendering is crucial for the realism of computer generated imagery, but is also a challenge in computer graphics for many years. Much effort has been made to accurately simulate the multiple-scattered light transport among fur fibers, but the computation cost is still very high, since the number of fur fibers is usually extremely large. In this paper, we aim at reducing the number of fur fibers while preserving realistic fur appearance. We present an aggregated fur appearance model, using one thick cylinder to accurately describe the aggregated optical behavior of a bunch of fur fibers, including the multiple scattering of light among them. Then, to acquire the parameters of our aggregated model, we use a lightweight neural network to map individual fur fiber's optical properties to those in our aggregated model. Finally, we come up with a practical heuristic that guides the simplification process of fur dynamically at different bounces of the light, leading to a practical level-of-detail rendering scheme. Our method achieves nearly the same results as the ground truth, but performs 3.8×-13.5× faster.

Abstract Monte Carlo rendering is widely used in the movie industry. Since it is costly to produce noise‐free results directly, Monte Carlo denoising is often applied as a post‐process. Recently, deep learning methods have been successfully leveraged in Monte Carlo denoising. They are able to produce high quality denoised results, even with very low sample rate, e.g. 4 spp (sample per pixel). However, for difficult scene configurations, some details could be blurred in the denoised results. In this paper, we aim at preserving more details from inputs rendered with low spp. We propose a novel denoising pipeline that handles three‐scale features ‐ pixel, sample and path ‐ to preserve sharp details, uses an improved Res2Net feature extractor to reduce the network parameters and a smooth feature attention mechanism to remove low‐frequency splotches. As a result, our method achieves higher denoising quality and preserves better details than the previous methods.

Abstract Real‐time ray tracing (RTRT) is being pervasively applied. The key to RTRT is a reliable denoising scheme that reconstructs clean images from significantly undersampled noisy inputs, usually at 1 sample per pixel as limited by current hardware's computing power. The state of the art reconstruction methods all rely on temporal filtering to find correspondences of current pixels in the previous frame, described using per‐pixel screen‐space motion vectors. While these approaches are demonstrated powerful, they suffer from a common issue that the temporal information cannot be used when the motion vectors are not valid, i.e. when temporal correspondences are not obviously available or do not exist in theory. We introduce temporally reliable motion vectors that aim at deeper exploration of temporal coherence, especially for the generally‐believed difficult applications on shadows, glossy reflections and occlusions, with the key idea to detect and track the cause of each effect. We show that our temporally reliable motion vectors produce significantly better temporal results on a variety of dynamic scenes when compared to the state of the art methods, but with negligible performance overhead.

Complex luminaires, such as grand chandeliers, can be extremely costly to render because the light-emitting sources are typically encased in complex refractive geometry, creating difficult light paths that require many samples to evaluate with Monte Carlo approaches. Previous work has attempted to speed up this process, but the methods are either inaccurate, require the storage of very large lightfields, and/or do not fit well into modern path-tracing frameworks. Inspired by the success of deep networks, which can model complex relationships robustly and be evaluated efficiently, we propose to use a machine learning framework to compress a complex luminaire's lightfield into an implicit neural representation. Our approach can easily plug into conventional renderers, as it works with the standard techniques of path tracing and multiple importance sampling (MIS). Our solution is to train three networks to perform the essential operations for evaluating the complex luminaire at a specific point and view direction, importance sampling a point on the luminaire given a shading location, and blending to determine the transparency of luminaire queries to properly composite them with other scene elements. We perform favorably relative to state-of-the-art approaches and render final images that are close to the high-sample-count reference with only a fraction of the computation and storage costs, with no need to store the original luminaire geometry and materials.

Rendering participating media is important to the creation of photorealistic images. Participating media has a translucent aspect that comes from light being scattered inside the material. For materials with a small mean-free-path (mfp), multiple scattering effects dominate. Simulating these effects is computationally intensive, as it requires tracking a large number of scattering events inside the material. Existing approaches precompute multiple scattering events inside the material and store the results in a table. During rendering time, this table is used to compute the scattering effects. While these methods are faster than explicit scattering computation, they incur higher storage costs. In this paper, we present a new representation for double and multiple scattering effects that uses a neural network model. The scattering response from all homogeneous participating media is encoded into a neural network in a preprocessing step. At run time, the neural network is then used to predict the double and multiple scattering effects. We demonstrate the effects combined with Virtual Ray Lights (VRL), although our approach can be integrated with other rendering algorithms. Our algorithm is implemented on GPU. Double and multiple scattering effects for the entire participating media space are encoded using only 23.6 KB of memory. Our method achieves 50 ms per frame in typical scenes and provides results almost identical to the reference.

Abstract Gradient‐domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient‐domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient‐domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation.

Monte Carlo based methods such as path tracing are widely used in movie production. To achieve low noise, they require many samples per pixel, resulting in long rendering time. To reduce the cost, one solution is Monte Carlo denoising, which renders the image with fewer samples per pixel (as little as 128) and then denoises the resulting image. Many Monte Carlo denoising methods rely on deep learning: they use convolutional neural networks to learn the relationship between noisy images and reference images, using auxiliary features such as position and normal together with image color as inputs. The network predicts kernels which are then applied to the noisy input. These methods show powerful denoising ability, but tend to lose geometric or lighting details and to blur sharp features during denoising. In this paper, we solve this issue by proposing a novel network structure, a new input feature—light transport covariance from path space—and an improved loss function. Our network separates feature buffers from the color buffer to enhance detail effects. The features are extracted separately and then integrated into a shallow kernel predictor. Our loss function considers perceptual loss, which also improves detail preservation. In addition, we use a light transport covariance feature in path space as one of the features, which helps to preserve illumination details. Our method denoises Monte Carlo path traced images while preserving details much better than previous methods.

Many materials combine a refractive boundary and a participating media on the interior. If the material has a low opacity, single scattering effects dominate in its appearance. Refraction at the boundary concentrates the incoming light, resulting in an important phenomenon called volume caustics. This phenomenon is hard to simulate. Previous methods used point-based light transport, but attributed point samples inefficiently, resulting in long computation time. In this paper, we use frequency analysis of light transport to allocate point samples efficiently. Our method works in two steps: in the first step, we compute volume samples along with their covariance matrices, encoding the illumination frequency content in a compact way. In the rendering step, we use the covariance matrices to compute the kernel size for each volume sample: small kernel for high-frequency single scattering, large kernel for lower frequencies. Our algorithm computes volume caustics with fewer volume samples, with no loss of quality. Our method is both faster and uses less memory than the original method. It is roughly twice as fast and uses one fifth of the memory. The extra cost of computing covariance matrices for frequency information is negligible.

Nowadays, many users select to outsource their job executions to service clouds. These users often have heterogeneous demands while they dynamically arrive at the clouds. For reducing the costs and risks, lots of service cloud operators purchase on-demand instances from public IaaS clouds and provide professional services elastically to users. However, without knowing the future information, it is hard for cloud operators to optimally determine the instance purchasing as well as job scheduling and pricing schemes. In order to achieve maximum social welfare, this paper targets to design an auction mechanism which executes in an online fashion for service clouds, with unique features of job-oriented users, pleasingly parallel jobs and soft deadline constraints. Such a mechanism ought to run in polynomial time, provide truthfulness guarantee, satisfy individual rationality and budget balance, and achieve competitive social welfare. Nevertheless, when designing mechanisms there are a few significant challenges, including the difficulty for finding optimal solution, the strategic behaviours of selfish users with private information and the online arrivals of users. Facing these challenges, we leverage the idea of proportional sharing and propose an online mechanism which is proven to achieve all desired properties. The efficiency of the proposed mechanism is validated by both theoretical analysis and extensive simulations which use both synthetic data and Google's job traces.

Abstract Microfacet theory is commonly used to build reflectance models for surfaces. While traditional microfacet‐based models assume that the distribution of a surface's microstructure is continuous, recent studies indicate that some surfaces with tiny, discrete and stochastic facets exhibit glittering visual effects, while some surfaces with structured features exhibit anisotropic specular reflection. Accordingly, this paper proposes an efficient and stationary method of surface material modeling to process both glittery and non‐glittery surfaces in a consistent way. Our method comprises two steps: in the preprocessing step, we take a fixed‐size sample normal map as input, then organize 4D microfacet trees in position and normal space for arbitrary‐sized surfaces; we also cluster microfacets into 4D K‐lobes via the adaptive k‐means method. In the rendering step, moreover, surface normals can be efficiently evaluated using pre‐clustered microfacets. Our method is able to efficiently render any structured, discrete and continuous micro‐surfaces using a precisely reconstructed surface NDF. Our method is both faster and uses less memory compared to the state‐of‐the‐art glittery surface modeling works.

Abstract Many real‐life materials have a sparkling appearance, whether by design or by nature. Examples include metallic paints, sparkling varnish but also snow. These sparkles correspond to small, isolated, shiny particles reflecting light in a specific direction, on the surface or embedded inside the material. The particles responsible for these sparkles are usually small and discontinuous. These characteristics make it difficult to integrate them efficiently in a standard rendering pipeline, especially for indirect illumination. Existing approaches use a 4‐dimensional hierarchy, searching for light‐reflecting particles simultaneously in space and direction. The approach is accurate, but still expensive. In this paper, we show that this 4‐dimensional search can be approximated using separate 2‐dimensional steps. This approximation allows fast integration of glint contributions for large footprints, reducing the extra cost associated with glints be an order of magnitude.

With the fast development of virtual reality games, one of the key research questions is how players may express their creativity and participate in the process of game design. In this paper, we present a new game genre which combines user-controlled game design in physical space with game play in virtual space on a mobile device. The new system supports authoring by anyone, creating virtual reality games that can be easily modified or developed for physical space, and be used anywhere by novice end-users without any knowledge of tracking technology. We present the design and implementation of the system, as well as a user experiment. Findings illustrate that the proposed system promotes participation and provides a richer, more interactive and engaging experience.

Progressive photon mapping [Hachisuka et al. 2008] (PPM) obtains increasingly accurate results with progressive visualization, but it is problematic when results are obtained through thousands of iterations. An uniform photon distribution is critical for the accurate result. In this work, we use the sample elimination [Yuksel 2015] (SE) in PPM to achieve optimal results and accelerate the iterations. Sample elimination can produce sample sets with more pronounced blue noise characteristics. We apply the feature of this elimination method to the progressive iterations.

Most of the previous 3 DOF (Degree Of Freedom) 3D touch positioning techniques require more than one finger (usually two hands) to be performed, which limits their using space on small mobile devices such as phones and tablets that need one hand to be held in most occasions. Given that the pressure sensitive touch screen would become ubiquitous in near future, we present the pressure-based 3DOF 3D objects positioning and manipulating techniques that only use one hand in operating.

Global illumination is the core part of photo-realistic rendering. The photon mapping algorithm is an effective method for computing global illumination with its obvious advantage of caustic and color bleeding rendering. It is an active research field that has been developed over the past two decades. The deficiency of precise details and efficient rendering are still the main challenges of photon mapping. This report reviews recent work and classifies it into a set of categories including radiance estimation, photon relaxation, photon tracing, progressive photon mapping, and parallel methods. The goals of our report are giving readers an overall introduction to photon mapping and motivating further research to address the limitations of existing methods.

Temporal coherence is one of the central challenges for rendering a stylized line. It is especially difficult for stylized contours of coarse meshes or non-uniformly sampled models, because those contours are polygonal feature edges on the models with no continuous correspondences between frames. We describe a novel and simple technique for constructing a 2D brush path along a 3D contour. We also introduce a 3D parameter propagation and re-parameterization procedure to construct stroke paths along the 2D brush path to draw coherently stylized feature lines with a wide range of styles. Our method runs in real-time for coarse or non-uniformly sampled models, making it suitable for interactive applications needing temporal coherence.

This paper presents a method to render smooth stylized contours of 3D meshes with temporal coherence. Contour triangles that contain countour curves perform two functions: they ensure that smooth contour curves are coherent and reconstructed faithfully during the motion of the viewpoint and they help to establish the corresponding relationship of contours which can propagate parameters on contours between frames against sliding effects. Our method can also deal with zig-zags and short segments for coarse meshes and nonuniform sampling meshes in a wide range of styles very well.

We propose a real-time rendering system for automatically creating a caricature drawing, i.e., an exaggerated portrait, of a human face, based on simultaneous use of a range image (or 3D mesh) and a registered photograph of the same face. Combining these information sources provides complementary information. Significant geometric lines such as occluding contours and suggestive contours are extracted from the range data, while textured areas corresponding to shading features are extracted from the photograph. These are combined, and then distorted to produce the final caricature. The final output may be produced using a choice of non-photorealistic rendering styles. Our system method works well for low resolution range images; for these it is fast enough to allow the viewpoint to be chosen in real time. The final output combines significant lines, textured areas, and optional shading, giving a pleasing result which preserves not only the shape cues of the geometric description, but also other essential visual characteristics of the facial image that cannot be deduced from geometry alone.

Coarse piecewise linear approximation of surfaces causes undesirable polygonal appearance of silhouettes. We present an efficient method for smoothing the silhouettes of coarse triangle meshes using efficient 3D curve reconstruction and simple local re-meshing. It does not assume the availability of a fine mesh and generates only moderate amount of additional data at run time. Furthermore, polygonal feature edges are also smoothed in a unified framework. Our method is based on a novel interpolation scheme over silhouette triangles and this ensures that smooth silhouettes are faithfully reconstructed and always change continuously with respect to continuous movement of the view point or objects. We speed up computation with GPU assistance to achieve real-time rendering of coarse meshes with the smoothed silhouettes. Experiments show that this method outperforms previous methods for silhouette smoothing.

Digital museums become necessary when we have to present the precious collections to more and more visitors while preventing the state of preservation of the collections deteriorating over the time. It has significance to share and protect expensive museum resources. However, most current digital museum systems present the vast amount information in a static and primitive fashion and are not fully capable of dealing with generally heterogeneous and isolated resources. In this paper, we propose a grid-enabled digital museum system to improve the quality, availability and variety of the information presentation and sharing by incorporating grid-computing technologies. By designing cooperative grid services, we could efficiently and effectively organize, present and share the resources over the grid. The framework and workflow of the system are illustrated and the implementation details are also given.

This paper mainly discusses how to design a virtual museum system used by many users at the same time, and how to manage the scene data. To a 3D indoor scene, the empty scene (only the building) without other objects such as collections can be created by sweeping a 2D polygon along an axis which is perpendicular to the horizontal plane. We can create the scene polygon's Voronoi diagram, which subdivides the polygon into some cells called Voronoi regions by the proximity characteristic. An edge of the polygon corresponding to a wall of the building has its own Voronoi region where all the points are nearest to the edge than others. Hence, every user can select one Voronoi region to lay out collections such as cultural relics on the exhibition platform and every collection is saved by the Voronoi region in which it is laid out. Based on the scene polygon's Voronoi diagram, the system can also support multi-user cooperative interaction in the created scene very well. The work also can be used to create virtual art gallery, product exhibition hall and so on.

This paper describes a system framework of 3D virtual museum supporting multi-user cooperative interaction. To a 3D virtual museum, the building (empty scene) can be projected to a 2D polygon. Hence, we can manage the data of collections and users based on the scene polygon's Voronoi diagram. We can get the Voronoi diagram by subdividing the polygon into some cells using the proximity characteristic. We also establish some communication rules between users. In this article, how to compute visibility and detect collisions based on polygon's Voronoi diagram is discussed and this method is aimed to improve the reality and coherence of cooperative interaction in virtual museum.