Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data. We introduce SinGRAF, a 3D-aware generative model that is trained with a few input images of a single scene. Once trained, SinGRAF generates different realizations of this 3D scene that preserve the appearance of the input while varying scene layout. For this purpose, we build on recent progress in 3D GAN architectures and introduce a novel progressive-scale patch discrimination approach during training. With several experiments, we demonstrate that the results produced by Sin-GRAF outperform the closest related works in both quality and diversity by a large margin.

Temporal interpolation often plays a crucial role to learn meaningful representations in dynamic scenes. In this paper, we propose a novel method to train spatiotemporal neural radiance fields of dynamic scenes based on temporal interpolation of feature vectors. Two feature interpolation methods are suggested depending on underlying representations, neural networks or grids. In the neural representation, we extract features from space-time inputs via multiple neural network modules and interpolate them based on time frames. The proposed multi-level feature interpolation network effectively captures features of both short-term and long-term time ranges. In the grid representation, space-time features are learned via four-dimensional hash grids, which remarkably reduces training time. The grid representation shows more than 100x faster training speed than the previous neural-net-based methods while maintaining the rendering quality. Concatenating static and dynamic features and adding a simple smoothness term further improve the performance of our proposed models. Despite the simplicity of the model architectures, our method achieved state-of-the-art performance both in rendering quality for the neural representation and in training speed for the grid representation.

Recently, there has been an upsurge of activity in image-based non-photorealistic rendering (NPR), and in particular portrait image stylisation, due to the advent of neural style transfer (NST). However, the state of performance evaluation in this field is poor, especially compared to the norms in the computer vision and machine learning communities. Unfortunately, the task of evaluating image stylisation is thus far not well defined, since it involves subjective, perceptual, and aesthetic aspects. To make progress towards a solution, this paper proposes a new structured, three-level, benchmark dataset for the evaluation of stylised portrait images. Rigorous criteria were used for its construction, and its consistency was validated by user studies. Moreover, a new methodology has been developed for evaluating portrait stylisation algorithms, which makes use of the different benchmark levels as well as annotations provided by user studies regarding the characteristics of the faces. We perform evaluation for a wide variety of image stylisation methods (both portrait-specific and general purpose, and also both traditional NPR approaches and NST) using the new benchmark dataset.

Removing specular reflections from images is critical for improving the performance of computer vision algorithms. Recently, state-of-the-art methods have demonstrated remarkably good performance at removing specular reflections from chromatic images. These methods are typically based on the chromatic pixels assumption; therefore, they are prone to failure in the achromatic regions. This paper presents a novel method that is applicable to natural images, because it is effective for both chromatic and achromatic regions. The proposed method is based on modeling the general properties of diffuse and specular reflections in a solid convex optimization framework. Considering the physical constraints, we determine the global optimal solution using the split Bregman method. Experimental results demonstrate the effectiveness of the proposed method, particularly for the achromatic regions, and its competence as a state-of-the-art method for removing specular reflections from the chromatic regions.

Abstract We present a novel method for enhancing details in a digital photograph, inspired by the principle of art photography. In contrast to the previous methods that primarily rely on tone scaling, our technique provides a flexible tone transform model that consists of two operators: shifting and scaling. This model permits shifting of the tonal range in each image region to enable significant detail boosting regardless of the original tone. We optimize these shift and scale factors in our constrained optimization framework to achieve extreme detail enhancement across the image in a piecewise smooth fashion, as in art photography. The experimental results show that the proposed method brings out a significantly large amount of details even from an ordinary low‐dynamic range image.

Abstract We propose various simulation strategies to generate single‐frame fire effects for images, as opposed to multi‐frame fire effects for animations. To accelerate 3D simulation and to provide a user with early hints on the final effect, we propose a 2D‐guided 3D simulation approach, which runs a faster 2D simulation first, and then guides 3D simulation using the 2D simulation result. To achieve this, we explore various boundary conditions and develop a constrained projection method. Since only the final frame will be used while intermediate frames are abandoned, earlier intermediate frames can take larger time steps and have large noise applied, quickly generating turbulent flow structures. As the final frame approaches, we increase the flow quality by reducing the time step and not adding any noise. This adaptive time stepping allows us to use more computational resource near or at the final frame. We also develop divergence and buoyancy modification methods to guide flames along arbitrary, even physically implausible, directions. Our simulation methods can effectively and efficiently generate a variety of fire effects useful for image decoration.

Abstract This paper presents an automatic method for producing stipple renderings from photographs, following the style of professional hedcut illustrations. For effective depiction of image features, we introduce a novel dot placement algorithm which adapts stipple dots to the local shapes. The core idea is to guide the dot placement along ‘feature flow’ extracted from the feature lines, resulting in a dot distribution that conforms to feature shapes. The sizes of dots are adaptively determined from the input image for proper tone representation. Experimental results show that such feature‐guided stippling leads to the production of stylistic and feature‐emphasizing dot illustrations.

We present a novel scheme for automatically generating line drawings from 2D images, aiming to facilitate effective visual communication. In contrast to conventional edge detectors, our technique imitates the human line drawing process and consists of two parts: line extraction and line rendering. We propose a novel line extraction method based on likelihood-function estimation, which effectively finds the genuine shape boundaries. We consider the feature scale and the blurriness of lines with which the detail and the focus-level of lines are controlled in the rendering. We also employ stroke textures to provide a variety of illustration styles. Experimental results demonstrate that our technique generates various kinds of line drawings from 2D images enabled by the control over detail, focus, and style.