Precise elevation design on an aerially reconstructed terrain can ensure an appropriate structural slope for good driving conditions and dovetail the design with real world. Traditional planners are used to drawing curves for 2D floor design, but there is no mechanism to design a 3D terrain based on editing the cross-section of selected 2D ground curves. Therefore, this work aims to bridge the gap between the intuitive 2D curve-based design practice and precise 3D terrain construction. Since floor planners generally refer to 2D cross-sections for elevations and are more familiar with curve manipulation instead of 3D mesh editing, this work proposes an elevation modeling tool based on editing the 2D cross-section of user-drawn ground-projected curves. The system is designed to align with intuitive 2D curve-based design practices by first letting users draw curves on the ground as 3D-constrained parametric curves. It constructs a ground-projected plane for 2D cross-sectional manipulation to achieve precise altitude control. The system diffuses the parametric elevations to their surroundings and constructs its height field by solving the corresponding differential equation. Finally, remeshing is applied to ensure patch boundaries precisely align with the constrained curves. Thus, our system can address the difficulty and complexity in precise elevation control and boundary alignment of brush-style, feature-based, and NURBS-based knot-tuning methods. Numerical experiments and a usability study of naive users and experts demonstrate our intuitiveness and effectiveness over state-of-the-art methods and verify our ability to mimic traditional 2D curve-based design practice.

Abstract Pailous are representative Chinese architectural works used for commemoration. However, their geometric structure and semantic construction rules are too complex for quick and intuitive modelling using traditional modelling tools. We propose an intuitive modelling system for the stylized creation of pailous for novices. Our system encapsulates structural components as icons and semantic layouts as topological graphs, using which users create and manipulate icons with topological recommendations. The interpreter automatically and immediately transforms a graph to its corresponding model using built‐in components with the proposed parametric L‐system grammars derived from architectural rules. Using this system to re‐create existing representative pailous and design imaginary ones yields results with the desired visual complexities. In comparison to Maya, a 3D modelling tool, when modelling a pailou and toukung, our system is effective and simple, and eliminates the need to remember and understand complex rules.

The vector graphics with gradient mesh can be attributed to their compactness and scalability; however, they tend to fall short when it comes to real-time editing due to a lack of real-time rasterization and an efficient editing tool for image details. In this paper, we encode global manipulation geometries and local image details within a hybrid vector structure, using parametric patches and detailed features for localized and parallelized thin-plate spline interpolation in order to achieve good compressibility, interactive expressibility, and editability. The proposed system then automatically extracts an optimal set of detailed color features while considering the compression ratio of the image as well as reconstruction error and its characteristics applicable to the preservation of structural and irregular saliency of the image. The proposed real-time vector representation makes it possible to construct an interactive editing system for detail-maintained image magnification and color editing as well as material replacement in cross mapping, without maintaining spatial and temporal consistency while editing in a raster space. Experiments demonstrate that our representation method is superior to several state-of-the-art methods and as good as JPEG, while providing real-time editability and preserving structural and irregular saliency information.

Generative models based on deep neural networks often have a high-dimensional latent space, ranging sometimes to a few hundred dimensions or even higher, which typically makes them hard for a user to explore directly. We propose differential subspace search to allow efficient iterative user exploration in such a space, without relying on domain- or data-specific assumptions. We develop a general framework to extract low-dimensional subspaces based on a local differential analysis of the generative model, such that a small change in such a subspace would provide enough change in the resulting data. We do so by applying singular value decomposition to the Jacobian of the generative model and forming a subspace with the desired dimensionality spanned by a given number of singular vectors stochastically selected on the basis of their singular values, to maintain ergodicity. We use our framework to present 1D subspaces to the user via a 1D slider interface. Starting from an initial location, the user finds a new candidate in the presented 1D subspace, which is in turn updated at the new candidate location. This process is repeated until no further improvement can be made. Numerical simulations show that our method can better optimize synthetic black-box objective functions than the alternatives that we tested. Furthermore, we conducted a user study using complex generative models and the results show that our method enables more efficient exploration of high-dimensional latent spaces than the alternatives.

Background extraction is generally the first step in many computer vision and augmented reality applications. Most existing methods, which assume the existence of a clean background during the reconstruction period, are not suitable for video sequences such as highway traffic surveillance videos, whose complex foreground movements may not meet the assumption of a clean background. Therefore, we propose a novel joint Gaussian conditional random field (JGCRF) background extraction algorithm for estimating the optimal weights of frame composition for a fixed-view video sequence. A maximum a posteriori problem is formulated to describe the intra- and inter-frame relationships among all pixels of all frames based on their contrast distinctness and spatial and temporal coherence. Because all background objects and elements are assumed to be static, patches that are motionless are good candidates for the background. Therefore, in the algorithm method, a motionless extractor is designed by computing the pixel-wise differences between two consecutive frames and thresholding the accumulation of variation across the frames to remove possible moving patches. The proposed JGCRF framework can flexibly link extracted motionless patches with desired fusion weights as extra observable random variables to constrain the optimization process for more consistent and robust background extraction. The results of quantitative and qualitative experiments demonstrated the effectiveness and robustness of the proposed algorithm compared with several state-of-the-art algorithms; the proposed algorithm also produced fewer artifacts and had a lower computational cost.

It is important to extract a clear background for computer vision and augmented reality. Generally, background extraction assumes the existence of a clean background shot through the input sequence, but realistically, situations may violate this assumption such as highway traffic videos. Therefore, our probabilistic model-based method formulates fusion of candidate background patches of the input sequence as a random walk problem and seeks a globally optimal solution based on their temporal and spatial relationship. Furthermore, we also design two quality measures to consider spatial and temporal coherence and contrast distinctness among pixels as background selection basis. A static background should have high temporal coherence among frames, and thus, we improve our fusion precision with a temporal contrast filter and an optical-flow-based motionless patch extractor. Experiments demonstrate that our algorithm can successfully extract artifact-free background images with low computational cost while comparing to state-of-the-art algorithms.

With advances in the movie industry, composite interactions and complex visual effects require us to shoot at the designed part of a scene for immersion. Traditionally, the director of photography plans a camera path by recursively reviewing and commenting path-planning rendered results. Since the adjust-render-review process is not immediate and interactive, miscommunications happen to make the process ineffective and time consuming. Therefore, this paper proposes a What-You-See-What-You-Get camera path reviewing system for the director to interactively instruct and design camera paths. Our system consists of a camera handle, a parameter control board, and a camera tracking box with mutually perpendicular marker planes. When manipulating the handle, the attached camera captures markers on visible planes with selected parameters to adjust the world rendering view. The director can directly examine results to give immediate comments and feedbacks on transformation and parameter adjustment in order to achieve effective communication and reduce the reviewing time. Finally, we conduct a set of qualitative and quantitative evaluations to show that our system is robust and efficient and can provide means to give interactive and immediate instructions for effective communication and efficiency enhancement during path design.

High-resolution textures are determinant of not only high rendering quality in gaming and movie industries, but also of burdens in memory usage, data transmission bandwidth, and rendering efficiency. Therefore, it is desirable to shade 3D objects with vector images such as scalable vector graphics (SVG) for compactness and resolution independence. However, complicated geometry and high rendering cost limit the rendering effectiveness and efficiency of vector texturing techniques. In order to overcome these limitations, this paper proposes a real-time resolution-independent vector-embedded shading method for 3D animated objects. Our system first decomposes a vector image consisting of layered close coloring regions into unifying-coloring units for mesh retriangulation and 1D coloring texture construction, where coloring denotes color determination for a point based on an intermediate medium such as a raster/vector image, unifying denotes the usage of the same set of operations, and unifying coloring denotes coloring with the same-color computation operations. We then embed the coloring information and distances to enclosed unit boundaries in retriangulated vertices to minimize embedded information, localize vertex-embedded shading data, remove overdrawing inefficiency, and ensure fixed-length shading instructions for data compactness and avoidance of indirect memory accessing and complex programming structures when using other shading and texturing schemes. Furthermore, stroking is the process of laying down a fixed-width pen-centered element along connected curves, and our system also decomposes these curves into segments using their curve-mesh intersections and embeds their control vertices as well as their widths in the intersected triangles to avoid expensive distance computation. Overall, our algorithm enables high-quality real-time Graphics Processing Unit (GPU)-based coloring for real-time 3D animation rendering through our efficient SVG-embedded rendering pipeline while using a small amount of texture memory and transmission bandwidth.

Introducing motion into existing static paintings is becoming a field that is gaining momentum. This effort facilitates keeping artworks current and translating them to different forms for diverse audiences. Chinese ink paintings and Japanese Sumies are well recognized in Western cultures, yet not easily practiced due to the years of training required. We are motivated to develop an interactive system for artists, non-artists, Asians, and non-Asians to enjoy the unique style of Chinese paintings. In this paper, our focus is on replacing static water flow scenes with animations. We include flow patterns, surface ripples, and water wakes which are challenging not only artistically but also algorithmically. We develop a data-driven system that procedurally computes a flow field based on stroke properties extracted from the painting, and animate water flows artistically and stylishly. Technically, our system first extracts water-flow-portraying strokes using their locations, oscillation frequencies, brush patterns, and ink densities. We construct an initial flow pattern by analyzing stroke structures, ink dispersion densities, and placement densities. We cluster extracted strokes as stroke pattern groups to further convey the spirit of the original painting. Then, the system automatically computes a flow field according to the initial flow patterns, water boundaries, and flow obstacles. Finally, our system dynamically generates and animates extracted stroke pattern groups with the constructed field for controllable smoothness and temporal coherence. The users can interactively place the extracted stroke patterns through our adapted Poisson-based composition onto other paintings for water flow animation. In conclusion, our system can visually transform a static Chinese painting to an interactive walk-through with seamless and vivid stroke-based flow animations in its original dynamic spirits without flickering artifacts.

Manga are a popular artistic form around the world, and artists use simple line drawing and screentone to create all kinds of interesting productions. Vectorization is helpful to digitally reproduce these elements for proper content and intention delivery on electronic devices. Therefore, this study aims at transforming scanned Manga to a vector representation for interactive manipulation and real-time rendering with arbitrary resolution. Our system first decomposes the patch into rough Manga elements including possible borders and shading regions using adaptive binarization and screentone detector. We classify detected screentone into simple and complex patterns: our system extracts simple screentone properties for refining screentone borders, estimating lighting, compensating missing strokes inside screentone regions, and later resolution independently rendering with our procedural shaders. Our system treats the others as complex screentone areas and vectorizes them with our proposed line tracer which aims at locating boundaries of all shading regions and polishing all shading borders with the curve-based Gaussian refiner. A user can lay down simple scribbles to cluster Manga elements intuitively for the formation of semantic components, and our system vectorizes these components into shading meshes along with embedded Bézier curves as a unified foundation for consistent manipulation including pattern manipulation, deformation, and lighting addition. Our system can real-time and resolution independently render the shading regions with our procedural shaders and drawing borders with the curve-based shader. For Manga manipulation, the proposed vector representation can be not only magnified without artifacts but also deformed easily to generate interesting results.

It is a necessary but challenging task for creative producers to have an idea of the depth perception of the target audience when watching a stereoscopic film in a cinema during production. This paper proposes a novel metric, geometric perceived depth percentage (GPDP), to numerate and depict the depth perception of a scene before rendering. In addition to the geometric relationship between the object depth and focal distance, GPDP takes the screen width and viewing distance into account. As a result, it provides a more intuitive means for predicting stereoscopy and is universal across different viewing conditions. Based on GPDP, we design a practical tool to visualize the stereoscopic perception without the need for any 3-D device or special environment. The tool utilizes the stereoscopic comfort volume, GPDP-based shading schemes, depth perception markers, and GPDP histograms as visual cues so that animators can set stereoscopic parameters more easily. The tool is easily implemented in any modern rendering pipeline, including interactive Autodesk Maya and offline Pixar's RenderMan renderer. It has been used in several production projects including commercial ones. Finally, two user studies show that GPDP is a proper depth perception indicator and the proposed tool can make the stereoscopic parameter setting process easier and more efficient.

Abstract Flow visualization is recognized as an essential tool for many scientific research fields and different visualization approaches are proposed. Several studies are also conducted to evaluate their effectiveness but these studies rarely examine the performance from the perspective of visual perception. In this paper, we aim at exploring how users’ visual perception is influenced by different 2D flow visualization methods. An eye tracker is used to analyze users’ visual behaviors when they perform the free viewing, advection prediction, flow feature detection, and flow feature identification tasks on the flow field images generated by different visualizations methods. We evaluate the illustration capability of five representative visualization algorithms. Our results show that the eye‐tracking‐based evaluation provides more insights to quantitatively analyze the effectiveness of these visualization methods.

Simplicity refers to one of the most important photography composition rules. Simplicity states that simplifying the image background can draw viewers' attention to the subject of interest in a photograph and help them better comprehend and appreciate it. Understanding whether a photo respects photography rules or not facilitates photo quality assessment. In this paper, we present a method to automatically detect whether a photo is composed according to the rule of simplicity. We design features according to the definition, implementation and effect of the rule. First, we make use of saliency analysis to infer the subject of interest in a photo and measure its compactness. Second, we segment an image into background and foreground and measure the homogeneity within the background as another feature. Third, when looking at an image created with the rule of simplicity, different viewers tend to agree on what the subject of interest is in this photo. We accordingly measure the consistency among various saliency detection results as a feature. We experiment with these features in a range of machine learning methods. Our experiments show that our methods, together with these features, provide an encouraging result in detecting the rule of simplicity in a photo.

Abstract We present the Optimizing Control Variate (OCV) estimator, a new estimator for Monte Carlo rendering. Based upon a deterministic sampling framework, OCV allows multiple importance sampling functions to be combined in one algorithm. Its optimizing nature addresses a major problem with control variate estimators for rendering: users supply a generic correlated function which is optimized for each estimate, rather than a single highly tuned one that must work well everywhere. We demonstrate OCV with both direct lighting and irradiance‐caching examples, showing improvements in image error of over 35% in some cases, for little extra computation time. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Three‐Dimensional Graphics and Realism Color, shading, shadowing, and texture G.3 [Probability and Statistics]: Probabilistic Algorithms Keywords: direct lighting, deterministic mixture sampling, control variates