Recently, there has been an upsurge of activity in image-based non-photorealistic rendering (NPR), and in particular portrait image stylisation, due to the advent of neural style transfer (NST). However, the state of performance evaluation in this field is poor, especially compared to the norms in the computer vision and machine learning communities. Unfortunately, the task of evaluating image stylisation is thus far not well defined, since it involves subjective, perceptual, and aesthetic aspects. To make progress towards a solution, this paper proposes a new structured, three-level, benchmark dataset for the evaluation of stylised portrait images. Rigorous criteria were used for its construction, and its consistency was validated by user studies. Moreover, a new methodology has been developed for evaluating portrait stylisation algorithms, which makes use of the different benchmark levels as well as annotations provided by user studies regarding the characteristics of the faces. We perform evaluation for a wide variety of image stylisation methods (both portrait-specific and general purpose, and also both traditional NPR approaches and NST) using the new benchmark dataset.

As a rule, user interface designers work iteratively. Over the course of a project, they repeatedly gather feedback, typically through in-person meetings, and update their designs accordingly. Through formative work, we find that design software tools do not support designers in managing meeting notes and previous design iterations as a cohesive whole. This causes designers to rely on ad-hoc practices for organizing work, which makes it hard for them to keep track of relevant feedback and explain their design decisions. To address this problem, we present Charrette, a system that allows designers to curate design iterations, attach meeting notes to the relevant content, and navigate sequences of design iterations with the associated notes to facilitate in-person discussions. In an exploratory user study, we evaluate how Charrette affects designers' self-reported ease in handling feedback during face-to-face discussions, compared with using their own tools. We find that using Charrette correlates with increased confidence and recall in discussing previous design decisions.

In the context of scene understanding, a variety of methods exists to estimate different information channels from mono or stereo images, including disparity, depth, and normals. Although several advances have been reported in the recent years for these tasks, the estimated information is often imprecise particularly near depth discontinuities or creases. Studies have however shown that precisely such depth edges carry critical cues for the perception of shape, and play important roles in tasks like depth-based segmentation or foreground selection. Unfortunately, the currently extracted channels often carry conflicting signals, making it difficult for subsequent applications to effectively use them. In this paper, we focus on the problem of obtaining high-precision depth edges (i.e., depth contours and creases) by jointly analyzing such unreliable information channels. We propose DepthCut , a data-driven fusion of the channels using a convolutional neural network trained on a large dataset with known depth. The resulting depth edges can be used for segmentation, decomposing a scene into depth layers with relatively flat depth, or improving the accuracy of the depth estimate near depth edges by constraining its gradients to agree with these edges. Quantitatively, we compare against 18 variants of baselines and demonstrate that our depth edges result in an improved segmentation performance and an improved depth estimate near depth edges compared to data-agnostic channel fusion. Qualitatively, we demonstrate that the depth edges result in superior segmentation and depth orderings. (Code and datasets will be made available.)

Vectorization turns photographs into vector art. Manual vectorization, where the artist traces over the image by hand, requires skill and time. On the other hand, automatic approaches allow users to generate a result by setting a few global parameters. However, global settings often leave too much detail/complexity in some parts of the image while missing important details in others. We propose interactive vectorization tools that offer more local control than automatic systems, but are more powerful and high-level than simple curve editing. Our system enables novices to vectorize images significantly faster than even experts with state-of-the-art tools.

Accurately drawing 3D objects is difficult for untrained individuals, as it requires an understanding of perspective and its effects on geometry and proportions. Step-by-step tutorials break the complex task of sketching an entire object down into easy-to-follow steps that even a novice can follow. However, creating such tutorials requires expert knowledge and is time-consuming. As a result, the availability of tutorials for a given object or viewpoint is limited. How2Sketch (H2S) addresses this problem by automatically generating easy-to-follow tutorials for arbitrary 3D objects. Given a segmented 3D model and a camera viewpoint, H2S computes a sequence of steps for constructing a drawing scaffold comprised of geometric primitives, which helps the user draw the final contours in correct perspective and proportion. To make the drawing scaffold easy to construct, the algorithm solves for an ordering among the scaffolding primitives and explicitly makes small geometric modifications to the size and location of the object parts to simplify relative positioning. Technically, we formulate this scaffold construction as a single selection problem that simultaneously solves for the ordering and geometric changes of the primitives. We generate different tutorials on man-made objects using our method and evaluate how easily the tutorials can be followed with a user study.

Complex software offers power for experts, yet overwhelms new users. Novices often do not know how to execute tasks, what they want to achieve, or even what is possible. To address this, we introduce the DiscoverySpace interface for executable action suggestions. DiscoverySpace is a prototype extension panel for Adobe Photoshop that suggests task-level action macros to apply to photographs based on visual features. DiscoverySpace harvests these one-click actions from the online Photoshop user community. A between-subjects study indicated that action suggestions may help novices maintain confidence, accomplish tasks, and discover features. This work demonstrates how interfaces can leverage user-generated content to help novices navigate complex software.

Current digital painting tools are primarily targeted at professionals and are often overwhelmingly complex for use by novices. At the same time, simpler tools may not invoke the user creatively, or are limited to plain styles that lack visual sophistication. There are many people who are not art professionals, yet would like to partake in digital creative expression. Challenges and rewards for novices differ greatly from those for professionals. In this paper, we leverage existing works in Creativity and Creativity Support Tools (CST) to formulate design goals specifically for digital art creation tools for novices. We implemented these goals within a digital painting system, called Painting with Bob. We evaluate the efficacy of the design and our prototype with a user study, and we find that users are highly satisfied with the user experience, as well as the paintings created with our system.

We present PortraitSketch, an interactive drawing system that helps novices create pleasing, recognizable face sketches without requiring prior artistic training. As the user traces over a source portrait photograph, PortraitSketch automatically adjusts the geometry and stroke parameters (thickness, opacity, etc.) to improve the aesthetic quality of the sketch. We present algorithms for adjusting both outlines and shading strokes based on important features of the underlying source image. In contrast to automatic stylization systems, PortraitSketch is designed to encourage a sense of ownership and accomplishment in the user. To this end, all adjustments are performed in real-time, and the user ends up directly drawing all strokes on the canvas. The findings from our user study suggest that users prefer drawing with some automatic assistance, thereby producing better drawings, and that assistance does not decrease the perceived level of involvement in the creative process.

Despite increasing popularity of stereo capture and display systems, creating stereo artwork remains a challenge. This paper presents a stereo painting system, which enables effective from-scratch creation of high-quality stereo artwork. A key concept of our system is a stereo layer, which is composed of two RGBAd (RGBA + depth) buffers. Stereo layers alleviate the need for fully formed representational 3D geometry required by most existing 3D painting systems, and allow for simple, essential depth specification. RGBAd buffers also provide scalability for complex scenes by minimizing the dependency of stereo painting updates on the scene complexity. For interaction with stereo layers, we present stereo paint and stereo depth brushes, which manipulate the photometric (RGBA) and depth buffers of a stereo layer, respectively. In our system, painting and depth manipulation operations can be performed in arbitrary order with real-time visual feedback, providing a flexible WYSIWYG workflow for stereo painting. Our data structures allow for easy interoperability with existing image and geometry data, enabling a number of applications beyond from-scratch art creation, such as stereo conversion of monoscopic artwork and mixed-media art. Comments from artists and experimental results demonstrate that our system effectively aides in the creation of compelling stereo paintings.

We present the beginnings of a Cognitive Theory of Creativity Support aimed specifically at understanding novices and their needs. Our theory identifies unique difficulties novices face and reasons that may keep them from engaging in creative endeavors, such as fear of failure, time commitment, and lack of skill. To test our theory, we use it to analyze existing creativity support tools from multiple domains. We also describe the design and initial implementation of a creativity support tool based on our theory. The creativity support tool, called StorySketch, is designed to empower storytellers without graphical skills to engage in visual storytelling.

Despite increasing popularity of stereo capture and display systems, creating stereo artwork remains a challenge. This paper presents a stereo painting system, which enables effective from-scratch creation of high-quality stereo artwork. A key concept of our system is a stereo layer, which is composed of two RGBAd (RGBA + depth) buffers. Stereo layers alleviate the need for fully formed representational 3D geometry required by most existing 3D painting systems, and allow for simple, essential depth specification. RGBAd buffers also provide scalability for complex scenes by minimizing the dependency of stereo painting updates on the scene complexity. For interaction with stereo layers, we present stereo paint and stereo depth brushes, which manipulate the photometric (RGBA) and depth buffers of a stereo layer, respectively. In our system, painting and depth manipulation operations can be performed in arbitrary order with real-time visual feedback, providing a flexible WYSIWYG workflow for stereo painting. Comments from artists and experimental results demonstrate that our system effectively aides in the creation of compelling stereo paintings.

To create collections, like music playlists from personal media libraries, users today typically do one of two things. They either manually select items one-by-one, which can be time consuming, or they use an example-based recommendation system to automatically generate a collection. While such automatic engines are convenient, they offer the user limited control over how items are selected. Based on prior research and our own observations of existing practices, we propose a semi-automatic interface for creating collections that combines automatic suggestions with manual refinement tools. Our system includes a keyword query interface for specifying high-level collection preferences (e.g., "some rock, no Madonna, lots of U2,") as well as three example-based collection refinement techniques: 1) a suggestion widget for adding new items in-place in the context of the collection; 2) a mechanism for exploring alternatives for one or more collection items; and 3) a two-pane linked interface that helps users browse their libraries based on any selected collection item. We demonstrate our approach with two applications. SongSelect helps users create music playlists, and PhotoSelect helps users select photos for sharing. Initial user feedback is positive and confirms the need for semi-automated tools that give users control over automatically created collections.

Abstract We present a complete system for designing and manipulating regular or near‐regular textures in 2D images. We place emphasis on supporting creative workflows that produce artwork from scratch. As such, our system provides tools to create, arrange, and manipulate textures in images with intuitive controls, and without requiring 3D modeling. Additionally, we ensure continued, non‐destructive editability by expressing textures via a fully parametric descriptor. We demonstrate the suitability of our approach with numerous example images, created by an artist using our system, and we compare our proposed workflow with alternative 2D and 3D methods.

Abstract We describe a rendering system that converts a 3D meshed model into the stylized 2D filled‐region vector‐art commonly found in clip‐art libraries. To properly define filled regions, we analyze and combine accurate but jagged face‐normal contours with smooth but inaccurate interpolated vertex normal contours, and construct a new smooth shadow contour that properly surrounds the actual jagged shadow contour. We decompose region definition into geometric and topological components, using machine precision for geometry processing and raster‐precision to accelerate topological queries. We extend programmable stylization to simplify, smooth and stylize filled regions. The result renders 10K‐face meshes into custom clip‐art in seconds.

We describe a new vector-based primitive for creating smooth-shaded images, called the diffusion curve . A diffusion curve partitions the space through which it is drawn, defining different colors on either side. These colors may vary smoothly along the curve. In addition, the sharpness of the color transition from one side of the curve to the other can be controlled. Given a set of diffusion curves, the final image is constructed by solving a Poisson equation whose constraints are specified by the set of gradients across all diffusion curves. Like all vector-based primitives, diffusion curves conveniently support a variety of operations, including geometry-based editing, keyframe animation, and ready stylization. Moreover, their representation is compact and inherently resolution-independent. We describe a GPU-based implementation for rendering images defined by a set of diffusion curves in realtime. We then demonstrate an interactive drawing system for allowing artists to create artworks using diffusion curves, either by drawing the curves in a freehand style, or by tracing existing imagery. The system is simple and intuitive: we show results created by artists after just a few minutes of instruction. Furthermore, we describe a completely automatic conversion process for taking an image and turning it into a set of diffusion curves that closely approximate the original image content.

We present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an approximation to anisotropic diffusion, and artificially increase contrast in higher contrast regions with difference-of-Gaussian edges. The abstraction step is extensible and allows for artistic or data-driven control. Abstracted images can optionally be stylized using soft color quantization to create cartoon-like effects with good temporal coherence. Our framework design is highly parallel, allowing for a GPU-based, real-time implementation. We evaluate the effectiveness of our abstraction framework with a user-study and find that participants are faster at naming abstracted faces of known persons compared to photographs. Participants are also better at remembering abstracted images of arbitrary scenes in a memory task.

We present an algorithm to automatically estimate three-dimensional light positions from an unordered set of images. We collect images using a single stationary camera while manually moving a light source around an object. Rather than measuring light positions directly, the algorithm extracts a threedimensional manifold of positions from the images using dimensionality reduction techniques. This obviates the need for calibration and specialized equipment, making our approach inexpensive, portable and applicable to objects of almost any size. We demonstrate our results using image-based relighting applications.

We extend the standard solution to comic rendering with a comic-style specular component. To minimise the computational overhead associated with this extension, we introduce two optimising approximations; the perspective correction angle and the vertex face-orientation measure. Both of these optimisations are generally applicable, but they are especially well suited for applications where a physically correct lighting simulation is not required. Using our optimisations we achieve performances comparable to the standard solution. As our approximations favour large models, we even outperform the standard approach for models consisting of 10,000 triangles or more, which we can render exceeding 40 frames per second, including the specular component. Categories and Subject Descriptors (according to ACM CSS): I.3.3 [Computer Graphics] Picture/Image Generation: Display algorithms; I.3.7 [Computer Graphics] Three-Dimensional Graphics and Realism: Color, shading, shadowing, and texture; I.4.3 [Image Processing and Computer Vision] Enhancements: Geometric Correction.