Abstract This paper proposes t‐Regina, a novel iterative scheme that automates the manipulation of grid handles in free‐form deformation (FFD) for deformable registrations of cartoon character sprites. First, we build a data‐driven FFD (dFFD) that enables users to handle locations of FFD handles from lower‐dimensional parameters. To prepare training data, we implement an FFD‐based puppet tool and recorded the user‐designed locations of grid handles. Second, we iteratively optimize the parameters of dFFD by using a locally optimal block matching algorithm with almost‐uniform sampling strategies. t‐Regina is effective and easy to integrate into existing drawing systems. This paper shows some examples of deformation results to demonstrate the robustness of t‐Regina.

Abstract This paper proposes a method to estimate the locations of grid handles in free-form deformation (FFD) while preserving the local shape characteristics of the 2D/3D input model embedded into the grid, named locality-preserving FFD (lp-FFD). Users first specify some vertex locations in the input model and grid handle locations. The system then optimizes all locations of grid handles by minimizing the distortion of the input model’s mesh elements. The proposed method is fast and stable, allowing the user to directly and indirectly make the deformed shape of the mesh model and grid. This paper shows some examples of deformation results to demonstrate the robustness of our lp-FFD. In addition, we conducted a user study and confirm our lp-FFD’s efficiency and effectiveness in shape deformation is higher than those of existing methods used in commercial software.

In deep learning application development, programmers tend to try different architectures and hyper-parameters until satisfied with the model performance. Nevertheless, program crashes due to tensor shape mismatch prohibit programmers, especially novice programmers, from smoothly going back and forth between neural network (NN) architecture editing and experimentation. We propose to leverage live programming techniques in NN architecture editing with an always-on visualization. When the user edits the program, the visualization can synchronously display tensor states and provide a warning message by continuously executing the program to prevent program crashes during experimentation. We implement the live visualization and integrate it into an IDE called ODEN that seamlessly supports the “edit→experiment→edit→···” repetition. With ODEN, the user can construct the neural network with the live visualization and transits into experimentation to instantly train and test the NN architecture. An exploratory user study is conducted to evaluate the usability, the limitations, and the potential of live visualization in ODEN.

Special skills are required in portrait painting, such as imagining geometric structures and facial detail for final portrait designs. This makes it a difficult task for users, especially novices without prior artistic training, to draw freehand portraits with high-quality details. In this paper, we propose dualFace, a portrait drawing interface to assist users with different levels of drawing skills to complete recognizable and authentic face sketches. Inspired by traditional artist workflows for portrait drawing, dualFace gives two-stages of drawing assistance to provide global and local visual guidance. The former helps users draw contour lines for portraits (i.e., geometric structure), and the latter helps users draw details of facial parts, which conform to the user-drawn contour lines. In the global guidance stage, the user draws several contour lines, and dualFace then searches for several relevant images from an internal database and displays the suggested face contour lines on the background of the canvas. In the local guidance stage, we synthesize detailed portrait images with a deep generative model from user-drawn contour lines, and then use the synthesized results as detailed drawing guidance. We conducted a user study to verify the effectiveness of dualFace, which confirms that dualFace significantly helps users to produce a detailed portrait sketch.

The cover image is based on the Research Article Interactive texture editing for garment line drawings by Tsukasa Fukusato et al., https://doi.org/10.1002/cav.2117

Abstract Adding two‐dimensional (2D) textures to garment line drawings (e.g., cartoon characters) remains challenging in the production pipeline of comics and illustrations since garment line drawings often have self‐occluded wrinkles. Although several techniques that can automatically deform and map 2D texture patterns to 2D line drawings have been proposed, their qualities are insufficient for representing 3D‐like realistic garment designs and manual editing of UV coordinates, which is labor‐intensive. In this article, we introduce an interactive tool to efficiently edit UV coordinates of 2D garment line drawings on the modeling panel with curve and point handles. Our algorithm is simple to integrate into existing image authoring tools. We conduct a user study with novice users and confirm that the proposed tool can effectively handle texture mapping envisioned by the users.

We introduce an interactive method to sequentially find viewpoints for constructing view-dependent models which represent view-specific deformations in classic 2D cartoons [Chaudhuri et al. 2004, 2007; Koyama and Igarashi 2013; Rademacher 1999]. As users design one view-specific model from a single-fixed viewpoint, the system searches successive viewpoints for subsequent modeling and instantly jumps to the next viewpoints. Thereby, the users can efficiently repeat the design process of view-specific deformations until they are satisfied. This method is simple enough to easily implement in an existing modeling system. We conduct a user study with novice and amateur users and confirm that the proposed system is effective for designing view-specific models envisioned by the users.

In this paper, we propose a user interface that enables users to intuitively retrieve relevant motions from a database and edit them by drawing motion trajectories on the screen. This system consists of two-stage operations to provide global-level and local-level motion editing: a global stage that enables users to design the body movement in virtual space roughly, and a local stage that enables users to design detailed movements such as limbs movement. We verified the proposed system with character animation editing with both global and local stages.

Writing high-performance image processing code is challenging and labor-intensive. To address this, we propose a programming support method called “guided optimization.” Guided optimization provides programmers a set of valid optimization options and interactive feedback about their current choices, which enables them to comprehend and efficiently optimize image processing code without the time-consuming trial-and-error process of traditional text editors. We implemented a proof-of-concept system, Roly-poly, which integrates guided optimization, program visualization, and schedule cost estimation to support the comprehension and development of efficient image processing code. We conducted a user study with novice Halide programmers and confirmed that Roly-poly and its guided optimization was informative, increased productivity, and resulted in higher-performing schedules in less time.

The development of a statistical modeling program requires example data to observe and verify the behavior of the program. Such example data are either taken from an existing dataset or synthesized using commands. Programmers may want to directly design an arbitrary dataset or modify it interactively, but it is difficult to do so in current development environments. We therefore propose combining a code editor with an interactive scatter plot editor to efficiently understand the behavior of statistical modeling algorithms. The user interactively creates and modifies the dataset on the scatter plot editor, while the system continuously executes the code in the editor, taking the data as input, and shows the result in the editor. This paper presents the design rationale behind the system and introduces several usage scenarios.

Game designers take into account the wide range of play-styles and skill levels of players to create enjoyable experiences. One important step in the game design process involves playtests with professional testers; this process is time-consuming and expensive. Hence, there exist several methods to create synthetic testers to test a game automatically. However, one shortcoming is the lack of realistic-playing with different play-styles and skill levels. In this paper, we propose a game level authoring tool that incorporates synthetic testers, which enable the control of play-styles and skill levels. Furthermore, we utilize visualization techniques to help assess the difficulty level of each part of the stage. Our user studies confirmed that our tool was effective for designing game stages appropriate for a particular type of player.

The cover image is based on the Special Issue Paper Sketch2VF: Sketch-based Flow Design with Conditional Generative Adversarial Network by Xie Haoran et al., https://doi.org/10.1002/cav.1889.

Abstract We present an interactive user interface to support sketch‐based fluid design with a perceptual understanding of human sketches. In particular, the proposed system generates a 2D fluid animation from hand‐drawn sketches. The proposed system utilizes a conditional generative adversarial network model to generate stationary velocity fields from a sketch input. The network model is trained with hand‐drawn strokes and corresponding 2D velocity fields. On the basis of the generated velocity field, the system calculates fluid dynamics using a semi‐Lagrangian method. We ran a user study of the proposed system and confirmed that the proposed interface is effective for a 2D fluid design and that the system achieves good results based on user input.

This paper presents an interactive face retrieval framework for clarifying an image representation envisioned by a user. Our system is designed for a situation in which the user wishes to find a person but has only visual memory of the person. We address a critical challenge of image retrieval across the user's inputs. Instead of target-specific information, the user can select several images (or a single image) that are similar to an impression of the target person the user wishes to search for. Based on the user's selection, our proposed system automatically updates a deep convolutional neural network. By interactively repeating these process (human-in-the-loop optimization), the system can reduce the gap between human-based similarities and computer-based similarities and estimate the target image representation. We ran user studies with 10 subjects on a public database and confirmed that the proposed framework is effective for clarifying the image representation envisioned by the user easily and quickly.

Accurately tracking a Head Mounted Display (HMD) with a 6 degree of freedom is essential to achieve a comfortable and a nausea free experience in Virtual Reality. Existing commercial HMD systems using synchronized Infrared (IR) camera and blinking IR-LEDs can achieve highly accurate tracking. However, most of the off-the-shelf cameras do not support frame synchronization. In this paper, we propose a novel method for real time HMD pose estimation without using any camera synchronization or LED blinking. We extended over the state of the art pose estimation algorithm by introducing geometrically constrained optimization. In addition, we propose a novel system to increase robustness to the blurred IR-LEDs patterns appearing at high-velocity movements. The quantitative evaluations showed significant improvements in pose stability and accuracy over wide rotational movements as well as a decrease in runtime.

This paper presents a method that enables the estimation and depiction of onomatopoeia in computer-generated animation based on physical parameters. Onomatopoeia is used to enhance physical characteristics and movement, and enables users to understand animation more intuitively. We experiment with onomatopoeia depiction in scenes within the animation process. To quantify onomatopoeia, we employ Komatsu's [2012] assumption, i.e., onomatopoeia can be expressed by n-dimensional vector. We also propose phonetic symbol vectors based on the correspondence of phonetic symbols to the impressions of onomatopoeia using a questionnaire-based investigation. Furthermore, we verify the positioning of onomatopoeia in animated scenes. The algorithms directly combine phonetic symbols to estimate optimum onomatopoeia. They use a view-dependent Gaussian function to display onomatopoeias in animated scenes. Our method successfully recommends optimum onomatopoeias using only physical parameters, so that even amateur animators can easily create onomatopoeia animation.