Conventional coverage path planning algorithms are mainly based on the zigzag and spiral patterns or their combinations. The traversal order is limited by the linear or inside-outside manner. We propose a new set of coverage patterns induced from geometric folding operations, called the geometric folding pattern, to make coverage paths with more flexible traversal order. We study the modeling and parameterization of the geometric folding patterns. Then, a sampling operator is introduced. Based on the computational tools, we demonstrate the application of the proposed patterns in designing coverage paths. We show that the simple geometric folding patterns are flexible and controllable, which enables more choices for the coverage path planning problem.

Origami has provided a potential way to construct 3D curved structures by folding flat sheet materials without cutting or stretching. As a traditional origami, waterbomb tessellation is widely studied from aspects of science and engineering. However, users cannot easily utilize this kind of origami to fit curved target surfaces because the underlying geometric constraints limit the design space. In this study, we propose computational design methods for approximating cylindrical and axisymmetric curved surfaces based on waterbomb tessellations. With consideration of symmetry and periodic repetition, a single strip of the waterbomb tessellation is first modeled and then longitudinally and circumferentially replicated to construct cylindrical and axisymmetric waterbomb tessellations, respectively. To fulfill flat-foldability, an optimization process is introduced for minimizing flat-foldable residuals iteratively and then a regulation process of the crease pattern is applied for further reducing such residuals. In addition, we demonstrate waterbomb-derivative tessellations with quad-paddings to expand the design variations. Furthermore, rigid-folding sequences and several physically engineered origami pieces are presented. The proposed methods can be utilized to facilitate the design of origami-inspired structures for various engineering design purposes, such as foldable shelters, tubular structures, metamaterials, and so on.

Abstract Modeling 3D origami pieces using conventional software is laborious due to the geometric constraints imposed by the complicated layered structure. Targeting origami models used in visual content such as CG illustrations and movies, we propose an interactive system that dramatically simplifies the modeling of 3D origami pieces with plausible outer shapes, while omitting accurate inner structures. By focusing on flat origami models with a front‐and‐back symmetry commonly found in traditional artworks, our system realizes easy and quick modeling via single‐view interface; given a reference image of the target origami piece, the user draws polygons of planar faces onto the image, and assigns annotations indicating the types of folding operations. Our system automatically rectifies the manually‐specified polygons, infers the folded structures that should yield the user‐specified polygons with reference to the depth order of layered polygons, and generates a plausible 3D model while accounting for gaps between layers. Our system is versatile enough for modeling pseudo‐origami models that are not realizable by folding a single sheet of paper. Our user study demonstrates that even novice users without the specialized knowledge and experience on origami and 3D modeling can create plausible origami models quickly.

Parsing of human images is a fundamental task for determining semantic parts such as the face, arms, and legs, as well as a hat or a dress. Recent deep-learning-based methods have achieved significant improvements, but collecting training datasets with pixel-wise annotations is labor-intensive. In this paper, we propose two solutions to cope with limited datasets. Firstly, to handle various poses, we incorporate a pose estimation network into an end-to-end human-image parsing network, in order to transfer common features across the domains. The pose estimation network can be trained using rich datasets and can feed valuable features to the human-image parsing network. Secondly, to handle complicated backgrounds, we increase the variation in image backgrounds automatically by replacing the original backgrounds of human images with others obtained from large-scale scenery image datasets. Individually, each solution is versatile and beneficial to human-image parsing, while their combination yields further improvement. We demonstrate the effectiveness of our approach through comparisons and various applications such as garment recoloring, garment texture transfer, and visualization for fashion analysis.

Abstract This paper presents a method that can convert a given 3D mesh into a flat‐foldable model consisting of rigid panels. A previous work proposed a method to assist manual design of a single component of such flat‐foldable model, consisting of vertically‐connected side panels as well as horizontal top and bottom panels. Our method semi‐automatically generates a more complicated model that approximates the input mesh with multiple convex components. The user specifies the folding direction of each convex component and the fidelity of shape approximation. Given the user inputs, our method optimizes shapes and positions of panels of each convex component in order to make the whole model flat‐foldable. The user can check a folding animation of the output model. We demonstrate the effectiveness of our method by fabricating physical paper prototypes of flat‐foldable models.

Inferring a high dynamic range (HDR) image from a single low dynamic range (LDR) input is an ill-posed problem where we must compensate lost data caused by under-/over-exposure and color quantization. To tackle this, we propose the first deep-learning-based approach for fully automatic inference using convolutional neural networks. Because a naive way of directly inferring a 32-bit HDR image from an 8-bit LDR image is intractable due to the difficulty of training, we take an indirect approach; the key idea of our method is to synthesize LDR images taken with different exposures (i.e., bracketed images ) based on supervised learning, and then reconstruct an HDR image by merging them. By learning the relative changes of pixel values due to increased/decreased exposures using 3D deconvolutional networks, our method can reproduce not only natural tones without introducing visible noise but also the colors of saturated pixels. We demonstrate the effectiveness of our method by comparing our results not only with those of conventional methods but also with ground-truth HDR images.

Abstract Edit propagation is a technique that can propagate various image edits (e.g., colorization and recoloring) performed via user strokes to the entire image based on similarity of image features. In most previous work, users must manually determine the importance of each image feature (e.g., color, coordinates, and textures) in accordance with their needs and target images. We focus on representation learning that automatically learns feature representations only from user strokes in a single image instead of tuning existing features manually. To this end, this paper proposes an edit propagation method using a deep neural network (DNN). Our DNN, which consists of several layers such as convolutional layers and a feature combiner, extracts stroke‐adapted visual features and spatial features, and then adjusts the importance of them. We also develop a learning algorithm for our DNN that does not suffer from the vanishing gradient problem, and hence avoids falling into undesirable locally optimal solutions. We demonstrate that edit propagation with deep features, without manual feature tuning, can achieve better results than previous work.

Abstract This paper presents an efficient approach for generating weathering effects with detailed appearance variations in a single image. Previous approaches merely change chroma or reflectance of weathered objects, which is not sufficient for materials with detailed shading and texture variations, such as growing moss and peeling plaster. Our method propagates such detailed features via seamless patch‐based synthesis driven by weathering degree distribution. Unlike previous methods, the weathering degrees are calculated efficiently using Radial Basis Functions even for materials with wide color variations. We use graph cut‐based optimization to identify the most weathered region as a “weathering exemplar”, from which we sample weathering patches. We demonstrate our method enables us to generate various types of detailed weathering effects interactively.

Iris folding is an art-form consisting of layered strips of paper, forming a spiral pattern behind an aperture, which can be used to make cards and gift tags. This paper describes an interactive computational tool to assist in the design and construction of original iris folding patterns. The design of iris folding patterns is formulated as the calculation of a circumscribed polygonal sequence around a seed polygon. While it is possible to compute the positions of vertices analytically for a regular polygon, it is not straightforward to do so for irregular polygons. We give a numerical method for irregular polygons, which can be applied to arbitrary convex seed polygons. The user can quickly experiment with various patterns using the system prior to constructing the art-form.

Abstract Particle‐based liquid is often rendered only with single refraction in real‐time applications, which deteriorates the reality of liquid. We present a screen‐space method for rendering particle‐based liquids with up to four refractions in real time. Our method separates liquid particles into splashes and aggregations (i.e., liquid bodies) and generates a pair of depth maps of front‐facing and back‐facing surfaces for each. We use the depth maps of splashes as they are but smooth those of aggregations to reduce small bumps. For smoothing depth, we iteratively fit planes locally using moving least squares, unlike previous filtering‐based approaches that cause undesirable refractions around depth boundaries. By calculating refractions with physically based light attenuation, our method can render liquids more realistically than previous methods. Copyright © 2016 John Wiley & Sons, Ltd.

We propose an image-based virtual fitting system to reproduce the appearance of fitting during online shopping for garments. Inputs are whole-body images of a fashion model and the customer. We create a garment image by cutting out the garment portion from the image of the fashion model. If the garment images were naïvely superimposed, the fitting result would look strange mainly because the shape of the garment will not match the body shape of the customer. In this paper, we therefore propose a method of reshaping the garment image based on human body shapes of the fashion model and the customer to make the fitting result more realistic. The body shape is automatically estimated from the contours of the human body, and can easily be retouched if necessary. The fitting result is refined further by automatic color correction with reference to the facial regions and a method of retouching parts that protrude from the rear of the garment image. We verified the effectiveness of our system through a user test.

Abstract We propose an image editing system for repositioning objects in a single image based on the perspective of the scene. In our system, an input image is transformed into a layer structure that is composed of object layers and a background layer, and then the scene depth is computed from the ground region that is specified by the user using a simple boundary line. The object size and order of overlapping are automatically determined during the reposition based on the scene depth. In addition, our system enables the user to move shadows along with objects naturally by extracting the shadow mattes using only a few user‐specified scribbles. Finally, we demonstrate the versatility of our system through applications to depth‐of‐field effects, fog synthesis and 3D walkthrough in an image.

Abstract In this paper, we propose an interactive technique for constructing a 3D scene via sparse user inputs. We represent a 3D scene in the form of a Layered Depth Image (LDI) which is composed of a foreground layer and a background layer, and each layer has a corresponding texture and depth map. Given user‐specified sparse depth inputs, depth maps are computed based on superpixels using interpolation with geodesic‐distance weighting and an optimization framework. This computation is done immediately, which allows the user to edit the LDI interactively. Additionally, our technique automatically estimates depth and texture in occluded regions using the depth discontinuity. In our interface, the user paints strokes on the 3D model directly. The drawn strokes serve as 3D handles with which the user can pull out or push the 3D surface easily and intuitively with real‐time feedback. We show our technique enables efficient modeling of LDI that produce sufficient 3D effects.

The availability of low-cost digital fabrication devices enables new groups of users to participate in the design and fabrication of things. However, software to assist in the transition from design to actual fabrication is currently overlooked. In this paper, we introduce PacCAM, a system for packing 2D parts within a given source material for fabrication using 2D cutting machines. Our solution combines computer vision to capture the source material shape with a user interface that incorporates 2D rigid body simulation and snapping. A user study demonstrated that participants could make layouts faster with our system compared with using traditional drafting tools. PacCAM caters to a variety of 2D fabrication applications and can contribute to the reduction of material waste.

Abstract We introduce soft folding, a new interactive method for designing and exploring thin‐plate forms. A user specifies sharp and soft folds as two‐dimensional(2D) curves on a flat sheet, along with the fold magnitude and sharpness of each. Then, based on the soft folds, the system computes the three‐dimensional(3D) folded shape. Internally, the system first computes a fold field, which defines local folding operations on a flat sheet. A fold field is a generalization of a discrete fold graph in origami, replacing a graph with sharp folds with a continuous field with soft folds. Next, local patches are folded independently according to the fold field. Finally, a globally folded 3D shape is obtained by assembling the locally folded patches. This algorithm computes an approximation of 3D developable surfaces with user‐defined soft folds at an interactive speed. The user can later apply nonlinear physical simulation to generate more realistic results. Experimental results demonstrated that soft folding is effective for producing complex folded shapes with controllable sharpness.

Existing 3D sketching methods typically allow the user to draw in empty space which is imprecise and lacks tactile feedback. We introduce a shape-stamping interface where users can model with tangible 3D primitive shapes. Each of these shapes represents a copy or a fragment of the construction material. Instead of modeling in empty space, these shapes allow us to use the real-world environment and other existing objects as a tangible guide during 3D modeling. We call this approach Situated Modeling: users can create new real-sized 3D objects directly in 3D space while using the nearby existing objects as the ultimate reference. We also describe a two-handed shape-stamping technique for stamping with tactile feedback. We show a variety of doit-yourself furniture and household products designed with our system, and perform a user study to compare our method with a related AR-based modeling system.

This paper introduces DressUp, a computerized system for designing dresses with 3D input using the form of the human body as a guide. It consists of a body-sized physical mannequin, a screen, and tangible prop tools for drawing in 3D on and around the mannequin. As the user draws, he/she modifies or creates pieces of digital cloth, which are displayed on a model of the mannequin on the screen. We explore the capacity of our 3D input tools to create a variety of dresses. We also describe observations gained from users designing actual physical garments with the system.

Abstract We propose 2D stick figures as a unified medium for visualizing and searching for human motion data. The stick figures can express a wide range or human motion, and they are easy to be drawn by people without any professional training. In our interface, the user can browse overall motion by viewing the stick figure images generated from the database and retrieve them directly by using sketched stick figures as an input query. We started with a preliminary survey to observe how people draw stick figures. Based on the rules observed from the user study, we developed an algorithm converting motion data to a sequence of stick figures. The feature‐based comparison method between the stick figures provides an interactive and progressive search for the users. They assist the user's sketching by showing the current retrieval result at each stroke. We demonstrate the utility of the system with a user study, in which the participants retrieved example motion segments from the database with 102 motion files by using our interface.

Abstract This paper introduces a framework that can extract an alpha matte from a single image with Fresnel reflection, and that can composite other objects with the image such that plausible reflections are included. Our method handles reflections in a plane with small undulations, for example, a water surface with waves or a glossy tabletop. During the matting stage, our method first estimates the transmission color, which is assumed to be uniform, and then calculates a reflection image and alpha matte based on user markups. However, accurate extraction of the matte becomes challenging when a plane has small undulations because these create perturbations in the matte. We therefore propose a filter that can refine the matte effectively. In the compositing stage, the reflection of a composited object is synthesized by ray tracing in real time. We demonstrate the effectiveness of our method through comparisons with ground‐truth data and results using natural images as inputs.

We introduce the interactive system "Beady" to assist the design and construction of customized 3D beadwork. The user first creates a polygonal mesh model called the design model that represents the overall structure of the beadwork. Each edge of the mesh model corresponds to a bead in the beadwork. We provide two methods to create the design model. One is interactive modeling from scratch. The user defines the mesh topology with gestural interaction and the system continuously adjusts edge lengths by considering the physical constraints among neighboring beads. The other is automatic conversion that takes an existing polygonal model as input and generates a near-hexagonal mesh model with a near-uniform edge length as output. The system then converts the design model into a beadwork model with the appropriate wiring. Computation of an appropriate wiring path requires careful consideration, and we present an algorithm based on face stripification of the mesh. The system also provides a visual step-by-step guide to assist the manual beadwork construction process. We show several beadwork designs constructed by the authors and by test users using the system.

Character pose design is one of the most fundamental processes in computer graphics authoring. Although there are many research efforts in this field, most existing design tools consider only character body structure, rather than its interaction with the environment. This paper presents an intuitive sketching interface that allows the user to interactively place a 3D human character in a sitting position on a chair. Within our framework, the user sketches the target pose as a 2D stick figure and attaches the selected joints to the environment (e.g., the feet on the ground) with a pin tool. As reconstructing the 3D pose from a 2D stick figure is an ill-posed problem due to many possible solutions, the key idea in our paper is to reduce solution space by considering the interaction between the character and environment and adding physics constraints, such as balance and collision. Further, we formulated this reconstruction into a nonlinear optimization problem and solved it via the genetic algorithm (GA) and the quasi-Newton solver. With the GPU implementation, our system is able to generate the physically correct and visually pleasing pose at an interactive speed. The promising experimental results and user study demonstrates the efficacy of our method.

Object delivery by pushing objects with mobile robots on a flat surface has been successfully demonstrated. However, existing methods can push objects that have a circular or rectangular shape. In this paper, we introduce a learning-based approach for pushing objects of any irregular shape to user-specified goal locations. We first automatically collect a set of data on how an irregular-shaped object moves given the robot's relative position and pushing direction. We collect this data with a randomized approach, and we demonstrate that this approach can successfully collect useful data. Object delivery is achieved by using the collected data with a non-parametric regression method. We demonstrate our approach with a number of irregular-shaped objects.

Beadwork is the art of connecting beads together by wires. While common beadwork is two-dimensional (2D), three-dimensional (3D) beadwork is also popular in oriental regions such as Japan and China. However, the design and construction of 3D beadwork is very difficult. The final shape is defined by the complicated three-dimensional interaction between beads and wires, thus making it very difficult to design manually. One also needs to specify an appropriate wire path to hold the beads together and to manually insert the wire into the beads one by one following the path to construct the beadwork. Careful observation of existing beadwork structures shows several geometrically interesting problems, which make beadwork design an interesting technical challenge.

Route setting is the process of creating climbing routes for the sport of free climbing. By fastening climbing holds to artificial climbing walls, a route emerges that provides a challenge for climbers. It is a difficult process usually requiring either a very skilled climber or a lot of time and test climbs. We propose a climbing route designer that aims to enable even novice climbers to create interesting and challenging routes by designing them on the computer using a simulated climber to analyze the route and visualize the result.

SketchChair is an application that allows novice users to control the entire process of designing and building their own chairs. Chairs are designed using a simple 2D sketch-based interface and design validation tools, and are then fabricated from sheet materials, cut by a laser cutter or CNC milling machine. This paper presents the concepts and details of SketchChair, and both miniature and full-sized chairs are designed using the application. We conclude with results and insights from a workshop in which novice users designed their own model chairs.

SketchChair is an application that allows novice users to take part in the entire process of designing and fabricating their own full-sized usable chairs from scratch. During the studio, participants will be able to design, build and take home their own scale model paper chair. Participants will be introduced to SketchChair whilst learning how to use a paper cutting plotter to make slice form models. Furthermore we will discuss the considerations around designing tools for customization and CNC production.

Although there is an abundance of 3D models available, most of them exist only in virtual simulation and are not immediately usable as physical objects in the real world. We solve the problem of taking as input a 3D model of a man-made object, and automatically generating the parts and connectors needed to build the corresponding physical object. We focus on furniture models, and we define formal grammars for IKEA cabinets and tables. We perform lexical analysis to identify the primitive parts of the 3D model. Structural analysis then gives structural information to these parts, and generates the connectors (i.e. nails, screws) needed to attach the parts together. We demonstrate our approach with arbitrary 3D models of cabinets and tables available online.

Spatial Sketch is a three-dimensional (3D) sketch application that bridges between physical movement and the fabrication of objects in the real world via cut planar materials. This paper explores the rationale and details behind the development of the Spatial Sketch application, and presents our observations from user testing and a hands-on lamp shade design workshop. Finally we reflect upon the relevance of embodied forms of human computer interaction for use in digital fabrication.

We introduce layer operations for single-view 3D deformable object manipulation, in which the user can control the depth order of layered 3D objects resting on a flat ground with simple clicks and drags, as in 2D drawing systems. We present two interaction techniques based on this idea and describe their implementation. The first technique is explicit layer swap. The user clicks the target layer, and the system swaps the layer with the one directly underneath it. The second technique is layer-aware dragging. As the user drags the object, the system adjusts its depth automatically to pass over or under a colliding object in the screen space, according to user control. Although the user interface is 2.5D, all scene representations are true 3D, and thus the system naturally supports local layering, self-occlusions, and folds. Internally, the system dynamically computes the apparent layer structure in the current configuration and makes appropriate depth adjustments to obtain the desired results. We demonstrate the effectiveness of this approach in cloth and rope manipulation systems.

An Origamic Architecture (OA) is a folded sheet of perforated paper from which a three-dimensional structure "pops up" when it is opened. It is similar to a "pop-up story book", but its unique feature is that it is made purely by cutting a single piece of paper. Because of this limitation, designing an OA requires considerable experience. We propose a computerised method which assists design of OAs. An OA is modelled using a set of planar polygons. This model must satisfy the conditions required of a valid, realisable OA. A unique point of our method is the application of Boolean set operations to the polygons on the unfolded pattern to guarantee that the model can be made from a single sheet of paper. We also present a procedure for checking the model's validity. Additionally, we propose methods for creating openings, for generating unfolded patterns, and for displaying folding animation. We have implemented a system based on these methods and demonstrated its usefulness for creating OA. Our system allows designers to intuitively design OA models and to easily generate the unfolded patterns.

We propose a new method for producing unfolded papercraft patterns of rounded toy animal figures from triangulated meshes by means of strip-based approximation. Although in principle a triangulated model can be unfolded simply by retaining as much as possible of its connectivity while checking for intersecting triangles in the unfolded plane, creating a pattern with tens of thousands of triangles is unrealistic. Our approach is to approximate the mesh model by a set of continuous triangle strips with no internal vertices. Initially, we subdivide our mesh into parts corresponding to the features of the model. We segment each part into zonal regions, grouping triangles which are similar topological distances from the part boundary. We generate triangle strips by simplifying the mesh while retaining the borders of the zonal regions and additional cut-lines. The pattern is then created simply by unfolding the set of strips. The distinguishing feature of our method is that we approximate a mesh model by a set of continuous strips, not by other ruled surfaces such as parts of cones or cylinders. Thus, the approximated unfolded pattern can be generated using only mesh operations and a simple unfolding algorithm. Furthermore, a set of strips can be crafted just by bending the paper (without breaking edges) and can represent smooth features of the original mesh models.