A fully automatic method for descreening halftone images is presented based on convolutional neural networks with end-to-end learning. Incorporating context level information, the proposed method not only removes halftone artifacts but also synthesizes the fine details lost during halftone. The method consists of two main stages. In the first stage, intrinsic features of the scene are extracted, the low-frequency reconstruction of the image is estimated, and halftone patterns are removed. For the intrinsic features, the edges and object-categories are estimated and fed to the next stage as strong visual and contextual cues. In the second stage, fine details are synthesized on top of the low-frequency output based on an adversarial generative model. In addition, the novel problem of rescreening is addressed, where a natural input image is halftoned so as to be similar to a separately given reference halftone image. To this end, a two-stage convolutional neural network is also presented. Both networks are trained with millions of before-and-after example image pairs of various halftone styles. Qualitative and quantitative evaluations are provided, which demonstrates the effectiveness of the proposed methods.

Abstract The human shoulder complex is perhaps the most complicated joint in the human body being comprised of a set of three bones, muscles, tendons, and ligaments. Despite this anatomical complexity, computer graphics models for motion capture most often represent this joint as a simple ball and socket. In this paper, we present a method to determine a shoulder skeletal model that, when combined with standard skinning algorithms, generates a more visually pleasing animation that is a closer approximation to the actual skin deformations of the human body. We use a data‐driven approach and collect ground truth skin deformation data with an optical motion capture system with a large number of markers (200 markers on the shoulder complex alone). We cluster these markers during movement sequences and discover that adding one extra joint around the shoulder improves the resulting animation qualitatively and quantitatively yielding a marker set of approximately 70 markers for the complete skeleton. We demonstrate the effectiveness of our skeletal model by comparing it with ground truth data as well as with recorded video. We show its practicality by integrating it with the conventional rendering/animation pipeline.

Virtual characters are much in demand for animated movies, games, and other applications. Rapid advances in performance capture and advanced rendering techniques have allowed the movie industry in particular to create characters that appear very human-like. However, with these new capabilities has come the realization that such characters are yet not quite “right.” One possible hypothesis is that these virtual humans fall into an “Uncanny Valley”, where the viewer's emotional response is repulsion or rejection, rather than the empathy or emotional engagement that their creators had hoped for. To explore these issues, we created three animated vignettes of an arguing couple with detailed motion for the face, eyes, hair, and body. In a set of perceptual experiments, we explore the relative importance of different anomalies using two different methods: a questionnaire to determine the emotional response to the full-length vignettes, with and without facial motion and audio; and a 2AFC (two alternative forced choice) task to compare the performance of a virtual “actor” in short clips (extracts from the vignettes) depicting a range of different facial and body anomalies. We found that the facial anomalies are particularly salient, even when very significant body animation anomalies are present.

In this paper, we present a data-driven technique for synthesizing skin deformation from skeletal motion. We first create a database of dynamic skin deformations by recording the motion of the surface of the skin with a very large set of motion capture markers. We then build a statistical model of the deformations by dividing them into two parts: static and dynamic. Static deformations are modeled as a function of pose. Dynamic deformations are caused by the actions of the muscles as they move the joints and the inertia of muscles and fat. We approximate these effects by fitting a set of dynamic equations to the pre-recorded data. We demonstrate the viability of this approach by generating skin deformations from the skeletal motion of an actor. We compare the generated animation both to synchronized video of the actor and to ground truth animation created directly from the large marker set.

In this paper, we deal with the problem of synthesizing novel motions of standing-up martial arts such as Kickboxing, Karate, and Taekwondo performed by a pair of human-like characters while reflecting their interactions. Adopting an example-based paradigm, we address three non-trivial issues embedded in this problem: motion modeling, interaction modeling, and motion synthesis. For the first issue, we present a semi-automatic motion labeling scheme based on force-based motion segmentation and learning-based action classification. We also construct a pair of motion transition graphs each of which represents an individual motion stream. For the second issue, we propose a scheme for capturing the interactions between two players. A dynamic Bayesian network is adopted to build a motion transition model on top of the coupled motion transition graph that is constructed from an example motion stream. For the last issue, we provide a scheme for synthesizing a novel sequence of coupled motions, guided by the motion transition model. Although the focus of the present work is on martial arts, we believe that the framework of the proposed approach can be conveyed to other two-player motions as well.

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.

Abstract In this paper, we present an integrated framework of on‐line motion blending for locomotion generation. We first provide a novel scheme for incremental timewarping, which always guarantees that the time goes forward. Combining the idea of motion blending with that of posture rearrangement, we introduce a motion transition graph to address on‐line motion blending and transition simultaneously. Guided by a stream of motion specifications, our motion synthesis scheme moves from node to node in an on‐line manner while blending a motion at a node and generating a transition motion at an edge. For smooth on‐line motion transition, we also attach a set of example transition motions to an edge. To represent similar postures consistently, we exploit the inter‐frame coherency embedded in the input motion specification. Finally, we provide a comprehensive solution to on‐line motion retargeting by integrating existing techniques. Copyright © 2004 John Wiley & Sons, Ltd.

Real-time animation of human-like characters is an active research area in computer graphics. The conventional approaches have, however, hardly dealt with the rhythmic patterns of motions, which are essential in handling rhythmic motions such as dancing and locomotive motions. In this paper, we present a novel scheme for synthesizing a new motion from unlabelled example motions while preserving their rhythmic pattern. Our scheme first captures the motion beats from the example motions to extract the basic movements and their transitions. Based on those data, our scheme then constructs a movement transition graph that represents the example motions. Given an input sound signal, our scheme finally synthesizes a novel motion in an on-line manner while traversing the motion transition graph, which is synchronized with the input sound signal and also satisfies kinematic constraints given explicitly and implicitly. Through experiments, we have demonstrated that our scheme can effectively produce a variety of rhythmic motions.