It has been demonstrated that the blurring process reduces the high-frequency information of the original sharp image, so the main challenge for image deblurring is to reconstruct high-frequency information from the blurry image. In this paper, we propose a novel image deblurring framework to focus on the reconstruction of high-frequency information, which consists of two main subnetworks: a high-frequency reconstruction subnetwork (HFRSN) and a multi-scale grid subnetwork (MSGSN). The HFRSN is built to reconstruct latent high-frequency information from multiple scale blurry images. The MSGSN performs deblurring processes with high-frequency guidance at different scales simultaneously. Besides, in order to better use high-frequency information to restore sharpening images, we designed a high-frequency information aggregation (HFAG) module and a high-frequency information attention (HFAT) module in MSGSN. The HFAG module is designed to fuse high-frequency features and image features at the feature extraction stage, and the HFAT module is built to enhance the feature reconstruction stage. Extensive experiments on different datasets show the effectiveness and efficiency of our method.

Exemplar-based colorization aims to add colors to a grayscale image guided by a content related reference image. Existing methods are either sensitive to the selection of reference images (content, position) or extremely time and resource consuming, which limits their practical application. To tackle these problems, we propose a deep exemplar colorization architecture inspired by the characteristics of stylization in feature extracting and blending. Our coarse- to-fine architecture consists of two parts: a fast transfer sub-net and a robust colorization sub-net. The transfer sub- net obtains a coarse chrominance map via matching basic feature statistics of the input pairs in a progressive way. The colorization sub-net refines the map to generate the final results. The proposed end-to-end network can jointly learn faithful colorization with a related reference and plausible color prediction with unrelated reference. Extensive experimental validation demonstrates that our approach outperforms the state-of-the-art methods in less time whether in exemplar-based colorization or image stylization tasks.

Abstract Traditional pencil drawing rendering algorithms when applied to video may suffer from temporal inconsistency and shower‐door effect due to the stochastic noise models employed. This paper attempts to resolve these problems with deep learning. Recently, many research endeavors have demonstrated that feed‐forward Convolutional Neural Networks (CNNs) are capable of using a reference image to stylize a whole video sequence while removing the shower‐door effect in video style transfer applications. Compared with video style transfer, pencil drawing video is more sensitive to the inconsistency of texture and requires a stronger expression of pencil hatching. Thus, in this paper we develop an approach by combining a latest Line Integral Convolution (LIC) based method, specializing in realistically simulating pencil drawing images, with a new feed‐forward CNN that can eliminate the shower‐door effect successfully. Taking advantage of optical flow, we adopt a feature‐map‐level temporal loss function and propose a new framework to avoid the temporal inconsistency between consecutive frames, enhancing the visual impression of pencil strokes and tone. Experimental comparisons with the existing feed‐forward CNNs have demonstrated that our method can generate temporally more stable and visually more pleasant pencil drawing video results in a faster manner.

Line Integral Convolution (LIC) has been widely adopted in pencil hatching generation, where an image degraded by random binary white noise (RBWN) is filtered by LIC along a priori vector field. Nonetheless, an RBWN degraded image through LIC produces hatching graduation only in terms of stroke intensity, and can neither produce hatching graduation explicitly in stroke density nor create visually clear hatching strokes while input pixel values are fairly low. In this paper we address these issues from a noise point of view by assessing several noise models and subsequently constructing a new noise model, called hybrid noise. The new noise model has been experimentally demonstrated in simulating hatching graduation in terms of both stroke intensity and stroke density with quantified graduality measurements. To oversee the effectiveness of hybrid noise we implement the whole pipeline of pencil drawing simulation and compare our results with those state of the art algorithms.

Abstract In this paper, we extend the use of the partial differential equation (PDE) method to head visualization with computed tomography (CT) data and show how the two primary medical visualization means, surface reconstruction, and volume rendering can be integrated into one single framework through PDEs. Our scheme first performs head segmentation from CT slices using a variational approach, the output of which can be readily used for extraction of a small set of PDE boundary conditions. With the extracted boundary conditions, head surface reconstruction is then executed. Because only a few slices are used, our method can perform head surface reconstruction more efficiently in both computational time and storage cost than the widely used marching cubes algorithm. By elaborately introducing a third parameterwto the PDE method, a solid head can be created, based on which the head volume is subsequently rendered with 3D texture mapping. Instead of designing a transfer function, we associate the alpha value of texels of the 3D texture with the PDE parameterwthrough a linear transform. This association enables the production of a visually translucent head volume. The experimental results demonstrate the feasibility of the developed head visualization method. Copyright © 2015 John Wiley & Sons, Ltd.

Abstract Embroidery is a traditional handicraft of sewing stitches into fabric or other materials in different patterns, and this ancient non‐photorealistic art form has not drawn enough attention thus far. In this paper, we present an image‐based method to simulate the traditional embroidery art. The method combines stroke‐based rendering techniques with the Phong lighting model to create picturesque embroidery‐like images. We first build a 3D stitch model and derive some most commonly used stitch patterns from it. Then we preprocess the input image by segmenting it into regions, from which the parameters to specify stitch patterns are obtained. Finally, we apply stitches back onto the desired regions and render them under a virtual light source. Experimental results show that our method, different from the existing schemes, is capable of performing fine embroidery simulations with the effects of lighting and shading based on an input image. Copyright © 2016 John Wiley & Sons, Ltd.

Both prominent feature points and geodesic distance are key factors for mesh segmentation. With these two factors, this paper proposes a graph cut based mesh segmentation method. The mesh is first preprocessed by Laplacian smoothing. According to the Gaussian curvature, candidate feature points are then selected by a predefined threshold. With DBSCAN (Density-Based Spatial Clustering of Application with Noise), the selected candidate points are separated into some clusters, and the points with the maximum curvature in every cluster are regarded as the final feature points. We label these feature points, and regard the faces in the mesh as nodes for graph cut. Our energy function is constructed by utilizing the ratio between the geodesic distance and the Euclidean distance of vertex pairs of the mesh. The final segmentation result is obtained by minimizing the energy function using graph cut. The proposed algorithm is pose-invariant and can robustly segment the mesh into different parts in line with the selected feature points.

Instant Messengers (IM) have become popular online chat tools in cyber worlds. The existing IMs mainly rely on text, audio, and video for user conversation and communication. The pure text chat is monotonous, while the video chat only occurs when users have their webcoms installed and wish to see each other. In this paper, we propose a new IM with personalized 3D Avatars, and present a prototype of this system. Through our IM, the user can synthesize a personalized 3D avatar by just inputting a 2D self face image. Expressions of the avatar are synthesized by a group of predefined phonemes and visemes, and driven by a text-to-visual speech engine. Moreover, our system supports 3D avatar decoration. During online chat, text is input through the text-to-visual speech engine to drive the generation of voice and animation for the 3D avatar. Tests have demonstrated that our IM with personalized 3D avatars can enhance the fun and vividness of online chat experience.

Various Partial Differential Equations (PDE) have been used in computer graphics for approximating surfaces of geometric shapes by finding solutions to PDEs subject to suitable boundary conditions. The PDE boundary conditions are defined as 3D curves on the surface of the shapes. We propose how to automatically derive these curves as boundaries of curved patches on the surface of the original polygon mesh. The analytic solution to the PDE used throughout this work is fully determined by finding a set of coefficients associated with parametric functions according to the particular set of boundary conditions. The PDE coefficients require an order of magnitude smaller space compared to the original polygon data and can be interactively rendered with different level of detail. It allows for an efficient exchange of the PDE shapes in 3D Cyber worlds and their web visualization. In this paper we analyze and formulate the requirements for extracting suitable boundary conditions, describe the algorithm for the automatic deriving of the boundary curves, and present its implementation as a part of the function-based extension of VRML and X3D.

Various 3-D face synthesis techniques have been proposed and extensively used in many applications. Compared with others, single view-based face synthesis technology allows unsupervised 3-D face reconstruction without any offline operations. Although many algorithms have been published, automatic and robust single view-based 3-D face synthesis still remains unsolved. In contrast to other methods, the single view-based 3-D face synthesis algorithm conducted in this paper enables automated 3-D face synthesis from an arbitrary head-and-shoulder image with the complex background. The developed system first detects the face using Bayesian skin-tone classification based on only the chrominance component, Cr. Based on the detected face, a few salient facial features, such as the corners of the eyebrows and contours of the eyes, mouth, and chin are in turn extracted using variant algorithms, including a dynamic chin extraction mechanism that will be detailed in this paper. Then, face model adaptation consisting of both global and local adaptations is imposed, according to geometric information provided by the extracted facial features. Finally, the 3-D specific face is synthesized using the adapted 3-D face model with a texture map directly derived from the input face image, followed by the implementation of facial animation using this synthesized face.