Automatic generation of Chinese fonts that consist of large numbers of glyphs with complicated structures is now still a challenging and ongoing problem in areas of AI and Computer Graphics (CG). Traditional CG-based methods typically rely heavily on manual interventions, while recentlypopularized deep learning-based end-to-end approaches often obtain synthesis results with incorrect structures and/or serious artifacts. To address those problems, this paper proposes a structure-guided Chinese font generation system, SCFont, by using deep stacked networks. The key idea is to integrate the domain knowledge of Chinese characters with deep generative networks to ensure that high-quality glyphs with correct structures can be synthesized. More specifically, we first apply a CNN model to learn how to transfer the writing trajectories with separated strokes in the reference font style into those in the target style. Then, we train another CNN model learning how to recover shape details on the contour for synthesized writing trajectories. Experimental results validate the superiority of the proposed SCFont compared to the state of the art in both visual and quantitative assessments.

In this paper, we present a novel approach for dynamic text effects transfer by using example-based texture synthesis. In contrast to previous works that require an input video of the target to provide motion guidance, we aim to animate a still image of the target text by transferring the desired dynamic effects from an observed exemplar. Due to the simplicity of target guidance and complexity of realistic effects, it is prone to producing temporal artifacts such as flickers and pulsations. To address the problem, our core idea is to find a common Nearest-neighbor Field (NNF) that would optimize the textural coherence across all keyframes simultaneously. With the static NNF for video sequences, we implicitly transfer motion properties from source to target. We also introduce a guided NNF search by employing the distance-based weight map and Simulated Annealing (SA) for deep direction-guided propagation to allow intense dynamic effects to be completely transferred with no semantic guidance provided. Experimental results demonstrate the effectiveness and superiority of our method in dynamic text effects transfer through extensive comparisons with state-of-the-art algorithms. We also show the potentiality of our method via multiple experiments for various application domains.

Abstract Despite the recent impressive development of deep neural networks, using deep learning based methods to generate large‐scale Chinese fonts is still a rather challenging task due to the huge number of intricate Chinese glyphs, e.g., the official standard Chinese charset GB18030‐2000 consists of 27,533 Chinese characters. Until now, most existing models for this task adopt Convolutional Neural Networks (CNNs) to generate bitmap images of Chinese characters due to CNN based models' remarkable success in various applications. However, CNN based models focus more on image‐level features while usually ignore stroke order information when writing characters. Instead, we treat Chinese characters as sequences of points (i.e., writing trajectories) and propose to handle this task via an effective Recurrent Neural Network (RNN) model with monotonic attention mechanism, which can learn from as few as hundreds of training samples and then synthesize glyphs for remaining thousands of characters in the same style. Experimental results show that our proposed FontRNN can be used for synthesizing large‐scale Chinese fonts as well as generating realistic Chinese handwritings efficiently.

Automatic generation of artistic glyph images is a challenging task that attracts many research interests. Previous methods either are specifically designed for shape synthesis or focus on texture transfer. In this paper, we propose a novel model, AGIS-Net, to transfer both shape and texture styles in one-stage with only a few stylized samples. To achieve this goal, we first disentangle the representations for content and style by using two encoders, ensuring the multi-content and multi-style generation. Then we utilize two collaboratively working decoders to generate the glyph shape image and its texture image simultaneously. In addition, we introduce a local texture refinement loss to further improve the quality of the synthesized textures. In this manner, our one-stage model is much more efficient and effective than other multi-stage stacked methods. We also propose a large-scale dataset with Chinese glyph images in various shape and texture styles, rendered from 35 professional-designed artistic fonts with 7,326 characters and 2,460 synthetic artistic fonts with 639 characters, to validate the effectiveness and extendability of our method. Extensive experiments on both English and Chinese artistic glyph image datasets demonstrate the superiority of our model in generating high-quality stylized glyph images against other state-of-the-art methods.

In this paper, we present a general-purpose solution to interactive texture transfer problems that better preserves both local structure and visual richness. It is challenging due to the diversity of tasks and the simplicity of required user guidance. The core idea of our common framework is to use multiple custom channels to dynamically guide the synthesis process. For interactivity, users can control the spatial distribution of stylized textures via semantic channels. The structure guidance, acquired by two stages of automatic extraction and propagation of structure information, provides a prior for initialization and preserves the salient structure by searching the nearest neighbor fields (NNF) with structure coherence. Meanwhile, texture coherence is also exploited to maintain similar style with the source image. In addition, we leverage an improved PatchMatch with extended NNF and matrix operations to obtain transformable source patches with richer geometric information at high speed. We demonstrate the effectiveness and superiority of our method on a variety of scenes through extensive comparisons with state-of-the-art algorithms.

Font recognition on a single Chinese character is a challenging task especially when the identity of the character is unknown and the number of possible font types is huge. In this paper, we propose a novel method using multi-scale sparse representation to solve the problem of large-scale font recognition on a single unknown Chinese character. Specifically, we first apply a saliency-based sampling approach, which exploits the saliency information of character contours, to segment local patches in multiple scales from salient regions. Then, corresponding local descriptors are extracted by implementing Sobel and Prewitt operators in 4 directions. After encoding the local descriptors into sparse codes, max pooling and spatial pyramid matching are employed to pool them into a sparse representation. Finally, a multi-scale sparse representation is obtained by concatenating three sparse representations which respectively correspond to three particular scales of local patches, and then the linear SVM classifier is utilized for font classification. Experiments performed on a large-scale database consisting of Chinese character images in 160 fonts show that our method achieves significantly better performance compared to the state of the art. Moreover, we also carry out experiments on a subset of the database to demonstrate the effectiveness of our saliency-based sampling approach and the proposed Sobel-Prewitt feature.

This paper proposes FlexiFont, a system designed to generate personal font libraries from the camera-captured character images. Compared with existing methods, our system is able to process most kinds of languages and the generated font libraries can be extended by adding new characters based on the user's requirement. Moreover, digital cameras instead of scanners are chosen as the input devices, so that it is more convenient for common people to use the system. First of all, the users should choose a default template or define their own templates, then write the characters on the printed templates according to the certain instructions. After the users upload the photos of the templates with written characters, the system will automatically correct the perspective and split the whole photo into a set of individual character images. As the final step, FlexiFont will denoise, vectorize, and normalize each character image before storing it into a TrueType file. Experimental results demonstrate the robustness and efficiency of our system.

This paper proposes a non-rigid point set registration method called Structure-Guided Coherent Point Drift (SGCPD). The key idea of our method is to utilize structural information and combine the global and local point registrations together to improve the original Coherent Point Drift (CPD) algorithm. Specifically, given two point sets, we first align them using the CPD method with Localized Operator (CPDLO). Then we divide the target point set into several subsets and apply CPDLO to each subset. Finally, we implement the above two procedures until convergence. In this manner, more detailed information can be well exploited and thus higher registration accuracy can be achieved. Experimental results demonstrate that our method outperforms the original CPD approach on both point registration accuracy and skeleton decomposition accuracy for Chinese characters.

How to automatically generate compact and high-quality vectorization for Chinese calligraphy images is a challenging problem, since these images usually suffer from noisy contours and discontinuous strokes. In this paper, we propose a skeleton guided approach to vectorize Chinese calligraphy images. Since the skeleton reflects the writing trace and it is less influenced by contour noises, our method could extract the important writing style from the noisy contours. Specifically, in our method, the calligraphy image is first preprocessed by binarization and denoising. Then salient contour points are detected by a novel algorithm. Afterwards, under the guidance of skeleton information, the salient points are classified into corner points and joint points. Finally, a dynamic curve fitting procedure is applied to generate the vectorization result. Experimental results demonstrate that our skeleton-guided approach could automatically distinguish tiny features from contour noises and thus obtains more visually satisfactory performance compared to other existing methods.

Automatically establishing correspondence between Chinese characters is a challenging task. In this paper, we propose a novel method to solve this problem. Given two Chinese characters, we first extract and properly prune the skeleton of each character to get the key points and the connectivity relations of these points. Then, the similarity between each pair of key points is calculated via the comparison of their local features. Afterwards, a set of edges are constructed by considering both the connectivity relations and k nearest neighbors (k-nn) of each point. Finally, correspondence between two characters is established by applying a guided graph matching algorithm. Experimental results demonstrate the effectiveness of our method for the correspondence problem of Chinese characters in both printing and handwritten styles. Moreover, we also show that our method can be utilized to automatically extract strokes from Chinese characters.

Extracting metadata from academic papers has attracted much attention from researchers in past years. But how to extract metadata automatically from books is still seldom discussed. In this paper, we address this task on Chinese books and present a system to extract metadata from the title page of a book. This system consists of three components: metadata segmentation, metadata labeling, and post-processing. Different strategies are adopted in the system to identify different metadata types, and a variety of information sources, including geometric layout, linguistic, semantic content and header-footer, are used to accommodate the wide range of metadata layouts. Experimental results on real-world data have demonstrated the effectiveness of the proposed system.

In this paper, a recently proposed reversible image watermarking algorithm based on reversible contrast mapping (RCM) is further developed. The integer transform, RCM, which is originally defined on a pair of integers, is extended to integer array of arbitrary length. Based on this generalization, the embedding capacity can be significantly increased. Meanwhile, the embedding distortion is well controlled by a suitable selection of embeddable pixel blocks. In fact, the hidden data is priorly embedded into the blocks that introduce less distortion. Furthermore, the proposed scheme does not need additional data compression, and it has low computational complexity. The experiments verify that the novel scheme outperforms the original RCM-based method and some state-of-the-art reversible watermarking algorithms, especially the ones based on integer transform.