Historical Chinese character recognition faces problems including low image quality and lack of labeled training samples. We propose a generative adversarial network (GAN) based transfer learning method to ease these problems. The proposed TH-GAN architecture includes a discriminator and a generator. The network structure of the discriminator is based on a convolutional neural network (CNN). Inspired by Wasserstein GAN, the loss function of the discriminator aims to measure the probabilistic distribution distance of the generated images and the target images. The network structure of the generator is a CNN based encoder-decoder. The loss function of the generator aims to minimize the distribution distance between the real samples and the generated samples. In order to preserve the complex glyph structure of a historical Chinese character, a weighted mean squared error (MSE) criterion by incorporating both the edge and the skeleton information in the ground truth image is proposed as the weighted pixel loss in the generator. These loss functions are used for joint training of the discriminator and the generator. Experiments are conducted on two tasks to evaluate the performance of the proposed TH-GAN. The first task is carried out on style transfer mapping for multi-font printed traditional Chinese character samples. The second task is carried out on transfer learning for historical Chinese character samples by adding samples generated by TH-GAN. Experimental results show that the proposed TH-GAN is effective.

Handwritten Chinese character recognition (HCCR) is a representative large character set pattern classification task. Recently, convolutional neural networks have provided promising solutions for this challenging task. This paper adopts the modified Inception-ResNet network for handwritten Chinese character recognition, and proposes a discriminant weighting method for cross-entropy loss calculation which focuses on recognition errors in the training stage. Sparse training technique is also incorporated. Under the specific condition of utilizing the testing mini-batch mean and variance for batch normalization, the proposed method achieves improved performance on the ICDAR-2013 offline handwritten Chinese character competition dataset.

Offline handwriting recognition is a well-known challenging task in the optical character recognition (OCR) field due to the difficulty caused by various unconstraint handwriting styles. In order to learn invariant feature representations for handwriting, we propose a novel method to incorporate pixel-level rectification into a CNN and RNN based model. We also propose an adjacent output mixup method for RNN layer's training to improve the generalization ability of the model, i.e., the previous output of an RNN layer is added to the current output with random weights. We additionally adopt a series of techniques including pre-training, data augmentation and language model, and further analyze their contributions to the improvement of the model performance. The proposed method performs well on three public benchmarks, including the IAM, Rimes and IFN/ENIT datasets.

Document Image Quality Assessment (DIQA) is an essential step preceding Optical Character Recognition (OCR). In this paper we propose an attention based Recurrent Neural Network (RNN) model for camera based DIQA. Convolutional Neural Network (CNN) and RNN are integrated into our model to capture spatial features for several glimpse regions step by step within an image patch. Reinforcement learning is adopted to train a locator to generate the optimal location of a glimpse region for the next time step so that attention can be payed to the salient part. Given an input document image, patches are generated with a sliding window, and the pure background ones are sifted out. Quality scores are obtained for all the sifted patches by applying the proposed attention based RNN method, and the patch scores are averaged over each input image as the result of DIQA. We conduct experiments on two public datasets and make comparisons with several other reported methods. Experimental results show that our model achieves the state of the art performance.

Although transfer learning has aroused researchers' great interest, how to utilize the unlabeled data is still an open and important problem in this area. We propose a novel semi-supervised transfer learning (STL) method by incorporating Multi-Kernel Maximum Mean Discrepancy (MK-MMD) loss into the traditional fine-tuned Convolutional Neural Network (CNN) transfer learning framework for Chinese character recognition. The proposed method includes three steps. First, a CNN model is trained by massive labeled samples in the source domain. Then the CNN model is fine-tuned by a few labeled samples in the target domain. Finally, the CNN model is trained with both a large number of unlabeled samples and the limited labeled samples in the target domain to minimize the MK-MMD loss. Experiments investigate detailed configurations and parameters of the proposed STL method with several frequently used CNN structures including AlexNet, GoogLeNet, and ResNet. Experimental results on practical Chinese character transfer learning tasks, such as Dunhuang historical Chinese character recognition, indicate that the proposed method can significantly improve recognition accuracy in the target domain.

Deep Recurrent Neural Networks (RNN) have been suffering from the overfitting problem due to the model redundancy of the network structures. We propose a novel temporal and spatial residual learning method for RNN, followed with sparse training by weight pruning to gain sparsity in network parameters. For a Long Short-Term Memory (LSTM) network, we explore the combination schemes and parameter settings for temporal and spatial residual learning with sparse training. Experiments are carried out on the IFN/ENIT database. For the character error rate on the testing set e while training with sets a, b, c, d, the previously reported best result is 13.42%, and the proposed configuration of temporal residual learning followed with sparse training achieves the state-of-the-art result 12.06%.

This paper investigates local discriminant training and global optimization methods for Convolutional Neural Network (CNN) to improve its discriminant ability and recognition accuracy. For local discriminant training, we propose to combine triplet loss and softmax with cross-entropy loss as the loss function. The triplet loss is incorporated into an additional fully-connected layer before the final fully-connected layer of a CNN model. For global optimization, we use Conditional Random Field (CRF) to further utilize the pairwise distance of the CNN feature vectors trained with triplet loss. Experiments with different CNN models on handwritten Chinese character samples show that the combined local discriminant training and global optimization scheme achieves better character recognition accuracy and confidence analysis performance.

Uyghur language is written in a modified Arabic script. Due to its cursive nature and the lack of enough labeled training samples, Uyghur document recognition is still a challenging problem. In this paper, we propose a new Recurrent Neural Network (RNN) based Uyghur text line recognition method combining Gated Recurrent Unit (GRU) and Restricted Boltzmann Machine (RBM) with pretraining mechanism. We also present a novel curriculum learning technique guided by sample distribution information. Experimental results on practical Uyghur printed document image dataset show that the proposed network architecture and training strategy not only achieve better recognition accuracy compared with traditional methods, but can accelerate the training speed as well.

Historical Chinese character recognition has been suffering from the problem of lacking sufficient labeled training samples. A transfer learning method based on Convolutional Neural Network (CNN) for historical Chinese character recognition is proposed in this paper. A CNN model L is trained by printed Chinese character samples in the source domain. The network structure and weights of model L are used to initialize another CNN model T, which is regarded as the feature extractor and classifier in the target domain. The model T is then fine-tuned by a few labeled historical or handwritten Chinese character samples, and used for final evaluation in the target domain. Several experiments regarding essential factors of the CNNbased transfer learning method are conducted, showing that the proposed method is effective.

Recognition of handwritten mathematical expressions (HMEs) has become a cutting edge research topic recently, as there are increasingly needs for pen-inputting applications. In this paper, we presented a novel framework to analyse HME layout and semantic information. This framework includes three steps, namely symbol segmentation, symbol recognition and semantic relationship analysis. For symbol segmentation, a decomposition on strokes is operated, then dynamic programming is adopted to find the paths corresponding to the best segmentation manner and reduce the stroke searching complexity. For symbol recognition, spatial geometry and directional element features are classified by a Gaussian Mixture Model learnt through Expectation-Maximization algorithm. At last, in the semantic relationship analysis module, a ternary tree is utilized to to store the ranked symbols through calculating the operator priorities. The motivation for our work comes from the apparent difference in writing styles across western and Chinese populations. Our results are reasonable and show promise on the private dataset.

Historical Chinese document recognition technology is important for digital library. However, historical Chinese character segmentation remains a difficult problem due to the complex structure of Chinese characters and various writing styles. This paper presents a novel method for historical Chinese character segmentation based on graph model. After a preliminary over-segmentation stage, the system applies a merging process. The candidate segmentation positions are denoted by the nodes of a graph, and the merging process is regarded as selecting an optimal path of the graph. The weight of edge in the graph is calculated by the cost function which considers geometric features and recognition confidence. Experimental results show that the proposed method is effective with a detection rate of 94.6% and an accuracy rate of 96.1% on a test set of practical historical Chinese document samples.

Historical Chinese character recognition has been a challenging topic in pattern recognition field because of large character set, various writing styles and lack of training samples. In this paper, we adopted Style Transfer Mapping (STM) method to historical Chinese character recognition. Optimal selection of parameters was discussed. Two sets of experiments were conducted. The first set of experiment was designed to test the performance of STM on different font styles by using available printed traditional Chinese characters. The second set of experiment was carried out on samples extracted from practical historical Chinese documents. Experimental results showed that supervised STM may improve the generalization ability of the classifier.

This paper proposes a method to recognize stroke orders of on-line handwritten Chinese characters based on analyzing both spatial and temporal information. A novel control-point-based analysis method is presented for spatial information analysis to match strokes with various shapes and styles. Its computation complexity is much lower than image correlation method and is suitable for applications on mobile devices. For temporal information analysis, Hidden Markov Model is adopted and a proposed rectification method is integrated to find the optimal pair-wise matching result of stroke sequences. Experimental results proved the effectiveness of the proposed method. The verification rate is 99.6% on the test set.

Cross-language document recognition and understanding have urgent realistic needs and extensive application prospects. In this paper, we propose a novel recognition-based Uighur and Tibetan document understanding method, termed "cross-language sensitive words distribution map" (CSWDM). In our unified recognition-understanding framework, digital Uighur/Tibetan document images are first recognized using OCR technology, and then CSWDM labels the Chinese information of sensitive words on the recognized transcriptions or directly on the original digital images, thus the space location and occurrence frequency of these sensitive words can be intuitively represented. With such information, readers can roughly understand the theme and meaning of the cross-language documents.

HMM-based analytical methods have been widely used for Arabic handwriting recognition. A key factor influencing the performance of HMM-based systems is the features extracted from a sliding window. In this paper, we propose a novel baseline-independent feature set extracted from a wider sliding window to directly capture the contextual information. This feature set is a combination of center of mass based log-space distribution features and inverse percentile features. Center of mass based log-space distribution features use a normalized histogram to describe the distribution of foreground pixels in different direction and distances with respect to the center of mass. Experiments on the IFN/ENIT database demonstrate the effectiveness of the proposed feature set. Further, this feature set can be combined with some popular baseline-independent features to form a large feature set, which achieves comparable results with several state-of-the-art systems using a simple HMM-based architecture.

Writer recognition is a very important branch of biometrics. In our previous research, a Grid Micro-structure Feature (GMSF) based text-independent and script-independent method was adopted and high performance was obtained. However, this method is sensitive to pen-width variation in practical situation. To solve this problem, an inner and inter class variances weighted high-dimensional feature matching method is proposed. The inner and inter class variances are estimated on handwriting samples with different pen-width written by different writers. Experimental results show that our method is effective.

Automatic performance evaluation for a document page segmentation module is necessary, as OCR products are used to manipulate large scale of documents with complex layout, especially for newspapers. The paper presents a region-based method to evaluate the performance of a page segmentation module by analyzing geometric region relationships between the segmentation results and the preset ground-truth. The ground-truth is not only the correct answer to page segmentation, but also the comparison benchmark of the automatic evaluation, so it has more restricted geometric constraints. The region-matching algorithm is realized by searching the equal region in the segmentation results for each region in the ground-truth. The performance parameters are calculated based on the matching results. An experiment is given to test two page segmentation modules in a popular Chinese OCR product-THOCR2000, and the results show this method is effective.