Recently, character model based on integrated convolutional neural network (CNN) and deep bidirectional long short-term memory (DBLSTM) has been demonstrated to be effective for online handwritten Chinese text recognition (HCTR). However, the reported CNN-DBLSTM topologies are too complex to be practically useful. In this paper, we propose a compact CNN-DBLSTM which has small footprint and low computation cost yet be able to accommodate multiple receptive fields for CNN-based feature extraction. By using the training set of a popular benchmark database, namely CASIA-OLHWDB, we trained a compact CNN-DBLSTM by a connectionist temporal classification (CTC) criterion with a multi-step training strategy. Combined this character model with a character trigram language model, our online HCTR system with a WFSTbased decoder has achieved state-of-the-art performance on both CASIA and ICDAR-2013 Chinese handwriting recognition competition test sets.

The accuracy of a typical state-of-the-art optical character recognition (OCR) system benefits greatly from using a language model (LM). However, a conventional LM has a limited vocabulary, resulting in out-of-vocabulary (OOV) words that cannot be recognized by the OCR system. In this paper, we present an open vocabulary OCR system based on a hybrid LM. The vocabulary of the hybrid LM consists of both words and subwords. OOV words can be generated by combinations of subwords. A refined hybrid LM training scheme is applied by interpolating a standard hybrid LM, a word-based LM and a subword-based LM. An efficient word combination method is performed by modeling optional space symbols in a decoding network. The overall system deals with OOV words in a general, data-driven and language-independent way. We conduct experiments on an English handwriting OCR task. Evaluations on three testing sets demonstrate that the OCR system with the proposed method achieves a word error rate of 33.4% on an OOV-only testing set, yet without degrading the recognition accuracies on the other two testing sets mainly consisting of in-vocabulary words.

Recently, character model based on integrated convolutional neural network (CNN) and deep bidirectional long short-term memory (DBLSTM) has achieved excellent performance for offline handwriting recognition (HWR). To deploy CNN-DBLSTM model in products, it is necessary to reduce the footprint and runtime latency as much as possible. In this paper, we study two methods to compress the CNN part: (1) Use Tucker decomposition to decompose pre-trained weights with low-rank approximation, followed by fine-tuning; (2) Use grouped convolution to construct sparse connections in channel domain. Experiments have been conducted on a large-scale offline English HWR task to compare the effectiveness of the above two techniques. Our results show that using Tucker decomposition alone offers a good solution to building a compact CNN-DBLSTM model which can reduce significantly both the footprint and latency yet without degrading recognition accuracy.

We study two sequence discriminative training criteria, i.e., Lattice-Free Maximum Mutual Information (LFMMI) and Connectionist Temporal Classification (CTC), for end-to-end training of Deep Bidirectional Long Short-Term Memory (DBLSTM) based character models of two offline English handwriting recognition systems with an input feature vector sequence extracted by Principal Component Analysis (PCA) and Convolutional Neural Network (CNN), respectively. We observe that refining CTC-trained PCA-DBLSTM model with an interpolated CTC and LFMMI objective function ("CTC+LFMMI") for several additional iterations achieves a relative Word Error Rate (WER) reduction of 24.6% and 13.9% on the public IAM test set and an in-house E2E test set, respectively. For a much better CTC-trained CNN-DBLSTM system, the proposed "CTC+LFMMI" method achieves a relative WER reduction of 19.6% and 8.3% on the above two test sets, respectively.

We propose a context-sensitive-chunk based back-propagation through time (BPTT) approach to training deep (bidirectional) long short-term memory ((B)LSTM) recurrent neural networks (RNN) that splits each training sequence into chunks with appended contextual observations for character modeling of offline handwriting recognition. Using short context-sensitive chunks in both training and recognition brings following benefits: (1) the learned (B)LSTM will model mainly local character image dependency and the effect of long-range language model information reflected in training data is reduced; (2) mini-batch based training on GPU can be made more efficient; (3) low-latency BLSTM-based handwriting recognition is made possible by incurring only a delay of a short chunk rather than a whole sentence. Our approach is evaluated on IAM offline handwriting recognition benchmark task and performs better than the previous state-of-the-art BPTT-based approaches.