In this paper, a new relation network based approach to curved text detection is proposed by formulating it as a visual relationship detection problem. The key idea is to decompose curved text detection into two subproblems, namely detection of text primitives and prediction of link relationship for each nearby text primitive pair. Specifically, an anchor-free region proposal network based text detector is first used to detect text primitives of different scales from different feature maps of a feature pyramid network, from which a manageable number of text primitive pairs are selected. Then, a relation network is used to predict whether each text primitive pair belongs to a same text instance. Finally, isolated text primitives are grouped into curved text instances based on link relationships of text primitive pairs. Because pairwise link prediction has used features extracted from the bounding boxes of each text primitive and their union, the relation network can effectively leverage wider context information to improve link prediction accuracy. Furthermore, since the link relationships of relatively distant text primitives can be predicted robustly, our relation network based text detector is capable of detecting text instances with large inter-character spaces. Consequently, our proposed approach achieves superior performance on not only two public curved text detection datasets, namely Total-Text and SCUT-CTW1500, but also a multi-oriented text detection dataset, namely MSRA-TD500.

With the recent success of convolutional neural network (CNN) based text detection approaches, designing better CNN-based text detection frameworks has become a major research focus to improve text detection accuracy. In this paper, instead of following this direction, we propose to use a born-again training strategy, which is based on teacher-student learning (TSL), to improve the accuracy of the state-of-the-art CNN-based text detectors. More specifically, given a well-trained CNN-based text detector, we take it as a teacher model and train from scratch a new student model with the same topology under the supervision of both the teacher model and ground-truth labels. Furthermore, we propose a new proposal-free multi-level feature mimicking approach to making multi-level convolutional feature maps be effectively mimicked in a unified manner. Experiments demonstrate that the student models trained by the proposed approach can achieve substantially better results than their teacher models and have better generalization abilities.

Detecting text from the images of whiteboards and handwritten notes is an important yet under-researched topic. In this paper, we present a robust approach to solving this challenging problem as follows. First, given a color image, colorenhanced Contrasting Extremal Regions (CERs) are extracted from its grayscale image as candidate text connected components (CCs). Second, four shallow neural networks are used to preprune efficiently most of unambiguous non-text CCs. Third, a Fast R-CNN based approach is proposed to filter out remaining nontext CCs by leveraging contextual information and to estimate the corresponding text-line orientation in the position of each remaining text CC. Fourth, each pair of the remaining text CCs within a certain distance and orientation constraint are connected to construct a directed graph. Finally, based on the estimated textline orientations, candidate text-lines are generated easily by pruning greedily redundant edges in the graph to make each vertex have at most one direct successor and one direct predecessor, respectively. Our proposed approach has achieved promising results on an in-house testing set consisting of 285 camera-captured images of whiteboards and handwritten notes.

Although Faster R-CNN based approaches have achieved promising results for text detection, their localization accuracy is not satisfactory in certain cases. In this paper, we propose to use a LocNet to improve the localization accuracy of a Faster R-CNN based text detector. Given a proposal generated by region proposal network (RPN), instead of predicting directly the bounding box coordinates of the concerned text instance, the proposal is enlarged to create a search region so that conditional probabilities to each row and column of this search region can be assigned, which are then used to infer accurately the concerned bounding box. Experiments demonstrate that the proposed approach boosts the localization accuracy for Faster R-CNN based text detection significantly. Consequently, our new text detector has achieved superior performance on ICDAR-2011, ICDAR-2013 and MULTILIGUL text detection benchmark tasks.

Although previous studies have achieved effective printed Chinese character recognition (PCCR) in the case a single font or a few different fonts, large scale multi-font PCCR remains a major challenge owing to the wide variety in the shape, layout, and grey-level distribution of single Chinese characters across different font styles. This paper applies multi-pooling and data augmentation with non-linear transformation to a convolutional neural network (CNN) for multi-font PCCR. We propose a multi-pooling layer on top of the final convolutional layer; this approach is found to be robust to spatial layout variations and deformations in multi-font printed Chinese characters. Experimental results show that multi-pooling significantly improves CNN performance. In addition, we adopt a distorted sample generation technique by applying non-linear warping functions along an original font image, which distorts the local density of image-based Chinese character strokes. We find that CNN performance is further boosted by the distorted samples technique. An input character image is transformed into four distorted images and the CNN learns the original image as well as the distorted samples to classify 3755 classes (level-1 set of GB2312-80) of printed Chinese characters in 280 widely varying fonts and 120 manually selected fonts. Outstanding recognition rates of 94.38% and 99.74% are achieved in the former and latter cases, respectively, which indicates the effectiveness of the proposed methods.

Just like its great success in solving many computer vision problems, the convolutional neural networks (CNN) provided new end-to-end approach to handwritten Chinese character recognition (HCCR) with very promising results in recent years. However, previous CNNs so far proposed for HCCR were neither deep enough nor slim enough. We show in this paper that, a deeper architecture can benefit HCCR a lot to achieve higher performance, meanwhile can be designed with less parameters. We also show that the traditional feature extraction methods, such as Gabor or gradient feature maps, are still useful for enhancing the performance of CNN. We design a streamlined version of GoogLeNet [13], which was original proposed for image classification in recent years with very deep architecture, for HCCR (denoted as HCCR-GoogLeNet). The HCCR-GoogLeNet we used is 19 layers deep but involves with only 7.26 million parameters. Experiments were conducted using the ICDAR 2013 offline HCCR competition dataset. It has been shown that with the proper incorporation with traditional directional feature maps, the proposed single and ensemble HCCR-GoogLeNet models achieve new state of the art recognition accuracy of 96.35% and 96.74%, respectively, outperforming previous best result with significant gap.