In this paper, we address the issue of scene text detection in the way of direct regression and successfully adapt an effective semantic segmentation model, DeepLab v3+ [1], for this application. In order to handle texts with arbitrary orientations and sizes and improve the recall of small texts, we propose to extract features of multiple scales by inserting multiple Atrous Spatial Pyramid Pooling (ASPP) layers to the DeepLab after the feature maps with different resolutions. Then, we set multiple auxiliary IoU losses at the decoding stage and make auxiliary connections from the intermediate encoding layers to the decoder to assist network training and enhance the discrimination ability of lower encoding layers. Experiments conducted on the benchmark scene text dataset ICDAR2015 demonstrate the superior performance of our proposed network, named as DeepText, over the state-of-the-art approaches.

Handwritten Chinese text recognition (HCTR) has received extensive attention from the community of pattern recognition in the past decades. Most existing deep learning methods consist of two stages, i.e., training a text recognition network on the base of visual information, followed by incorporating language constrains with various language models. Therefore, the inherent linguistic semantic information is often neglected when designing the recognition network. To tackle this problem, in this work, we propose a novel multi-level multimodal fusion network and properly embed it into an attention-based LSTM so that both the visual information and the linguistic semantic information can be fully leveraged when predicting sequential outputs from the feature vectors. Experimental results on the ICDAR-2013 competition dataset demonstrate a comparable result with the state-of-the-art approaches.

We propose a hierarchical CNN model for the recognition of confusable similar handwritten Chinese characters, which are automatically extracted from a large character set by utilizing a classifier's recognition result. The proposed hierarchical CNN model takes advantage of deep networks and traditional hierarchical methods, and consists of two stages, which are expected to differentiate inter-group characters and intra-group characters, respectively. Different from traditional ways of expanding depth and/or width of general sole classifier CNNs, we explore the way of designing multiple parallel CNN classifiers to capture critical regions of similar characters. Each classifier along with their feature extraction layers is trained only with a group of similar characters so that the subtle shape difference can be captured. Totally, 368 similar characters (categorized into 172 groups) are extracted from 3755 frequently used Chinese characters. Experimental results on these similar characters demonstrate the superiority of the proposed method to the expanded CNN models.

Different recognizers may result in different mistakes when they are used to recognize a Chinese address. In this paper, we present a method of combining multiple Chinese address recognition outputs to improve Chinese address recognition accuracy. The method first employs multiple sequence alignment to generate a lattice of candidate hypotheses from multiple different recognizer outputs and then applies statistical language model to choose the maximum likelihood candidate sequence. Taking the maximum as the final decision, the performance of our method is superior, compared to the single recognizers and Miyao's method. The experiments on the address images of real envelopes demonstrate that the proposed method increases the character recognition accuracy rate from 95.80% to 98.38%, with 61.30% error reduction. Furthermore, the corrected sorting rate of an automatic mail sorting system increases from 84.11% to 93.72%.

We present a text detection method in natural scene images based on two-stage nontext filtering. Firstly, we detect multi-channel maximally stable extremal regions (MSERs) as character candidates. To reduce the amount of repeating components, we merge the MSERs by choosing the most character-like ones when overlap happens. Then nontext components are filtered out by a two-stage labeling procedure, wherein we combine random forests with CRF. Finally, components labeled as text are grouped into words by an edge-cut strategy, and false positives are eliminated by a HOG-based classifier. The experimental results on the ICDAR2013 database show the effectiveness of the proposed method.

This paper presents a method for text-independent writer identification using SIFT descriptor and contour-directional feature (CDF). The proposed method contains two stages. In the first stage, a codebook of local texture patterns is constructed by clustering a set of SIFT descriptors extracted from images. Using this codebook, the occurrence histograms are calculated to determine the similarities between different images. For each image, we obtain a candidate list of reference images. The next stage is to refine the candidate list using the contour-directional feature and SIFT descriptor. The proposed method is evaluated with two datasets: the ICFHR2012-Latin dataset and the ICDAR2013 dataset. Experimental results show that the proposed method outperforms the state-of-the-art algorithms and archives the best performance.

We present a Histogram of Oriented Gradient (HoG) based two-directional Dynamic Time Warping (DTW) matching method for handwritten word spotting. Firstly, we extract HoG descriptors from each cell in the normalized images. Then we connect the HoG descriptors in the same column and get a sequence of feature vectors. We do the same operation for the HoG descriptors in the same row. We then apply the two-directional DTW method to calculate the distance between the feature vectors sequences extracted from the query word and the candidate one. The experimental results show that the two-directional DTW is more robust to word deformation than the traditional DTW. And the local features such as HoG, LBP and SIFT combined with the two-directional DTW method outperform the method using the local feature descriptors directly. The HoG based two-directional DTW get the highest mean average precision on both the George Washington dataset and the CASIA-HWDB 2.1 dataset.

The issue of near duplicate document image retrieval is addressed in this paper, which is characterized by not only encoding each individual word in the image but also modeling its local spatial configuration. On representing each word in the image as a string in terms of its shape characteristics, a lexicon is first learnt from a training set. Then a word in an arbitrary document image can be soft assigned to a weighted combination of several nearest neighbors in the lexicon. The rationale behind soft-assignment is to tolerate the distortions induced by character segmentations which are error-prone in degraded document images. Most importantly, we look beyond the single word and capture the local spatial configuration for each word which plays a very important role in human perception. It provides much useful information in discriminating between different document images compared with the single word. A graph, benefitting from its great representative power, is built for each word to model its relationships with the neighborhoods locally. The local word spatial configurations are integrated within the inverted file index structure to achieve scalable retrieval. Thus the retrieval of near duplicate document images is formulated as a voting problem. Experimental results on 45,000 document images demonstrate that the proposed approach brings significant improvements in successful retrieval of near duplicate images.

A graph matching approach is proposed to retrieve envelope images from a large image database. First, the graph representation of an envelop image is generated based on the image segmentation results, in which each node corresponds to one segmented region. The attributes of nodes and edges in the graph are described by characteristics of the envelope image. Second, a minimum weighted bipartite graph matching method is employed to compute the distance between two graphs. Finally, the whole retrieval system including two principal stages is presented, namely, rough matching and fine matching. The experiments on a database of envelope images captured from real-life mail pieces demonstrate that the proposed method achieves promising results.

Voronoi diagram of image elements provides an intuitive and appealing definition of proximity, which has been suggested as an effective tool for the description of relations among the neighboring objects in a digital image. In this paper, a fast implementation algorithm is proposed for generating area Voronoi diagram of connected components in document images. A closed convex polygon is utilized to bound each connected component, and the contour is represented using Freeman chain coding, from which we can compute the corresponding Freeman chain coding of its expanded contour directly, without recourse to the operation on pixels. While the contours iteratively expand outwards, the Voronoi diagram is constructed by the intersections of the expanded contours from different connected components. The experimental results show that our proposed approach significantly improves the speed of constructing area Voronoi diagram.

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

With the rising popularity and importance of document images as an information source, information retrieval in document image databases has become a growing and challenging problem. In this paper, we propose an approach with the capability of matching partial word images to address two issues in document image retrieval: word spotting and similarity measurement between documents. First, each word image is represented by a primitive string. Then, an inexact string matching technique is utilized to measure the similarity between the two primitive strings generated from two word images. Based on the similarity, we can estimate how a word image is relevant to the other and, thereby, decide whether one is a portion of the other. To deal with various character fonts, we use a primitive string which is tolerant to serif and font differences to represent a word image. Using this technique of inexact string matching, our method is able to successfully handle the problem of heavily touching characters. Experimental results on a variety of document image databases confirm the feasibility, validity, and efficiency of our proposed approach in document image retrieval.

Summary form only given. A compressed pattern matching method for searching keywords from the CCIT group 4-compressed document images, without explicit decompression, is presented. According to the CCIT Group 4 standards, each coded position indicates current pixel color is different from its previous pixel, except for the next coded positions of the pass mode. The changing elements from the compressed images are extracted and are then utilized to segment and bound the word objects and to measure the similarity of two word images. A two-stage matching strategy is constructed to measure the dissimilarity between the template image of the user's query word and the word extracted from document images. Experiments were conducted to verify the validity of the approach. The results show that the proposed approach was much faster than the traditional approach, because it avoids the pixel-level processing for analyzing the connected components and extracting word features.

In this paper, we present a compressed pattern matching method for searching user queried words in the CCITT Group 4 compressed document images, without decompressing. The feature pixels composed of black changing elements and white changing elements are extracted directly from the CCITT Group 4 compressed document images. The connected components are labeled based on a line-by-line strategy according to the relative positions between the changing elements of the current coding line and the changing elements of the reference line. Word boxes are bounded by merging the connected components. A two-stage matching strategy is constructed to measure the dissimilarity between the template image of the user's query word and the words extracted from document images. Experimental results confirmed the validity of the proposed approach.

The nearest-neighbor based document skew detection methods do not require the presence of a predominant text area, and are not subject to skew angle limitation. However, the accuracy of these methods is not perfect in general. In this paper, we present an improved nearest-neighbor based approach to perform accurate document skew estimation. Size restriction is introduced to the detection of nearest-neighbor pairs. Then the chains with a largest possible number of nearest-neighbor pairs are selected, and their slopes are computed to give the skew angle of document image. Experimental results on various types of documents containing different linguistic scripts and diverse layouts show that the proposed approach has achieved an improved accuracy for estimating document image skew angle and has an advantage of being language independent.

An approach to word image matching based on weighted Hausdorff distance (WHD) is proposed in this paper to facilitate the detection and location of the user-specified words in the document images. Preprocessing such as eliminating the space between adjacent characters in the word images and scale normalization is first done before the WHD is utilized to measure the distance between the template image and the word image extracted from the document image. Experimental results in the application of detecting the user-specified words from both English and Chinese document images show that it is a promising approach for word image matching.