The presence of both caption/graphics/superimposed and scene texts in video frames is the major cause for the poor accuracy of text recognition methods. This paper proposes an approach for identifying tampered information by analyzing the spatial distribution of DCT coefficients in a new way for classifying caption and scene text. Since caption text is edited/superimposed, which results in artificially created texts comparing to scene texts that exist naturally in frames. We exploit this fact to identify the presence of caption and scene texts in video frames based on the advantage of DCT coefficients. The proposed method analyzes the distributions of both zero and non-zero coefficients (only positive values) locally by moving a window, and studies histogram operations over each input text line image. This generates line graphs for respective zero and non-zero coefficient coordinates. We further study the behavior of text lines, namely, linearity and smoothness based on centroid location analysis, and the principal axis direction of each text line for classification. Experimental results on standard datasets, namely, ICDAR 2013 video, 2015 video, YVT video and our own data, show that the performances of text recognition methods are improved significantly after-classification compared to before-classification.

Developing a unified text detection and recognition method is hard for different video types due to varying characteristics in video. This paper proposes a new method for categorizing different types of video text frames, namely, videos containing advertisement, signboard, license plate, front page of book or magazine, street view, and video of general items, for better text detection and recognition rate. We propose symmetry features using gradient vector flow for Canny and Sobel edge images of each input frame to identify candidate edge components. Then for a candidate edge component image, we extract both global and local features using colors from different channels in a new way. Besides, the proposed method extracts statistical and structural features from the spatial distribution of candidate pixels in a multi-scale environment. Lastly, the extracted features are fed to a logistic classifier for categorization. The features extracted locally and globally are tested both separately and altogether in terms of confusion matrix. The performance of the proposed categorization method is evaluated through several text detection and recognition experiments before and after categorization. We noted that the proposed categorization method is very useful in improving text detection and recognition performance.

Text recognition in video/natural scene images has gained significant attention in the field of image processing in many computer vision applications, which is much more challenging than recognition in plain background images. In this paper, we aim to restore complete character contours in video/scene images from gray values, in contrast to the conventional techniques that consider edge images/binary information as inputs for text detection and recognition. We explore and utilize the strengths of zero crossing points given by the Laplacian to identify stroke candidate pixels (SPC). For each SPC pair, we propose new symmetry features based on gradient magnitude and Fourier phase angles to identify probable stroke candidate pairs (PSCP). The same symmetry properties are proposed at the PSCP level to choose seed stroke candidate pairs (SSCP). Finally, an iterative algorithm is proposed for SSCP to restore complete character contours. Experimental results on benchmark databases, namely, the ICDAR family of video and natural scenes, Street View Data, and MSRA data sets, show that the proposed technique outperforms the existing techniques in terms of both quality measures and recognition rate. We also show that character contour restoration is effective for text detection in video and natural scene images.

The prevalent scene text detection approach follows four sequential steps comprising character candidate detection, false character candidate removal, text line extraction, and text line verification. However, errors occur and accumulate throughout each of these sequential steps which often lead to low detection performance. To address these issues, we propose a unified scene text detection system, namely Text Flow, by utilizing the minimum cost (min-cost) flow network model. With character candidates detected by cascade boosting, the min-cost flow network model integrates the last three sequential steps into a single process which solves the error accumulation problem at both character level and text line level effectively. The proposed technique has been tested on three public datasets, i.e, ICDAR2011 dataset, ICDAR2013 dataset and a multilingual dataset and it outperforms the state-of-the-art methods on all three datasets with much higher recall and F-score. The good performance on the multilingual dataset shows that the proposed technique can be used for the detection of texts in different languages.

Achieving a good recognition rate for scene characters is a big challenge due to non-uniform illumination effects, perspective distortions, multiple colors or contrasts, different fonts and their various sizes, background or orientation variations, etc. Unlike the existing recognition methods that use binary information or the features extracted from different domains, the proposed method explores gray information in the form of a filter bank to extract the discriminative power for all the 62 scene character classes. We propose a sliding window (patch) operation over a character image for learning the global features, which represent the structures of character images of all the classes by reconstructing a filter bank from the original data. We introduce shareable constrains to activate class-specific filters from the filter bank. Further, we propose constraints by studying the nearest neighbor patches and exemplar selection to maximize the gap between inter-classes and minimize the gap between intra-classes. The method is evaluated and compared with several existing recognition methods in terms of character recognition rate. Experimental results show that the proposed method outperforms the existing methods.

Character segmentation is an important topic to improve the overall performance of text recognition methods due to low resolution, complex background and lots of visual variations in video. This paper presents a novel idea for segmenting characters from arbitrarily-oriented text lines based on wavelet and Laplacian combination. Firstly, we explore wavelet which decomposes a given input image into sub-levels like a pyramid structure for segmenting words based on the fact that as decomposition level increases, the gap between characters decreases due to the reduction in the size of the input image, which results in a single component for each word. Secondly, for each segmented word, we propose Laplacian wavelet combination in a new way to extract text candidates. Thirdly, we propose horizontal and vertical sampling for character segmentation from words. The proposed method is tested on curved, non-horizontal and horizontal text lines of video and the ICDAR 2005 natural scene dataset to evaluate its performance. A comparative study with an existing method shows that the proposed method outperforms it in terms of precision and f-measure.

Recognition of handwritten text is a useful technique that can be applied in different applications, such as signature recognition, bank check recognition, etc. However, the off-line handwritten text recognition in an unconstrained situation is still a very challenging task due to the high complexity of text strokes and image background. This paper presents a novel segmented handwritten text recognition technique that ensembles recurrent neural network (RNN) classifiers. Two RNN models are first trained that take advantage of the widely used geometrical feature and the Histogram of Oriented Gradient (HOG) feature, respectively. Given a handwritten word image, the optimal recognition result is then obtained by integrating the two trained RNN models together with a lexicon. Experiments on public datasets show the superior performance of our proposed technique.

Text segmentation provides important clues for the accurate identification of character locations and the analysis of character properties such as shape estimation and texture synthesis. In this paper, we propose a robust text segmentation method that employs Markov Random Field (MRF) and use graph cut algorithms to solve the energy minimization problem. To effectively select accurate seeds to boost the text segmentation performance, stroke feature transform is adopted to robustly identify text seeds and text edges. Background seeds are obtained near the text edges in order to well preserve the text boundaries. The energy functions are defined as an MRF consisting of data energy and smoothness energy which can be efficiently solved by graph cut algorithms. One distinctive property of the proposed technique is that it can identify more distinctive seeds so that only one cut is needed to well separate the text regions from the background, hence much faster than the existing iterative graph cut approach. Experiments on ICDAR 2003 and ICDAR 2011 datasets show that the proposed technique obtains superior performance on both pixel level and atom level segmentation.

Scene text detection from video as well as natural scene images is challenging due to the variations in background, contrast, text type, font type, font size, and so on. Besides, arbitrary orientations of texts with multi-scripts add more complexity to the problem. The proposed approach introduces a new idea of convolving Laplacian with wavelet sub-bands at different levels in the frequency domain for enhancing low resolution text pixels. Then, the results obtained from different sub-bands (spectral) are fused for detecting candidate text pixels. We explore maxima stable extreme regions along with stroke width transform for detecting candidate text regions. Text alignment is done based on the distance between the nearest neighbor clusters of candidate text regions. In addition, the approach presents a new symmetry driven nearest neighbor for restoring full text lines. We conduct experiments on our collected video data as well as several benchmark data sets, such as ICDAR 2011, ICDAR 2013, and MSRA-TD500 to evaluate the proposed method. The proposed approach is compared with the state-of-the-art methods to show its superiority to the existing methods.

Text detection and tracking in video is challenging due to contrast, resolution and background variations, and different orientations and text movements. In addition, the presence of both caption and scene texts in video aggravates the problem because these two text types differ in characteristics significantly . This paper proposes a new technique for detecting and tracking video texts of any orientation by using spatial and temporal information, respectively. The technique explores gradient directional symmetry at component level for smoothing edge components before text detection. Spatial information is preserved by forming Delaunay triangulation in a novel way at this level, which results in text candidates. Text characteristics are then proposed in a different way for eliminating false text candidates , which results in potential text candidates. Then grouping is proposed for combining potential text candidates regardless of orientation based on the nearest neighbor criterion. To tackle the problems of multi-font and multi-sized texts, we propose multi-scale integration by a pyramid structure, which helps in extracting full text lines. Then, the detected text lines are tracked in video by matching the subgraphs of triangulation. Experimental results for text detection and tracking on our video dataset, the benchmark video datasets, and the natural scene image benchmark datasets show that the proposed method is superior to the state-of-the-art methods in terms of recall, precision , and F-measure.

The presence of both graphics and scene text in video frames makes text detection and recognition problem more challenging because the nature of the two texts differs significantly. This paper aims to propose a novel method for separation of graphics and scene text to achieve good recognition rate based on the fact that Canny and Sobel edge pattern share common property for text. We propose to use Ring Radius Transform to identify the radius that represents the medial axis in the edge image. We study the intra relationship between bins of the histograms over respective radius values, resulting in intra line graphs. In this way, the method finds intra line graphs for both Canny and Sobel edge images of the input text lines. To identify the unique distribution for separation of graphics and scene texts, we explore the inter relationship between intra line graphs of Canny and Sobel edge image with respective medial axes values. This results in Gaussian distribution for graphics and non-Gaussian for scene text. Experimental results on horizontal, non-horizontal, different scripts etc. show that the proposed method is effective for classification and the results of baseline recognition methods show that recognition rate is significantly improved after classification.

Word segmentation from video text line is challenging because video poses several challenges, such as complex background, low resolution, arbitrary orientation, etc. Besides, word segmentation is essential for improving text recognition accuracy. Therefore, we propose a novel method for segmenting words by exploring zero crossing points for each sliding window over text line. The candidate zero crossing pointes are defined based on characteristics of positive and negative Laplacian values at text region and non-text region. The percentage of candidate zero crossing points is calculated for each sliding window and is used for identifying the seed window that represents space between words. For the seed window, we propose a novel idea of horizontal and vertical sampling based on the percentage values to estimate the width and the height of the word spacing. Then the width and the height of the word spacing are used to validate the actual word spacing. Experimental results comparing with an existing method show that the proposed method is better than the existing method in terms of recall, precision and f-measure on curved, horizontal, non-horizontal, Hua's video data, as well as ICDAR data. We also test it on our own data containing multiscript text lines to show the robustness of the proposed method.

Text detection and tracking in video sequence is gaining interest due to the challenges posed by low resolution and complex background. This paper proposes a new method for text detection by estimating trajectories between the corners of texts in video sequence over time. Each trajectory is considered as one node to form a graph for all trajectories and Delaunay triangulation is used to obtain edges to connect nodes of the graph. In order to identify the edges that represent text regions, we propose four pruning criteria based on spatial proximity, motion coherence, local appearance and canny rate. This results in several sub-graphs. Then we use depth first search to collect corner points, which essentially represent text candidates. False positives are eliminated using heuristics and missing trajectories will be obtained by tracking the corners in temporal frames. We test the method on different videos and evaluate the method in terms of recall, precision, f-measure with existing results. Experimental result shows that the proposed method is superior to existing method.

In this paper, a segmentation-free keyword spotting method is proposed for Bangla handwritten documents. In order to tolerate large variations in handwritten scenarios, we extracted key points based on SIFT key point detector, and the end and intersection points found by morphological operations. Heat Kernel signature (HKS) is used to present the local characteristics of detected key points. Instead of using the same size of patch for all the key points, we apply a method dynamically deciding the patch size. Furthermore, our spotting method reduces the scope of searching on the document by only considering the candidate local zones with similar candidate key points, and does not need pre-processing steps. From the experiment on Bangla handwritten text we obtained encouraging results.

This paper proposes a language-independent method for segmenting text lines from handwritten document images. Our method is based on the seam carving, which has been already used for text line segmentation, but in order to tolerate multi-skewed text lines even in the same document image, we propose a constrained seam carving method, which can constrain energy to be passed along the connected components in the same text line as much as possible. Moreover, our proposed method tries to extract all the text lines by computing the energy map only once. In the experiments, our method is tested on the Greek, English and Indian document images, and get 98.41% FM score.

Because of the unconstrained environment of scene text, traditional Optical Character Recognition (OCR) engines fail to achieve satisfactory results. In this paper, we propose a new technique which employs first order Histogram of Oriented Gradient (HOG) through a spatial pyramid. The spatial pyramid can encode the relative spatial layout of the character parts while HOG can only include the local image shape without spatial relation. A feature descriptor combining these two can extracts more useful information from the image for text recognition. Chi-square kernel based Support Vector Machine is employed for classification based on the proposed feature descriptors. The method is tested on three public datasets, namely ICDAR2003 robust reading dataset, Street View Text (SVT) dataset and IIIT 5K-word dataset. The results on these dataset are comparable with the state-of-the-art methods.

It has been estimated that autoimmune diseases are among the top ten leading causes of death among women in all age groups up to 65 years. However, the detection of it by indirect immunofluorescence (IIF) image analysis depends heavily on the experience of the physicians. An accurate and automatic Computer Aided Diagnosis (CAD) system will help greatly for the classification of the Human Epithelial type 2 (HEp-2) cell images with little human intervention. In this paper we present an automatic HEp-2 cell image classification technique that exploits different spatial scaled image representation and sparse coding of SIFT features. Additionally, spatial max pooling of sparse coding at different scales is used to boost the classification performance. The proposed method is tested on the ICPR 2012 contest dataset and experiments show that it clearly outperforms state-of-the-art techniques in cell and image level as well as two intensity level images.

A novel statistical framework for modeling the intrinsic structure of crowded scenes and detecting abnormal activities is presented in this paper. The proposed framework essentially turns the anomaly detection process into two parts, namely, motion pattern representation and crowded context modeling. During the first stage, we averagely divide the spatiotemporal volume into atomic blocks. Considering the fact that mutual interference of several human body parts potentially happen in the same block, we propose an atomic motion pattern representation using the Gaussian Mixture Model (GMM) to distinguish the motions inside each block in a refined way. Usual motion patterns can thus be defined as a certain type of steady motion activities appearing at specific scene positions. During the second stage, we further use the Markov Random Field (MRF) model to characterize the joint label distributions over all the adjacent local motion patterns inside the same crowded scene, aiming at modeling the severely occluded situations in a crowded scene accurately. By combining the determinations from the two stages, a weighted scheme is proposed to automatically detect anomaly events from crowded scenes. The experimental results on several different outdoor and indoor crowded scenes illustrate the effectiveness of the proposed algorithm.

Script identification at the word level is challenging because of complex backgrounds and low resolution of video. The presence of graphics and scene text in video makes the problem more challenging. In this paper, we employ gradient angle segmentation on words from video text lines. This paper presents new Gradient-Angular-Features (GAF) for video script identification, namely, Arabic, Chinese, English, Japanese, Korean and Tamil. This work enables us to select an appropriate OCR when the frame has words of multi-scripts. We employ gradient directional features for segmenting words from video text lines. For each segmented word, we study the gradient information in effective ways to identify text candidates. The skeleton of the text candidates is analyzed to identify Potential Text Candidates (PTC) by filtering out unwanted text candidates. We propose novel GAF for the PTC to study the structure of the components in the form of cursiveness and softness. The histogram operation on the GAF is performed in different ways to obtain discriminative features. The method is evaluated on 760 words of six scripts having low contrast, complex background, different font sizes, etc. in terms of the classification rate and is compared with an existing method to show the effectiveness of the method. We achieve 88.2% average classification rate.

Clinical features found in brain CT scan images are widely used in traumatic brain injury (TBI) as indicators for Glasgow Outcome Scale (GOS) prediction. However, due to the lack of automated methods to measure and quantify the CT scan image features, the computerized prediction of GOS in TBI has not been well studied. This paper introduces an automated GOS prediction system for traumatic brain CT images. Different from most existing systems that perform the prognosis based on pre-processed data, our system directly works on brain CT scan images based on the image features. Our system can also be extended to large dataset with easy adaptation. For each new image of a CT scan series, our proposed system first makes use of sparse representation model that predicts the GOS of each CT image slice using Gabor features. Logistic regression, which integrates the GOS of each CT scan slice with a pre-trained model, is then applied to estimate the GOS score for the new case which contains multiple CT slices. Evaluation of the system has shown promising results in prediction of GOS of traumatic brain injury cases.

Recognition of characters in natural images is a challenging task due to the complex background, variations of text size and perspective distortion, etc. Traditional optical character recognition (OCR) engine cannot perform well on those unconstrained text images. A novel technique is proposed in this paper that makes use of convolutional cooccurrence histogram of oriented gradient (ConvCoHOG), which is more robust and discriminative than both the histogram of oriented gradient (HOG) and the co-occurrence histogram of oriented gradients (CoHOG). In the proposed technique, a more informative feature is constructed by exhaustively extracting features from every possible image patches within character images. Experiments on two public datasets including the ICDAr 2003 Robust Reading character dataset and the Street View Text (SVT) dataset, show that our proposed character recognition technique obtains superior performance compared with state-of-the-art techniques.

The segmentation of scene text from the image background has shown great importance in scene text recognition. In this paper, we propose a multi-level MSER technology that identifies the best-quality text candidates from a set of stable regions that are extracted from different color channel images. In order to identify the best-quality text candidates, a segmentation score is defined which exploits four measures to evaluate the text probability of each stable region including: 1) Stroke width that measures the small stroke width variation of the text, 2) Boundary curvature that measures the smoothness of the stable region boundary, 3) Character confidence that measures the likelihood of a stable region being text based on a pre-trained support vector classifier, 4) Color constancy that measures the global color consistency of each selected text candidate. Finally, the MSERs with the best segmentation score from each channel are combined to form the final segmentation. The proposed method is evaluated on the ICDAR2003 and SVT datasets and experiments show that it outperforms both popular document image binarization methods and state of the art scene text segmentation methods.

Achieving good accuracy for text detection and recognition is a challenging and interesting problem in the field of video document analysis because of the presences of both graphics text that has good clarity and scene text that is unpredictable in video frames. Therefore, in this paper, we present a novel method for classifying graphics texts and scene texts by exploiting temporal information and finding the relationship between them in video. The method proposes an iterative procedure to identify Probable Graphics Text Candidates (PGTC) and Probable Scene Text Candidates (PSTC) in video based on the fact that graphics texts in general do not have large movements especially compared to scene texts which are usually embedded on background. In addition to PGTC and PSTC, the iterative process automatically identifies the number of video frames with the help of a converging criterion. The method further explores the symmetry between intra and inter character components to identify graphics text candidates and scene text candidates. Boundary growing method is employed to restore the complete text line. For each segmented text line, we finally introduce Eigen value analysis to classify graphics and scene text lines based on the distribution of respective Eigen values. Experimental results with the existing methods show that the proposed method is effective and useful to improve the accuracy of text detection and recognition.

Text detection and recognition is a challenging problem in document analysis due to the presence of the unpredictable nature of video texts, such as the variations of orientation, font and size, illumination effects, and even different 2D/3D text shadows. In this paper, we propose a novel horizontal and vertical symmetry feature by calculating the gradient direction and the gradient magnitude of each text candidate, which results in Potential Text Candidates (PTCs) after applying the k-means clustering algorithm on the gradient image of each input frame. To verify PTCs, we explore temporal information of video by proposing an iterative process that continuously verifies the PTCs of the first frame and the successive frames, until the process meets the converging criterion. This outputs Stable Potential Text Candidates (SPTCs). For each SPTC, the method obtains text representatives with the help of the edge image of the input frame. Then for each text representative, we divide it into four quadrants and check a new Mutual Nearest Neighbor Symmetry (MNNS) based on the dominant stroke width distances of the four quadrants. A voting method is finally proposed to classify each text block as either 2D or 3D by counting the text representatives that satisfy MNNS. Experimental results on classifying 2D and 3D text images are promising, and the results are further validated by text detection and recognition before classification and after classification with the exiting methods, respectively.

To get high recognition accuracy, we should train the recognizer with sufficient training data to capture characteristics of various handwriting styles and all possible occurring words. However, in most of the cases, available training data are not satisfactory and enough, especially for unseen data. In this paper, we try to improve the recognition accuracy for unseen data with randomly selected training data, by splitting the training data into two parts based on trigrams and training two recognizers separately. We also propose a modified version of token passing algorithm, which makes use of the outputs of the two recognizers to improve the recognition accuracy.

Although a large number of methods for video text detection and recognition have been proposed over the past years, it is hard to find the best state-of-the-art method because of nonavailability of standard datasets, ground truth, and common evaluation measures. Therefore, in this paper, we propose a semiautomatic system for ground truth generation for video text detection and recognition, which includes English and Chinese text of different orientation. The system has a facility to allow the user to manually correct the ground truth if the automatic method produces incorrect results. We propose eleven attributes at the word level, namely: line index, word index, coordinate values of bounding box, area, content, script type, orientation information, type of text (caption/scene), condition of text (distortion/distortion free), start frame, and end frame to evaluate the performance of the method. We also introduce a new dataset that consists of 466 video frames collected from TRECVID 2005 and 2006 databases. The video frames in our dataset contain both horizontal texts (278 frames: 181 with English texts and 97 with Chinese texts) and nonhorizontal texts (188 frames: 140 English and 48 Chinese). Furthermore, the performance of the proposed system is compared with existing text detection methods by calculating measures manually and automatically to show usefulness of our semiautomatic system. The ground truth and the semiautomatic system will be released to the public.

This paper proposes an effective approach to model the emotional space of words to infer their Sense Sentiment Similarity (SSS). SSS reflects the distance between the words regarding their senses and underlying sentiments. We propose a probabilistic approach that is built on a hidden emotional model in which the basic human emotions are considered as hidden. This leads to predict a vector of emotions for each sense of the words, and then to infer the sense sentiment similarity. The effectiveness of the proposed approach is investigated in two Natural Language Processing tasks: Indirect yes/no Question Answer Pairs Inference and Sentiment Orientation Prediction.

In this paper we classify the temporal relations between pairs of events on an article-wide basis.This is in contrast to much of the existing literature which focuses on just event pairs which are found within the same or adjacent sentences.To achieve this, we leverage on discourse analysis as we believe that it provides more useful semantic information than typical lexico-syntactic features.We propose the use of several discourse analysis frameworks, including 1) Rhetorical Structure Theory (RST), 2) PDTB-styled discourse relations, and 3) topical text segmentation.We explain how features derived from these frameworks can be effectively used with support vector machines (SVM) paired with convolution kernels.Experiments show that our proposal is effective in improving on the state-of-the-art significantly by as much as 16% in terms of F 1 , even if we only adopt less-than-perfect automatic discourse analyzers and parsers.Making use of more accurate discourse analysis can further boost gains to 35%.

Lexical chains provide a representation of the lexical cohesion structure of a text.In this paper, we propose two lexical chain based cohesion models to incorporate lexical cohesion into document-level statistical machine translation: 1) a count cohesion model that rewards a hypothesis whenever a chain word occurs in the hypothesis, 2) and a probability cohesion model that further takes chain word translation probabilities into account.We compute lexical chains for each source document to be translated and generate target lexical chains based on the computed source chains via maximum entropy classifiers.We then use the generated target chains to provide constraints for word selection in document-level machine translation through the two proposed lexical chain based cohesion models.We verify the effectiveness of the two models using a hierarchical phrase-based translation system.Experiments on large-scale training data show that they can substantially improve translation quality in terms of BLEU and that the probability cohesion model outperforms previous models based on lexical cohesion devices.

This paper presents an approach to text recognition in natural scene images. Unlike most existing works which assume that texts are horizontal and frontal parallel to the image plane, our method is able to recognize perspective texts of arbitrary orientations. For individual character recognition, we adopt a bag-of-key points approach, in which Scale Invariant Feature Transform (SIFT) descriptors are extracted densely and quantized using a pre-trained vocabulary. Following [1, 2], the context information is utilized through lexicons. We formulate word recognition as finding the optimal alignment between the set of characters and the list of lexicon words. Furthermore, we introduce a new dataset called StreetViewText-Perspective, which contains texts in street images with a great variety of viewpoints. Experimental results on public datasets and the proposed dataset show that our method significantly outperforms the state-of-the-art on perspective texts of arbitrary orientations.

This paper presents a method for temporal integration, which can be used to improve the recognition accuracy of video texts. Given a word detected in a video frame, we use a combination of Stroke Width Transform and SIFT (Scale Invariant Feature Transform) to track it both backward and forward in time. The text instances within the word's frame span are then extracted and aligned at pixel level. In the second step, we integrate these instances into a text probability map. By thresholding this map, we obtain an initial binarization of the word. In the final step, the shapes of the characters are refined using the intensity values. This helps to preserve the distinctive character features (e.g., sharp edges and holes), which are useful for OCR engines to distinguish between the different character classes. Experiments on English and German videos show that the proposed method outperforms existing ones in terms of recognition accuracy.

This paper presents a two-stage method for multi-oriented video character segmentation. Words segmented from video text lines are considered for character segmentation in the present work. Words can contain isolated or non-touching characters, as well as touching characters. Therefore, the character segmentation problem can be viewed as a two stage problem. In the first stage, text cluster is identified and isolated (non-touching) characters are segmented. The orientation of each word is computed and the segmentation paths are found in the direction perpendicular to the orientation. Candidate segmentation points computed using the top distance profile are used to find the segmentation path between the characters considering the background cluster. In the second stage, the segmentation results are verified and a check is performed to ascertain whether the word component contains touching characters or not. The average width of the components is used to find the touching character components. For segmentation of the touching characters, segmentation points are then found using average stroke width information, along with the top and bottom distance profiles. The proposed method was tested on a large dataset and was evaluated in terms of precision, recall and f-measure. A comparative study with existing methods reveals the superiority of the proposed method.

In this paper, we address curved text detection in video through a new enhancement criterion and the use of quad tree. The proposed method makes use of the quad tree to simplify the task of handling the entire frame at each stage. The proposed method employs a novel criterion for grouping of pixels based on their R, G and B values to enhance text information. As generally, a text detection problem is a two class problem, we used k-means with k=2 to identify potential text candidate pixels. From these potential candidates, connected components are then extracted and subjected to further analysis, where symmetry property based on stroke width is used for further authentication of the text representatives. These authenticated text representatives are then exploited as seed points to restore the text information with reference to the Sobel edge frame of the original input frame. To preserve the spatial information of text pixels the concept of quad tree is applied. From these seed blocks, text lines are extracted by the use of a region growing approach driven completely based on Sobel edge map. The proposed method is tested on curved video data and Hua's horizontal video text data in terms of recall, precision, f-measure, misdetection rate and processing time. The results are compared and analyzed to show that the proposed method outperforms several existing methods in terms of accuracy and efficiency.

Document Image Binarization is a technique to segment text out from the background region of a document image, which is a challenging task due to high intensity variations of the document foreground and background. Recently, a series of document image binarization contests (DIBCOs) had been held that have drawn great research interest in this area. Several document binarization techniques have been proposed and achieve great performance on the contest datasets. However, those proposed techniques may not perform well on all kinds of degraded document images because it is difficult to design a classification method that correctly models the non-uniform degraded document background and text foreground simultaneously. In this paper, we propose a self learning classification framework that combines binary outputs of different binarization methods. The proposed framework makes used of the sparse representation to re-classify the document pixels and produces a better binary results. The experimental results on the recent DIBCO contests show the great performance and robustness of our proposed framework on different kinds of degraded document images.

Scene text recognition is a fundamental step in End-to-End applications where traditional optical character recognition (OCR) systems often fail to produce satisfactory results. This paper proposes a technique that uses co-occurrence histogram of oriented gradients (Co-HOG) to recognize the text in scenes. Compared with histogram of oriented gradients (HOG), Co-HOG is a more powerful tool that captures spatial distribution of neighboring orientation pairs instead of just a single gradient orientation. At the same time, it is more efficient compared with HOG and therefore more suitable for real-time applications. The proposed scene text recognition technique is evaluated on ICDAR2003 character dataset and Street View Text (SVT) dataset. Experiments show that the Co-HOG based technique clearly outperforms state-of-the-art techniques that use HOG, Scale Invariant Feature Transform (SIFT), and Maximally Stable Extremal Regions (MSER).

Character shape reconstruction for the scene character is challenging and interesting because scene character usually suffers from uneven illumination, complex background, perspective distortion. To address such ill conditions, we propose to utilize Histogram Gradient Division (HGD) and Reverse Gradient Orientation (RGO) to select Candidate Text Pixels (CTPs) for a given input character. Ring Radius Transform is applied on each pixel in a CTP image to obtain radius map where each pixel is assigned a value which is the radius to the nearest CTP. Candidate medial axis pixels are those having maximum radius values in their neighborhoods. We find such pixels on horizontal, vertical, principal diagonal and secondary diagonal directions to determine the respective medial axis pixels. The union of all medial axis pixels at each pixel location is considered as a candidate medial axis pixel of the character. Then color difference and k-means clustering are employed to eliminate false candidate medial axis. The potential medial axis values are used to reconstruct the shape of the character. The method is tested on 1025 characters of complex foreground and background from ICDAR 2003 dataset in terms of shape reconstruction and recognition rate. Experimental results demonstrate the effectiveness of our proposed method for complex foreground and background characters in terms of character recognition rate and reconstruction error.

Abstract—We propose a new segmentation-free method for keyword spotting in handwritten documents based on Heat Kernel Signature (HKS). After keypoints are located by the keypoint detector for SIFT on the document pages and the query image, HKS descriptors are extracted from a local patch centered at each keypoint. In order to locate the positions where the query image appears in document pages, we present a searching method which tries to locate a local zone which contains enough matching keypoints corresponding to the query image. Our method does not need any pre-processing steps. Keyword Spotting is a method used to spot query words in a large collection of documents. Unlike Optical Character Recognition (OCR), which tries to translate every character into ASCII text, keyword spotting always returns a ranking

We propose a method to try to model fashionable dresses in this paper. We first discover common visual patterns that appear in dress images using a human-in-the-loop, active clustering approach. A fashionable dress is expected to contain certain visual patterns which make it fashionable. An approach is proposed to jointly identify fashionable visual patterns and learn a discriminative fashion classifier. The results show that interesting fashionable patterns can be discovered on a newly collected dress dataset. Furthermore, our model can also achieve high accuracy on distinguishing fashionable and unfashionable dresses.

Computer aided diagnosis (CAD) in medical imaging is of growing interest in recent years. Our proposed CAD system aims to enhance diagnosis and prognosis of traumatic brain injury (TBI) patients with hematomas. Hematoma caused by blood vessel rupture is the major lesion in TBI cases and is usually assessed using head computed tomography (CT). In our CAD system, we segment the hematoma region from each slice of a CT series, extract features from the hematoma segments, and automatically classify the hematoma types using machine learning methods. We propose two sets of shape based features for each segmented hematoma region. The first set contains primitive features describing the overall shape of a hematoma region. The features in the second set are based on the dissimilarities of the shapes of hematoma regions measured by geodesic distances. After feature extraction, we classify the hematoma regions into three types -- epidural hematoma, sub-dural hematoma, and intracerebral hematoma, using random forest. Each tree of the random forest votes one class for each hematoma, and the random forest takes the class label with the majority votes for the hematoma. As hematomas are volumetric in nature, some hematomas are observed across several consecutive slices in the same CT series. For each class, we add the votes from each hematoma slice that comprises the volumetric hematoma in that class, then we take the class with the majority of the summed votes as the class label for that volumetric hematoma. The overall classification accuracies for hematoma region from each CT slice are 80.7%, 81.3%, and 81.1% using primitive features only, geodesic distance features only, or both sets of features, respectively. For volumetric hematoma classification, the overall accuracies are 80.9%, 81.5%, and 81.5% respectively. The results are promising to radiologists and neurosurgeons specialized in this field of research.

Text detection in videos is challenging due to low resolution and complex background of videos. Besides, an arbitrary orientation of scene text lines in video makes the problem more complex and challenging. This paper presents a new method that extracts text lines of any orientations based on gradient vector flow (GVF) and neighbor component grouping. The GVF of edge pixels in the Sobel edge map of the input frame is explored to identify the dominant edge pixels which represent text components. The method extracts edge components corresponding to dominant pixels in the Sobel edge map, which we call text candidates (TC) of the text lines. We propose two grouping schemes. The first finds nearest neighbors based on geometrical properties of TC to group broken segments and neighboring characters which results in word patches. The end and junction points of skeleton of the word patches are considered to eliminate false positives, which output the candidate text components (CTC). The second is based on the direction and the size of the CTC to extract neighboring CTC and to restore missing CTC, which enables arbitrarily oriented text line detection in video frame. Experimental results on different datasets, including arbitrarily oriented text data, nonhorizontal and horizontal text data, Hua's data and ICDAR-03 data (camera images), show that the proposed method outperforms existing methods in terms of recall, precision and f-measure.

Segmentation of text from badly degraded document images is a very challenging task due to the high inter/intra-variation between the document background and the foreground text of different document images. In this paper, we propose a novel document image binarization technique that addresses these issues by using adaptive image contrast. The adaptive image contrast is a combination of the local image contrast and the local image gradient that is tolerant to text and background variation caused by different types of document degradations. In the proposed technique, an adaptive contrast map is first constructed for an input degraded document image. The contrast map is then binarized and combined with Canny's edge map to identify the text stroke edge pixels. The document text is further segmented by a local threshold that is estimated based on the intensities of detected text stroke edge pixels within a local window. The proposed method is simple, robust, and involves minimum parameter tuning. It has been tested on three public datasets that are used in the recent document image binarization contest (DIBCO) 2009 & 2011 and handwritten-DIBCO 2010 and achieves accuracies of 93.5%, 87.8%, and 92.03%, respectively, that are significantly higher than or close to that of the best-performing methods reported in the three contests. Experiments on the Bickley diary dataset that consists of several challenging bad quality document images also show the superior performance of our proposed method, compared with other techniques.

Event coreference is an important task in event extraction and other natural language processing tasks. Despite its importance, it was merely discussed in previous studies. In this article, we present a global coreference resolution system dedicated to various sophisticated event coreference phenomena. First, seven resolvers are utilized to resolve different event and object coreference mention pairs with a new instance selection strategy and new linguistic features. Second, a global solution—a modified random walk partitioning—is employed for the chain formation. Being the first attempt to apply the random walk model for coreference resolution, the revised model utilizes a sampling method, termination criterion, and stopping probability to greatly improve the effectiveness of random walk model for event coreference resolution. Last but not least, the new model facilitates a convenient way to incorporate sophisticated linguistic constraints and preferences, the related object mention graph, as well as pronoun coreference information not used in previous studies for effective chain formation. In total, these techniques impose more than 20% F-score improvement over the baseline system.

This paper describes an emotion-based approach to acquire sentiment similarity of word pairs with respect to their senses. Sentiment similarity indicates the similarity between two words from their underlying sentiments. Our approach is built on a model which maps from senses of words to vectors of twelve basic emotions. The emotional vectors are used to measure the sentiment similarity of word pairs. We show the utility of measuring sentiment similarity in two main natural language processing tasks, namely, indirect yes/no question answer pairs (IQAP) Inference and sentiment orientation (SO) prediction. Extensive experiments demonstrate that our approach can effectively capture the sentiment similarity of word pairs and utilize this information to address the above mentioned tasks.

Text detection in video frames plays a vital role in enhancing the performance of information extraction systems because the text in video frames helps in indexing and retrieving video efficiently and accurately. This paper presents a new method for arbitrarily-oriented text detection in video, based on dominant text pixel selection, text representatives and region growing. The method uses gradient pixel direction and magnitude corresponding to Sobel edge pixels of the input frame to obtain dominant text pixels. Edge components in the Sobel edge map corresponding to dominant text pixels are then extracted and we call them text representatives. We eliminate broken segments of each text representatives to get candidate text representatives. Then the perimeter of candidate text representatives grows along the text direction in the Sobel edge map to group the neighboring text components which we call word patches. The word patches are used for finding the direction of text lines and then the word patches are expanded in the same direction in the Sobel edge map to group the neighboring word patches and to restore missing text information. This results in extraction of arbitrarily-oriented text from the video frame. To evaluate the method, we considered arbitrarily-oriented data, non-horizontal data, horizontal data, Hua's data and ICDAR-2003 competition data (Camera images). The experimental results show that the proposed method outperforms the existing method in terms of recall and f-measure.

Musical staff line detection and removal techniques detect the staff positions in musical documents and segment musical score from musical documents by removing those staff lines. It is an important preprocessing step for ensuing the Optical Music Recognition tasks. This paper proposes an effective staff line detection and removal method that makes use of the global information of the musical document and models the staff line shape. It first estimates the staff height and space, and then models the shape of the staff line by examining the orientation of the staff pixels. At last the estimated model is used to find out the location of staff lines and hence to remove those detected staff lines. The proposed technique is simple, robust, and involves few parameters. It has been tested on the dataset of the recent staff removal competition held under the International Conference of Document Analysis and Recognition(ICDAR) 2011. Experimental results show the effectiveness and robustness of our proposed technique on musical documents with various types of deformations.

In this paper, we present new features based on Spatial-Gradient-Features (SGF) at block level for identifying six video scripts namely, Arabic, Chinese, English, Japanese, Korean and Tamil. This works helps in enhancing the capability of the current OCR on video text recognition by choosing an appropriate OCR engine when video contains multi-script frames. The input for script identification is the text blocks obtained by our text frame classification method. For each text block, we obtain horizontal and vertical gradient information to enhance the contrast of the text pixels. We divide the horizontal gradient block into two equal parts as upper and lower at the centroid in the horizontal direction. Histogram on the horizontal gradient values of the upper and the lower part is performed to select dominant text pixels. In the same way, the method selects dominant pixels from the right and the left parts obtained by dividing the vertical gradient block vertically. The method combines the horizontal and the vertical dominant pixels to obtain text components. Skeleton concept is used to reduce pixel width to a single pixel to extract spatial features. We extract four features based on proximity between end points, junction points, intersection points and pixels. The method is evaluated on 770 frames of six scripts in terms of classification rate and is compared with an existing method. We have achieved 82.1% average classification rate.

In this paper, we propose to organize web images by actively creating visual clusters via crowd sourcing. We develop a two-phase framework to efficiently and effectively combine computers and a large number of human workers to build high quality visual clusters. The first phase partitions an image collection into multiple clusters, the second phase refines each generated cluster independently. In both phases, informative images are selected by computers and manually labeled by the crowds to learn improved models. Our method can be naturally extended to discover object categories in a collection of image segments. Experimental results on several data sets demonstrate the promise of our developed approach on both web image organization and object discovery tasks.

Automatically annotating words for images is a key to semantic-level image retrieval. Recently, several embedding learning based methods achieve good performance in this task which inspires this paper. Here we propose a novel word embedding model in which both images and words can be represented in the same embedding space. The embedding space is learnt in a discriminative nearest neighbor manner such that the annotation information could be propagated among neighbors. In order to accelerate model learning and testing, approximate-nearest-neighbor search is performed, and word embedding space is learnt in a stochastic manner. The experimental results demonstrate the effectiveness of the proposed method.

Implicit discourse relation classification is a challenge task due to missing discourse connective. Some work directly adopted machine learning algorithms and linguistically informed features to address this task. However, one interesting solution is to automatically predict implicit discourse connective. In this paper, we present a novel two-step machine learning-based approach to implicit discourse relation classification. We first use machine learning method to automatically predict the discourse connective that can best express the implicit discourse relation. Then the predicted implicit discourse connective is used to classify the implicit discourse relation. Experiments on Penn Discourse Treebank 2.0 (PDTB) and Biomedical Discourse Relation Bank (BioDRB) show that our method performs better than the baseline system and previous work.

The problem of text detection in natural scene images is challenging because of the unconstrained sizes, colors, backgrounds and alignments of the characters. This paper proposes novel symmetry features for this task. Within a text line, the intra-character symmetry captures the correspondence between the inner contour and the outer contour of a character while the inter-character symmetry helps to extract information from the gap region between two consecutive characters. A formulation based on Gradient Vector Flow is used to detect both types of symmetry points. These points are then grouped into text lines using the consistency in sizes, colors, and stroke and gap thickness. Therefore, unlike most existing methods which use only character features, our method exploits both the text features and the gap features to improve the detection result. Experimentally, our method compares well to the state-of-the-art on public datasets for natural scenes and street-level images, an emerging category of image data. The proposed technique can be used in a wide range of multimedia applications such as content-based image/video retrieval, mobile visual search and sign translation.

Multioriented text detection in video frames is not as easy as detection of captions or graphics or overlaid texts, which usually appears in the horizontal direction and has high contrast compared to its background. Multioriented text generally refers to scene text that makes text detection more challenging and interesting due to unfavorable characteristics of scene text. Therefore, conventional text detection methods may not give good results for multioriented scene text detection. Hence, in this paper, we present a new enhancement method that includes the product of Laplacian and Sobel operations to enhance text pixels in videos. To classify true text pixels, we propose a Bayesian classifier without assuming a priori probability about the input frame but estimating it based on three probable matrices. Three different ways of clustering are performed on the output of the enhancement method to obtain the three probable matrices. Text candidates are obtained by intersecting the output of the Bayesian classifier with the Canny edge map of the input frame. A boundary growing method is introduced to traverse the multioriented scene text lines using text candidates. The boundary growing method works based on the concept of nearest neighbors. The robustness of the method has been tested on a variety of datasets that include our own created data (nonhorizontal and horizontal text data) and two publicly available data, namely, video frames of Hua and complex scene text data of ICDAR 2003 competition (camera images). Experimental results show that the performance of the proposed method is encouraging compared with results of existing methods in terms of recall, precision, F-measures, and computational times.

We propose Tree Sequence Kernel (TSK), which implicitly exhausts the structure features of a sequence of subtrees embedded in the phrasal parse tree. By incorporating the capability of sequence kernel, TSK enriches tree kernel with tree sequence features so that it may provide additional useful patterns for machine learning applications. Two approaches of penalizing the substructures are proposed and both can be accomplished by efficient algorithms via dynamic programming. Evaluations are performed on two natural language tasks, i.e. Question Classification and Relation Extraction. Experimental results suggest that TSK outperforms tree kernel for both tasks, which also reveals that the structure features made up of multiple subtrees are effective and play a complementary role to the single tree structure.

Opinion question answering (QA) requires automatic and correct interpretation of an answer relative to its question. However, the ambiguity that often exists in the question-answer pairs causes complexity in interpreting the answers. This paper aims to infer yes/no answers from indirect yes/no question-answer pairs (IQAPs) that are ambiguous due to the presence of ambiguous sentiment adjectives. We propose a method to measure the uncertainty of the answer in an IQAP relative to its question. In particular, to infer the yes or no response from an IQAP, our method employs antonyms, synonyms, word sense disambiguation as well as the semantic association between the sentiment adjectives that appear in the IQAP. Extensive experiments demonstrate the effectiveness of our method over the baseline.

In this paper, we present a new method for video script identification which is essential before choosing an appropriate OCR engine for identifying text lines when a video frame contains more than one language. The input for script identification is the text lines obtained by our text detection method. We extract upper and lower extreme points for each connected component of Canny edges of text lines. The extracted points are connected to study the behavior of upper and lower lines. The direction of each 10-pixel segment of the lines is determined using PCA. The average angle of the segments of the upper and lower lines is computed to study the smoothness and cursiveness of the lines. In addition, to discriminate the scripts accurately, the method divides a text line into five equal zones horizontally to study the smoothness and cursiveness of the upper and lower lines of each zone. We evaluate the method by conducting experiments on different combinations of languages such as English and Chinese, English and Tamil, Chinese and Tamil, and English, Chinese and Tamil.

In this paper, we propose a method based on gradient vector flow for video character segmentation. By formulating character segmentation as a minimum cost path finding problem, the proposed method allows curved segmentation paths and thus it is able to segment overlapping characters and touching characters due to low contrast and complex background. Gradient vector flow is used in a new way to identify candidate cut pixels. A two-pass path finding algorithm is then applied where the forward direction helps to locate potential cuts and the backward direction serves to remove the false cuts, i.e. those that go through the characters, while retaining the true cuts. Experimental results show that the proposed method outperforms an existing method on multi-oriented English and Chinese video text lines. The proposed method also helps to improve binarization results, which lead to a better character recognition rate.

This paper presents a new method based on Fourier and moments features to extract words and characters from a video text line in any direction for recognition. Unlike existing methods which output the entire text line to the ensuing recognition algorithm, the proposed method obtains each extracted character from the text line as input to the recognition algorithm because the background of a single character is relatively simple compared to the text line and words. Max-Min clustering criterion is introduced to obtain text cluster from the extracted Fourier and moments feature set. Union of the text cluster with Canny operation of the input video text line is proposed to obtain missing text candidates. Then a run length criterion is used for extraction of words. From the words, we propose a new idea for extracting characters from the text candidates of each word image based on the fact that the text height difference at the character boundary column is smaller than that at other columns of the word image. We evaluate the method on a large dataset at three levels namely text line, words and characters in terms of recall, precision and f-measure. In addition to this, we show that the recognition result for the extracted character is better than words and lines. Our experimental set up involves 3527 characters including Chinese. The dataset is selected from TRECVID database of 2005 and 2006.

The current OCR cannot segment words and characters from video images due to complex background as well as low resolution of video images. To have better accuracy, this paper presents a new gradient based method for words and character segmentation from text line of any orientation in video frames for recognition. We propose a Max-Min clustering concept to obtain text cluster from the normalized absolute gradient feature matrix of the video text line image. Union of the text cluster with the output of Canny operation of the input video text line is proposed to restore missing text candidates. Then a run length algorithm is applied on the text candidate image for identifying word gaps. We propose a new idea for segmenting characters from the restored word image based on the fact that the text height difference at the character boundary column is smaller than that of the other columns of the word image. We have conducted experiments on a large dataset at two levels (word and character level) in terms of recall, precision and f-measure. Our experimental setup involves 3527 characters of English and Chinese, and this dataset is selected from TRECVID database of 2005 and 2006.

We present a new video character recognition method based on hierarchical classification. In the first step, we propose a method for character segmentation of the text line detected by the text detection method. The segmentation algorithm uses dynamic programming to find least-cost paths in the gray domain to identify the spaces between characters. For the segmented characters, we get a Canny edge image as input for the character recognition step. We introduce hierarchical classification based on voting criteria with structural features to classify 62 character classes into different smaller classes. We divide the perimeter of a character into 8 segments according to 8 directions at the centroid. Then the shape of each segment is studied to recognize the characters based on distances between the centroid and end points, and distances between the midpoint and end points. Our experiments on 1462 characters of upper case, lower case and numerals shows that 10% samples per class for training is enough to obtain 94.5% recognition accuracy. The dataset is chosen from TRECVID database of 2005 and 2006.

Web has become increasingly oriented to multimedia content. Most information on the web is conveyed from images. Text localization in web image plays an important role in web image information extraction and retrieval. Current works on text localization in web images assume that text regions are in homogenous color and high contrast. Hence, the approaches may fail when text regions are in multi-color or imposed in complex background. In this paper, we propose a text extraction algorithm from web images based on the probabilistic candidate selection model. The model firstly segments text region candidates from input images using wavelet, Gaussian mixture model (GMM) and triangulation. The likelihood of a candidate region containing text is then learnt using a Bayesian probabilistic model from two features, namely, histogram of oriented gradient (HOG) and local binary pattern histogram Fourier feature (LBP-HF). Finally best candidate regions are integrated to form text regions. The algorithm is evaluated using 155 non-homogenous web images containing around 600 text regions. The results show that the proposed model is able to extract text regions from non-homogenous images effectively.

Document image binarization has been studied for decades, and many practical binarization techniques have been proposed for different kinds of document images. However, many state-of-the-art methods are particularly suitable for the document images that suffer from certain specific type of image degradation or have certain specific type of image characteristics. In this paper, we propose a classification framework to combine different thresholding methods and produce better performance for document image binarization. Given the binarization results of some reported methods, the proposed framework divides the document image pixels into three sets, namely, foreground pixels, background pixels and uncertain pixels. A classifier is then applied to iteratively classify those uncertain pixels into foreground and background, based on the pre-selected froeground and background sets. Extensive experiments over different datasets including the Document Image Binarization Contest(DIBCO)2009 and Handwritten Document Image Binarization Competition(H-DIBCO)2010 show that our proposed framework outperforms most state-of-the-art methods significantly.

This paper presents a fully automatic framework for the restoration of double-sided historical manuscripts which are impaired by ink bleed-through distortions. First, the recto side and the verso side of a manuscript are precisely registered with a non-rigid registration method. The registration procedure selects salient control points as features, employs a free-form transformation model as mapping function and uses residual complexity as similarity measure. Next, we extract prominent foreground feature images with the two registered images. By referring to the feature images, we iteratively perform wavelet decomposition and construction to smear bleed-through interferences and to enhance foreground texts on the manuscript. We test the proposed framework with real manuscripts from the archives and the experimental results are encouraging.

Automatic medical image classification is difficult because of the lacking of training data. As manual labeling is too costly, we provide an automatic labeling solution to this problem by making use of the radiology report associated with the medical images. We first segment and reconstruct the 3D regions of interest (ROIs) from the medical images, and extract pathology and anatomy information from the associated report. We use an anatomical atlas to map the ROIs to the anatomy part(s) and match the pathology information of the same anatomy part(s) from the text. In this way, the ROIs are automatically labeled with pathology types which can be served as class labels, and a training data set of a large number of training instances is generated automatically. We extract the volume, color, location, and shape features of the ROIs, and classify the types of ROIs using these features. The overall evaluation result is promising to doctors and medical professionals. Our experiment is conducted using traumatic brain injury CT images; however, our framework of automatically labeling and classifying medical cases can be extended to medical images in other modality or of other anatomical part.

10.5591/978-1-57735-516-8/IJCAI11-319

Many digital images contain blurred regions which are caused by motion or defocus. Automatic detection and classification of blurred image regions are very important for different multimedia analyzing tasks. This paper presents a simple and effective automatic image blurred region detection and classification technique. In the proposed technique, blurred image regions are first detected by examining singular value information for each image pixels. The blur types (i.e. motion blur or defocus blur) are then determined based on certain alpha channel constraint that requires neither image deblurring nor blur kernel estimation. Extensive experiments have been conducted over a dataset that consists of 200 blurred image regions and 200 image regions with no blur that are extracted from 100 digital images. Experimental results show that the proposed technique detects and classifies the two types of image blurs accurately. The proposed technique can be used in many different multimedia analysis applications such as image segmentation, depth estimation and information retrieval.

In this paper, we propose a method based on the Laplacian in the frequency domain for video text detection. Unlike many other approaches which assume that text is horizontally-oriented, our method is able to handle text of arbitrary orientation. The input image is first filtered with Fourier-Laplacian. K-means clustering is then used to identify candidate text regions based on the maximum difference. The skeleton of each connected component helps to separate the different text strings from each other. Finally, text string straightness and edge density are used for false positive elimination. Experimental results show that the proposed method is able to handle graphics text and scene text of both horizontal and nonhorizontal orientation.

Wavelet kernels have been introduced for both support vector regression and classification. Most of these wavelet kernels do not use the inner product of the embedding space, but use wavelets in a similar fashion to radial basis function kernels. Wavelet analysis is typically carried out on data with a temporal or spatial relation between consecutive data points. We argue that it is possible to order the features of a general data set so that consecutive features are statistically related to each other, thus enabling us to interpret the vector representation of an object as a series of equally or randomly spaced observations of a hypothetical continuous signal. By approximating the signal with compactly supported basis functions and employing the inner product of the embedding L2 space, we gain a new family of wavelet kernels. Empirical results show a clear advantage in favor of these kernels.

In this paper, a novel approach for detection of text and non-text regions in video frames is proposed. The proposed approach performs block wise eigen analysis on the gradient image of the video frame. For each block of the gradient frame, the dominant eigen value is computed to decide if the block could be a candidate text block. The K-means clustering is then applied to further identify text blocks among the candidate blocks. From each of the identified candidate text blocks edges are extracted using the sobel operator, and then by the use of horizontal and vertical profiles a bounding rectangle is fixed up. Further, geometric properties of the identified text regions are studied to eliminate false text regions. In order to validate the efficacy of the proposed approach, experimentation on a dataset containing 800 video frames has been carried out. The obtained results ensure that the proposed approach is with increased text detection rate with very low false and misdetection rates when compared to the other existing state of the art techniques.

Patent document images maintained by the U.S. patent database have a specific format, in which figures and text descriptions are separated into different sections. This makes it difficult for users to refer to a figure while reading the description or vice versa. This paper introduces a method to associate figures with corresponding description paragraphs, and thus help to make patent documents more friendly for users to browse. In this method, after extracting individual figures out of the drawing section, figures and relevant descriptions are associated by evaluating the similarity between the text content of figures and description paragraphs using vector space model.

In this paper, we propose a method based on the skeletonization operation for multi-oriented video text detection. The first step uses our existing Laplacian-based method to identify candidate text regions. In the second step, each region is classified as either a simple connected component (a single text string) or a complex connected component (multiple text strings that are connected to each other) depending on the number of intersection points in its skeleton. Complex connected components are then segmented into constituent parts based on the skeleton segments in order to separate the text strings from each other. Finally, text string straightness and edge density are used for false positive elimination. Experimental results show that the proposed method is able to detect multi-oriented graphics text and scene text.

In this paper, we present a new method based on wavelet-median-moments and a novel idea of angle projection for detecting multi-oriented text in video. The proposed method uses wavelet decomposition first to obtain three high frequency sub-bands (LH, HL and HH) and then median moments are computed on the average sub-bands of the three high frequency sub-bands to brighten the text pixels. K-means clustering (K=2) is used for obtaining text pixels from the wavelet-median-moments features (WMMF). Text candidates are obtained by mapping the output of K-means on Sobel edge map of the original input frame. To deal with multi-oriented text, we introduce a new idea of Angle Projection (AP) based on boundary growing and nearest neighbor concepts from the text candidates instead of conventional projection profiles. The proposed method is experimented on horizontal text data, non-horizontal text data, temporal data, non-text data and camera based images (scene text data of ICDAR 2003 competition) to show that the proposed method is superior to existing methods.

This paper presents a new document image binarization technique that segments the text from badly degraded historical document images. The proposed technique makes use of the image contrast that is defined by the local image maximum and minimum. Compared with the image gradient, the image contrast evaluated by the local maximum and minimum has a nice property that it is more tolerant to the uneven illumination and other types of document degradation such as smear. Given a historical document image, the proposed technique first constructs a contrast image and then detects the high contrast image pixels which usually lie around the text stroke boundary. The document text is then segmented by using local thresholds that are estimated from the detected high contrast pixels within a local neighborhood window. The proposed technique has been tested over the dataset that is used in the recent Document Image Binarization Contest (DIBCO) 2009. Experiments show its superior performance.

In this paper, we propose a novel post processing approach for on-line handwriting recognition. Differing from the existing linguistic knowledge-based methods, we make use of domain specific knowledge to improve the performance of recognition. Our system recognizes doctor's handwriting which often poses great challenges in readability, and then enhances the quality of recognized text by analyzing and restoring the text with a medical knowledge model. We show experiments with this approach on a set of medical handwriting data provided by the doctor, and the results are promising.

There are many video images where hand written text may appear. Therefore handwritten scene text detection in video is essential and useful for many applications for efficient indexing, retrieval etc. Also there are many video frames where text line may be multi-oriented in nature. To the best of our knowledge there is no work on handwritten text detection in video, which is multi-oriented in nature. In this paper, we present a new method based on maximum color difference and boundary growing method for detection of multi-oriented handwritten scene text in video. The method computes maximum color difference for the average of R, G and B channels of the original frame to enhance the text information. The output of maximum color difference is fed to a K-means algorithm with K = 2 to separate text and non-text clusters. Text candidates are obtained by intersecting the text cluster with the Sobel output of the original frame. To tackle the fundamental problem of different orientations and skews of handwritten text, boundary growing method based on a nearest neighbor concept is employed. We evaluate the proposed method by testing on our own handwritten text database and publicly available video data (Hua's data). Experimental results obtained from the proposed method are promising.

Computed tomography (CT) is used widely in traumatic brain injury diagnosis. One axial brain CT scan consists of multiple slices with different heights along the brain axial direction. Indexing of brain CT slices is to order the slices and align each individual slice onto the corresponding brain axial height, which is an important step in content-based image retrieval and computer-assisted diagnosis. Current existing methods for this indexing task are through the image registration techniques by registering 2D image slices onto a 3D brain atlas. In this paper, instead of using the registration methods, we propose a fast indexing method using anatomical feature classification. In our method, the brain CT scan is divided into 6 height levels along the axial direction so that slices in each level share similar anatomical structure. In this way, the indexing problem becomes a classification problem that one series of scan slices are to be classified into 6 classes. Experimental results show that the proposed method is effective and efficient.

Large number of medical images are produced daily in hospitals and medical institutions, the needs to efficiently process, index, search and retrieve these images are great. In this paper, we propose a pathology based medical image annotation framework using a statistical machine translation approach. After pathology terms and regions of interest (ROIs) are extracted from training text and images respectively, we use machine translation model IBM Model 1 to iteratively learn the alignment between the ROIs and the pathology terms and generate an ROI-to-pathology translation table. In testing phase, we annotate the ROI in the image with the pathology label of the highest probability in the translation table. The overall annotation results and the retrieval performance are promising to doctors and medical professionals.

This paper proposes the use of a new symmetry property based on proximity of the median moments in the wavelet domain. The method divides a given frame into 16 equally sized blocks to classify the true text frame. The average of high frequency subbands of a block is used for computing median moments to brighten the text pixel in a block of video frame. Then K-means clustering with K=2 is applied on the median moments of the block to classify it as a probable text block. For classified blocks, average wavelet median moments are computed for a sliding window. We introduce Max-Min cluster to classify the probable text pixel in each probable text block. The four quadrants are formed from the centroid of the probable text pixels. The new concept called symmetry is introduced to identify the true text block based on proximity between probable text pixels in each quadrant. If the frame produces at least one true text block, it is considered as a text frame otherwise a non-text frame. The method is tested on three datasets to evaluate the robustness of the method in classification of text frames in terms of recall and precision.

Automatic text detection in video is an important task for efficient and accurate indexing and retrieval of multimedia data such as events identification, events boundary identification etc. This paper presents a new method comprising of wavelet decomposition and color features namely R, G and B. The wavelet decomposition is applied on three color bands separately to obtain three high frequency sub bands (LH, HL and HH) and then the average of the three sub bands for each color band is computed further to enhance the text pixels in video frame. To take advantage of wavelet and color information, we again take the average of the three average images (AoA) obtained by the former step to increase the gap between text and non text pixels. Our previous Laplacian method is employed on AoA for text detection. The proposed method is evaluated by testing on a large dataset which includes publicly available data, non text data and ICDAR-03 data. Comparative study with existing methods shows that the results of the proposed method are encouraging and useful.

Text frame classification is needed in many applications such as event identification, exact event boundary identification, navigation, video surveillance in multimedia etc. To the best of our knowledge, there are no methods reported solely dedicated to text frame classifications so far. Hence this paper presents a new approach to text frame classification in video based on capturing local observable edge properties of text frames, by virtue of the strong presence of sharp edges, straight appearances of edges and consistent proximity between edges. The approach initially classifies the blocks of the frame into text blocks and non-text blocks. The true text block is then identified among classified text blocks to detect text frames by the proposed features. If the text frame produces one true text block then it is considered as a text frame otherwise a non-text frame. We evaluate the proposed approach on a large database containing both text and nontext frames and publicly available data at two levels, i.e., estimating recall and precision at the block level and the frame level.

Document Image Binarization techniques have been studied for many years, and many practical binarization techniques have been developed and applied successfully on commercial document analysis systems. However, the current state-of-the-art methods, fail to produce good binarization results for many badly degraded document images. In this paper, we propose a self-training learning framework for document image binarization. Based on reported binarization methods, the proposed framework first divides document image pixels into three categories, namely, foreground pixels, background pixels and uncertain pixels. A classifier is then trained by learning from the document image pixels in the foreground and background categories. Finally, the uncertain pixels are classified using the learned pixel classifier. Extensive experiments have been conducted over the dataset that is used in the recent Document Image Binarization Contest (DIBCO) 2009. Experimental results show that our proposed framework significantly improves the performance of reported document image binarization methods.

This paper presents a non-rigid registration method for the restoration of double-sided historical manuscripts. Firstly, the gradient direction maps of the two images of a manuscript are examined to identify candidate control points. Then the correspondences of these points are established by minimizing a disimilarity measure consisting of intensity, gradient and displacement. To fully capture the spatial relationship between the two images, a mapping function is defined as the combination of a global affine and local b-splines transformation. The cost function for optimization consists of two parts: normalized mutual information for the goal of similarity and space integral of the square of the second order derivatives for smoothness. To evaluate the proposed method, a wavelet based restoration procedure is applied to registered images. Real documents from the National Archives of Singapore are used for testing and the experimental results are impressive.

This paper introduces a binarization method based on edge for video text images, especially for images with complex background or low contrast. The binarization method first detects the contour of the text, and utilizes a local thresholding method to decide the inner side of the contour, and then fills up the contour to form characters that are recognizable to OCR software. Experiment results show that our method is especially effective on complex background and low contrast images.

In recent years, with the rapid proliferation of digital images, the need to search and retrieve the images accurately, efficiently, and conveniently is becoming more acute. Automatic image annotation with image semantic content has attracted increasing attention, as it is the preprocess of annotation based image retrieval which provides users accurate, efficient, and convenient image retrieval with image understanding. Different machine learning approaches have been used to tackle the problem of automatic image annotation; however, most of them focused on exploring the relationship between images and annotation words and neglected the relationship among the annotation words. In this paper, we propose a framework of using language models to represent the word-to-word relation and thus to improve the performance of existing image annotation approaches utilizing probabilistic models. We also propose a specific language model - the semantic similarity language model to estimate the semantic similarity among the annotation words so that annotations that are more semantically coherent will have higher probability to be chosen to annotate the image. To illustrate the general idea of using language model to improve current image annotation systems, we added the language model on top of the two specific image annotation models - the translation model (TM) and the cross media relevance model (CMRM). We tested the improved models on a widely used image annotation corpus - the Corel 5K dataset. Our results show that by adding the semantic similarity language model, the performance of image annotation improves significantly in comparison with the original models. Our proposed language model can also be applied to other image annotation approaches using word probability conditioned on image or word-image joint probability as well.

A common problem encountered in recognizing real-scene symbols is the perspective deformation. In this paper, a recognition method resistant to perspective deformation is proposed, based on Cross-Ratio Spectrum descriptor. This method shows good resistance to severe perspective deformation and good discriminating power to similar symbols.

In this paper, we propose new Fourier-statistical features (FSF) in RGB space for detecting text in video frames of unconstrained background, different fonts, different scripts, and different font sizes. This paper consists of two parts namely automatic classification of text frames from a large database of text and non-text frames and FSF in RGB for text detection in the classified text frames. For text frame classification, we present novel features based on three visual cues, namely, sharpness in filter-edge maps, straightness of the edges, and proximity of the edges to identify a true text frame. For text detection in video frames, we present new Fourier transform based features in RGB space with statistical features and the computed FSF features from RGB bands are subject to K-means clustering to classify text pixels from the background of the frame. Text blocks of the classified text pixels are determined by analyzing the projection profiles. Finally, we introduce a few heuristics to eliminate false positives from the frame. The robustness of the proposed approach is tested by conducting experiments on a variety of frames of low contrast, complex background, different fonts, and sizes of text in the frame. Both our own test dataset and a publicly available dataset are used for the experiments. The experimental results show that the proposed approach is superior to existing approaches in terms of detection rate, false positive rate, and misdetection rate.

With large databases of document images available,a method for users to find keywords in documents will be useful. One approach is to perform Optical Character Recognition (OCR) on each document followed by indexing of the resulting text. However, if the quality of the document is poor or time is critical,complete OCR of all images is infeasible. This paper build upon previous works on Word Shape Coding to propose an alternative technique and combination of feature descriptors for keyword spotting without the use of OCR. Different sequence alignment similarity measures can be used for partial or whole word matching. The proposed technique is tolerant to serifs,font styles and certain degrees of touching, broken or overlapping characters. It improves over previous works with not only better precision and lower collision rate, but more importantly, the ability for partial matching. Experiment results show that it is about 15 times faster than OCR. It is a promising technique to boost better document image retrieval.

In this paper, we propose an efficient text detection method based on the Laplacian operator. The maximum gradient difference value is computed for each pixel in the Laplacian-filtered image. K-means is then used to classify all the pixels into two clusters: text and non-text. For each candidate text region, the corresponding region in the Sobel edge map of the input image undergoes projection profile analysis to determine the boundary of the text blocks. Finally, we employ empirical rules to eliminate false positives based on geometrical properties. Experimental results show that the proposed method is able to detect text of different fonts, contrast and backgrounds. Moreover, it outperforms three existing methods in terms of detection and false positive rates.

Text detection in video images has received increasing attention, particularly in scene text detection in video images, as it plays a vital role in video indexing and information retrieval. This paper proposes a new and robust gradient difference technique for detecting both graphics and scene text in video images. The technique introduces the concept of zero crossing to determine the bounding boxes for the detected text lines in video images, rather than using the conventional projection profiles based method which fails to fix bounding boxes when there is no proper spacing between the detected text lines. We demonstrate the capability of the proposed technique by conducting experiments on video images containing both graphics text and scene text with different font shapes and sizes, languages, text directions, background and contrasts. Our experimental results show that the proposed technique outperforms existing methods in terms of detection rate for large video image database.

In this paper, we propose a new method based on wavelet transform, statistical features and central moments for both graphics and scene text detection in video images. The method uses wavelet single level decomposition LH, HL and HH subbands for computing features and the computed features are fed to k means clustering to classify the text pixel from the background of the image. The average of wavelet subbands and the output of k means clustering helps in classifying true text pixel in the image. The text blocks are detected based on analysis of projection profiles. Finally, we introduce a few heuristics to eliminate false positives from the image. The robustness of the proposed method is tested by conducting experiments on a variety of images of low contrast, complex background, different fonts, and size of text in the image. The experimental results show that the proposed method outperforms the existing methods in terms of detection rate, false positive rate and misdetection rate.

Image registration is crucial for various image analysis tasks. In particular, most approaches to correction of bleed-through distortion on handwritten document images require the recto image and the verso image to be precisely registered. In this paper, we present a fully automatic method which detects specific number of corresponding control points from historical documents for the purpose of registration. First, candidate points are located by inspecting the gradient direction maps of document images. Corresponding control points are selected based on a dissimilarity metric that incorporates image intensity, gradient magnitude, gradient orientation and displacement. To improve the quality of the detected control points, median filers and consistency checking are applied to correct mismatches. Experiments on real historical document images have shown encouraging results and further improvements can be made by exploiting more sophisticated similarity metric tailored to historical documents' characteristics.

Perspective deformation is one of the main issues needed to be addressed in real-scene character recognition. An effective recognition approach, which is able to handle severe perspective deformation, is to employ cross ratio spectrum and dynamic time warping techniques. However, this solution suffers from a time complexity of O(n4). In this paper, a clustering based indexing method is proposed to index cross ratio spectra and thus expedite the recognition. Cross ratio spectra of all templates are clustered. A query is compared with the centroid of each cluster instead of spectra of all templates. Our method is 40 times faster than the previous method, and has archived about 15-time speed up while preserving almost the same recognition accuracy in the real scene character recognition experiment.

In intracranial pathological examinations using CT scan, brain midline shift (MLS) is an important diagnostic feature indicating the pathological severity and patient's survival possibility. In this paper, we develop a new method of tracing the brain midline shift in traumatic brain injury (TBI) CT images using its original cause - the hemorrhage. Firstly, we model the relationship between the hemorrhage and the midline deformation caused by it using a linear regression model (H-MLS model). Secondly, using the H-MLS model, the deformed midline is predicted from the hemorrhage detected in CT images. Finally, the predicted deformed midline is adjusted according to the visual symmetry information. Preliminary experiments show that the proposed method is effective and time-efficient.

Text detection plays a vital role in retrieving and browsing video data efficiently and accurately. In this paper, we propose a method for detecting both graphics and scene text in video images by proposing initial text block identification, text portion segmentation and new edge features for false positive elimination. The heuristic rules based on filters and edge analysis are formed to identify the initial text block and to segment the complete text portion from the image. The new edge features such as straightness and cursiveness are explored to eliminate false positives. To evaluate the performance of the proposed method, we introduce misdetection rate and processing time in addition to detection rate and false positive rate. The experimental results show that the proposed method outperforms existing methods in terms of the above metrics.

This paper describes an attempt to devise a knowledge discovery model that is inspired from the two theoretical frameworks of selectionism and constructivism in human cognitive learning. The "selectionist" nature of human decision making indicates the use of an evolutionary paradigm for composing rudimentary neural network units, while the "constructivist" component takes the form of neural weight training during the learning process. We explore the possibility of amalgamating these two ideas into a neural learning system for the discovery of meaningful rules in the context of pattern discovery in data.

CpG islands (CGIs) play a fundamental role in genome analysis as genomic markers and tumor markers. Identification of potential CGIs has contributed not only to the prediction of promoters of most house-keeping genes and many tissue-specific genes but also to the understanding of the epigenetic causes of cancer. The most current methods for identifying CGIs suffered from various limitations and involved a lot of human intervention for search purpose. In this paper, we implement a HMM-based CGIs identification system, namely CpG-Discover. Experiments have been conducted on the EMBL human DNA database and in comparison with other widely-used tools. The controlled experimental results indicate that our system is a promising tool and has the capability of locating CGIs accurately. In addition, our system has significant differences from other tools in that it avoids the disadvantages of using sliding windows and it reduces the large amount of human intervention needed to search for or to combine potential CGIs (such as, the thresholds of initial density or distance seed). Therefore, given annotated training data set, our system has the adaptability to find other specific nucleotides sequences in DNA.

Text classification by support vector machines can benefit from semantic smoothing kernels that regard semantic relations among index terms while computing similarity. Adding expansion terms to the vector representation can also improve effectiveness. However, existing semantic smoothing kernels do not employ term expansion. This paper proposes a new non-linear kernel for text classification to exploit semantic relations between terms to add weighted expansion terms.

In vector space model (VSM), text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a computer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the performance of text categorization. In this study, we investigate several widely-used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with SVM and kappa NN algorithms. In consideration of the distribution of relevant documents in the collection, we propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms' discriminating power for text categorization task. From the controlled experimental results, these supervised term weighting methods have mixed performance. Specifically, our proposed supervised term weighting method, tf.rf, has a consistently better performance than other term weighting methods while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets.

Patent document images maintained by the U.S. patent database have a specific format, in which figures and text descriptions are separated into different sections. This makes it difficult for users to refer to a figure while reading the description or vice versa. The system introduced in this paper is to prepare these patent images for a friendly user browsing interface. The system is able to extract captions and labels from figures. After obtaining captions and labels, figures and the relevant descriptions are linked together. Hence, users are able to easily find the relevant figure by clicking captions or labels in the description, or vice versa.

Both graphic text and scene text detection in video images with complex background and low resolution is still a challenging and interesting problem for researchers in the field of image processing and computer vision. In this paper, we present a novel technique for detecting both graphic text and scene text in video images by finding segments containing text in an input image and then using statistical features such as vertical and horizontal bars for edges in the segments for detecting true text blocks efficiently. To identify a segment containing text, heuristic rules are formed based on combination of filters and edge analysis. Furthermore, the same rules are extended to grow the boundaries of a candidate segment in order to include complete text in the input image. The experimental results of the proposed method show that the technique performs better than existing methods in terms of a number of metrics.

Double-sided manuscripts are often degraded by bleed-through interference. Such degradation must be corrected to facilitate human perception and machine recognition. Most approaches to bleed-through removal rely on perfect alignment between the recto and verso images of a document. This paper presents a two-stage hierarchical alignment technique that can efficiently and accurately align the two sides of a document. Our approach first coarsely aligns the two images using a pair of anchors extracted from the recto and verso images respectively. The coarsely aligned images are then precisely aligned using block matching and radial basis function (RBF) based interpolation techniques. To evaluate the proposed alignment technique, we build a classification and recovery system to remove bleed-through interference and restore historical manuscripts. The accuracy of our alignment approach is then assessed with the accuracy of bleed-through correction.

Medical text mining has gained increasing interest in recent years. Radiology reports contain rich information describing radiologistpsilas observations on the patientpsilas medical conditions in the associated medical images. However, as most reports are in free text format, the valuable information contained in those reports cannot be easily accessed and used, unless proper text mining has been applied. In this paper, we propose a text mining system to extract and use the information in radiology reports. The system consists of three main modules: a medical finding extractor, a report and image retriever, and a text-assisted image feature extractor. In evaluation, the overall precision and recall for medical finding extraction are 95.5% and 87.9% respectively, and for all modifiers of the medical findings 88.2% and 82.8% respectively. The overall result of report and image retrieval module and text-assisted image feature extraction module is satisfactory to radiologists.

This paper presents our work on automatically detecting moving rigid text in digital videos. The temporal information is obtained by dividing a video frame into sub-blocks and calculating inter-frame motion vector for each sub-block. Text blocks are then extracted through both intra-frame classification and inter-frame spatial relationship checking. Unlike previous works, our method achieves both detection and tracking of moving text at the same time. The method works very well detecting scrolling text in news clips and movies, and is robust towards low resolution and complex background. The computational efficiency of the method is also discussed.

This paper reports a statistical script identification technique that determines the script of document images, especially camera-based images which suffer from perspective distortion. The identification technique represents a document image by a frequency vector of affine invariant signatures of characters, and identifies the script by comparing the vector with pre-prepared script templates. Experimental results show that our method is tolerant to moderate perspectives, document skew and various image noises.

A common problem encountered in signboard recognition is the perspective distortion of characters. In this paper, we propose a method which is able to directly recognize characters under severe perspective distortion without perspective rectification. In this method, a character is represented by a sequence of cross ratio spectra, in which the perspective effect can be modeled as an one-dimensional uneven stretching. Dynamic Time Warping algorithm is employed to estimate the pairwise similarity between spectra of the query and spectra of a fronto-parallel template. Then, it is again used to find out the pixel-level correspondence and the similarity between the query and the template. The experiment results showed that the proposed method worked well on synthetic character images and signboards in real scene under severe perspective projections.

Multi-slice computer tomography (CT) scans are widely used in todaypsilas diagnosis of head traumas. It is effective to disclose the bleeding and fractures. In this paper, we present an automated detection of CT scan slices which contain hemorrhages. Our method is robust towards various rotation, displacement and motion blur. Detection of these pathological slices will be useful for further diagnosis and retrieval.

In this paper, we explore new edge features such as straightness for the elimination of non significant edges from the segmented text portion of a video frame to detect accurate boundary of the text lines in video images. To segment the complete text portions, the method introduces candidate text block selection from a given image. Heuristic rules are formed based on combination of filters and edge analysis for identifying a candidate text block in the image. Furthermore, the same rules are extended to grow boundary of candidate text block in order to segment complete text portions in the image. The experimental results of the proposed method show that the method outperforms an existing method in terms of a number of metrics.

10.1145/1390334.1390496

The traditional single-candidate learning model for anaphora resolution considers the antecedent candidates of an anaphor in isolation, and thus cannot effectively capture the preference relationships between competing candidates for its learning and resolution. To deal with this problem, we propose a twin-candidate model for anaphora resolution. The main idea behind the model is to recast anaphora resolution as a preference classification problem. Specifically, the model learns a classifier that determines the preference between competing candidates, and, during resolution, chooses the antecedent of a given anaphor based on the ranking of the candidates. We present in detail the framework of the twin-candidate model for anaphora resolution. Further, we explore how to deploy the model in the more complicated coreference resolution task. We evaluate the twin-candidate model in different domains using the Automatic Content Extraction data sets. The experimental results indicate that our twin-candidate model is superior to the single-candidate model for the task of pronominal anaphora resolution. For the task of coreference resolution, it also performs equally well, or better.

This paper presents a document retrieval technique that is capable of searching document images without OCR (optical character recognition). The proposed technique retrieves document images by a new word shape coding scheme, which captures the document content through annotating each word image by a word shape code. In particular, we annotate word images by using a set of topological shape features including character ascenders/descenders, character holes, and character water reservoirs. With the annotated word shape codes, document images can be retrieved by either query keywords or a query document image. Experimental results show that the proposed document image retrieval technique is fast, efficient, and tolerant to various types of document degradation.

This paper reports an identification technique that detects scripts and languages of noisy and degraded document images. In the proposed technique, scripts and languages are identified through the document vectorization, which converts each document image into a document vector that characterizes the shape and frequency of the conta ned character or word images. Document images are vectorized by using vertical component cuts and character extremum points, which are both tolerant to the variation in text fonts and styles, noise, and various types of document degradation. For each script or language under study, a script or language template is first constructed through a training process. Scripts and languages of document images are then determined according to the distances between converted document vectors and the pre-constructed script and language templates. Experimental results show that the proposed technique is accurate, easy for extension, and tolerant to noise and various types of document degradation.

In document digitization through camera-based systems, simple imaging setups often produce geometric distortions in the resultant 2D images because of the non-planar geometric shapes of certain documents such as thick bound books, rolled, folded or crumpled materials, etc. Previous works have demonstrated that arbitrary warped documents can be successfully restored by flattening a 3D scan of the document. These approaches use physically-based or relaxation-based techniques in their flattening process. While this has been demonstrated to be effective in rectifying the image content and improving OCR, these previous approaches have several limitations in terms of speed and stability. In this paper, we propose a distance-based penalty metric to replace the mass-spring model and introduce additional bending resistance and drag forces to improve the efficiency of the existing approaches. The use of Verlet integration and special plane collision handling schemes also help to achieve better stability without sacrificing efficiency. Experiments on various document images captured from books, brochures and historical documents with arbitrary warpings have demonstrated large improvements over the existing approaches in terms of stability and efficiency.

As a kind of Shallow Semantic Parsing, Semantic Role Labeling (SRL) is gaining more attention as it benefits a wide range of natural language processing applications. Given a sentence, the task of SRL is to recognize semantic arguments (roles) for each predicate (target verb or noun). Feature-based methods have achieved much success in SRL and are regarded as the state-of-the-art methods for SRL. However, these methods are less effective in modeling structured features. As an extension of feature-based methods, kernel-based methods are able to capture structured features more efficiently in a much higher dimension. Application of kernel methods to SRL has been achieved by selecting the tree portion of a predicate and one of its arguments as feature space, which is named as predicate-argument feature (PAF) kernel. The PAF kernel captures the syntactic tree structure features using convolution tree kernel, however, it does not distinguish between the path structure and the constituent structure. In this article, a hybrid convolution tree kernel is proposed to model different linguistic objects. The hybrid convolution tree kernel consists of two individual convolution tree kernels. They are a Path kernel, which captures predicate-argument link features, and a Constituent Structure kernel, which captures the syntactic structure features of arguments. Evaluations on the data sets of the CoNLL-2005 SRL shared task and the Chinese PropBank (CPB) show that our proposed hybrid convolution tree kernel statistically significantly outperforms the previous tree kernels. Moreover, in order to maximize the system performance, we present a composite kernel through combining our hybrid convolution tree kernel method with a feature-based method extended by the polynomial kernel. The experimental results show that the composite kernel achieves better performance than each of the individual methods and outperforms the best reported system on the CoNLL-2005 corpus when only one syntactic parser is used and on the CPB corpus when automated syntactic parse results and correct syntactic parse results are used respectively.

Convolution tree kernel has shown promising results in semantic role labeling (SRL). However, this kernel does not consider much linguistic knowledge in kernel design and only performs hard matching between subtrees. To overcome these constraints, this paper proposes a grammar-driven convolution tree kernel for SRL by introducing more linguistic knowledge. Compared with the standard convolution tree kernel, the proposed grammar-driven kernel has two advantages: 1) grammar-driven approximate substructure matching, and 2) grammar-driven approximate tree node matching. The two approximate matching mechanisms enable the proposed kernel to better explore linguistically motivated structured knowledge. Experiments on the CoNLL-2005 SRL shared task and the PropBank I corpus show that the proposed kernel outperforms the standard convolution tree kernel significantly. Moreover, we present a composite kernel to integrate a feature-based polynomial kernel and the proposed grammar-driven convolution tree kernel for SRL. Experimental results show that our composite kernel-based method significantly outperforms the previously best-reported ones.

In this paper, we describe a 2-pass iterative scheme to solve the general partial differential equation (PDE) related to the Shape-from-Shading (SFS) problem under both distant and close point light sources. In particular, we discuss its applications in restoring warped document images that often appear in the daily snapshots. The proposed method consists of two steps. First the image irradiance equation is formulated as a static Hamilton-Jacobi (HJ) equation and solved using a fast sweeping strategy with Lax-Friedrichs Hamiltonian. However, abrupt errors may arise when applying to real document images due to noises in the approximated shading image. To reduce the noise sensitivity, a minimization method thus follows to smooth out the abrupt ridges in the initial result and produce a better reconstruction. Experiments on synthetic surfaces show promising results comparing to the ground truth data. Moreover, a general framework is developed, which demonstrates that the SFS method can help to remove both geometric and photometric distortions in warped document images for better visual appearance and higher recognition rate.

Information graphics, or infographics, are visual representations of information, data or knowledge. Understanding of infographics in documents is a relatively new research problem, which becomes more challenging when infographics appear as raster images. This paper describes technical details and practical applications of the system we built for recognizing and understanding imaged infographics located in document pages. To recognize infographics in raster form, both graphical symbol extraction and text recognition need to be performed. The two kinds of information are then auto-associated to capture and store the semantic information carried by the infographics. Two practical applications of the system are introduced in this paper, including supplement to traditional optical character recognition (OCR) system and providing enriched information for question answering (QA). To test the performance of our system, we conducted experiments using a collection of downloaded and scanned infographic images. Another set of scanned document pages from the University of Washington document image database were used to demonstrate how the system output can be used by other applications. The results obtained confirm the practical value of the system.

This paper presents a document image thresholding technique that binarizes badly illuminated document images by the photometric correction. Based on the observation that illumination normally varies smoothly and document images often contain a uniformly colored background, the global shading variation is estimated by using a two-dimensional Savitzky-Golay filter that fits a least square polynomial surface to the luminance of a badly illuminated document image. With the knowledge of the global shading variation, shading degradation is then corrected through a compensation process that produces animage with roughly uniform illumination. Badly illuminated document images are accordingly binarized through the global thresholding of the compensated ones. Experiments show that the proposed thresholding technique is fast, robust, and efficient for the binarization of badly illuminated document images.

This paper presents a restoration framework for correcting both photometric and geometric distortions in camera-based images of non-planar shaped documents to facilitate human perception and machine recognition. The photometric distortions, usually perceived as shading artifacts, are corrected by separating the shading image from the reflectance image through digital inpainting and surface fitting techniques. Meanwhile, the extracted shading image is also used to recover the document's surface shape through a Shape-from-Shading (SFS) method with a generic formulation of the image irradiance under arbitrary illumination conditions. The recovered surface shape is then employed to correct the geometric distortions through a physically-based flattening process. Results on real document images demonstrate the performance of each sub-task and the functionality of the whole framework. OCR results on restored images also show great improvements over the original distorted images.

Graphical components information extraction is a crucial step in the chart recognition and understanding process. However, existing methods of information extraction from chart images either are type-dependent or rely on certain assumptions. In this paper, we present a general method to extract vectorized graphical information from scientific chart images. Our algorithm firstly constructs a data structure called directional single-connected chains (DSCC). It then employs ellipse-specific fitting and orthogonal diagonalization to calculate the curvatures of the chains and classify the chains into either straight lines or arcs. Finally we combine all straight lines and all arcs accordingly and use linear regression to compute their attributes. The DSCC has a good property in that it is less susceptible to noise. The experiment results show that our algorithm is efficient, robust and accurate.

In order to capture the content of an imaged document but avoid the time-consuming full-scale OCR which is fragile to handle touching characters, a fast and segmentation- free keyword spotting method is proposed in this paper. The keyword spotting method is based on word shape coding technique. The proposed coding scheme has little ambiguity, and can be swiftly executed. It is a promising technique to boost better document image retrieval. The strength of the proposed method is demonstrated in a document filtering experiment. The experimental results show that document filtering based on the proposed method is more than 20 times faster than the one based on OCR, and has comparable filtering accuracy.

This paper presents a language identification technique that detects Latin-based languages of imaged documents without OCR. The proposed technique detects languages through the word shape coding, which converts each word image into a word shape code and accordingly transforms each document image into an electronic document vector. For each Latin-based language under study, a language template is first constructed through a corpus-based learning process. The underlying language of the query document is then determined based on the similarity between the query document vector and multiple constructed language templates. Compared with the reported methods, the proposed language identification technique is fast, accurate, and tolerant to text segmentation error caused by noise and various types of document degradation. Experimental results show some promising results.

This paper presents an identification technique that automatically detects the underlying script and orientation of scanned document images. In the proposed technique, document script and orientation are identified by using the stroke density and distribution, which convert each document image into a document vector. For each script at each orientation, a number of reference document vectors are first constructed. Script and orientation of the query document are then determined according to the similarity between the query document vector and multiple pre- constructed reference document vectors by using the K-nearest neighbor algorithm. Experiments show that the proposed technique is tolerant to the document skew and able to detect orientations of documents of different scripts.

This paper presents a document image binarization technique that segments text from badly illuminated document images. Based on the observations that text documents normally lie over a planar or smoothly curved surface and have a uniformly colored background, badly illuminated document images are binarized by using a smoothing polynomial surface, which estimates the shading variation and compensates the shading degradation based on the estimated shading variation. Badly illuminated document images are accordingly binarized through the global thresholding of the compensated document images. Compared with the reported methods, the proposed technique is tolerant to the variations in text size and document contrast. At the same time, it is much faster and able to produce a binary text image with little background noise.

This paper presents a keyword spotting technique that locates keywords within document images captured by a digital camera. In the proposed technique, the shape of word images in perspective view is captured by using three perspective invariants, namely, holes, water reservoirs, and character ascenders and descenders. Given a camera image of document, text line and word images are first segmented through the connected component analysis. The three perspective invariants are then detected through two rounds of scanning process, which transliterate each character image into a character shape code of dimension six and so convert each word image into a word shape code. Keywords within camera images of documents are finally located through a partial matching process. Experiments show some promising results.

This paper presents a document skew and orientation detection technique. The proposed technique estimates document skew and orientation based on the observation that text images normally hold a large amount of equidistant interline spacings and the number of character ascenders is statistically much larger than that of character descenders. Given a document image with arbitrary skew and orientation, white run histograms are first constructed through scanning documents in horizontal and vertical directions. Document skew is then estimated by using the white runs that exactly span the interline spacing. Lastly, document orientation is determined according to the numbers of character ascenders and descenders, which are detected by using the white runs that cross the interline spacing and lie over character ascenders and descenders. Experiments show that the proposed technique is fast, accurate, and capable of detecting arbitrary document skew and orientation.

Shading distortions are often perceived in geometrically distorted document images due to the change of surface normal with respect to the illumination direction. Such distortions are undesirable because they hamper OCR performance tremendously even when the geometric distortions are corrected. In this paper, we propose an effective method that removes shading distortions in images of documents with various geometric shapes based on the notion of intrinsic images. We first try to derive the shading image using an inpainting technique with an automatic mask generation routine and then apply a surface fitting procedure with radial basis functions to remove pepper noises in the inpainted image and return a smooth shading image. Once the shading image is extracted, the reflectance image can be obtained automatically. Experiments on a wide range of distorted document images demonstrate a robust performance. Moreover, we also show its potential applications to the restoration of historical handwritten documents.

In vector space model (VSM), textual documents are represented as vectors in the term space. Therefore, there are two issues in this representation, i.e. (1) what should a term be and (2) how to weight a term. This paper examined ways to represent text from the above two aspects to improve the performance of text categorization. Different representations have been evaluated using SVM on three biomedical corpora. The controlled experiments showed that the straightforward usage of named entities as terms in VSM does not show performance improvements over the bag-of-words representation. On the other hand, the term weighting method slightly improved the performance. However, to further improve the performance of text categorization, more advanced techniques and more effective usages of natural language processing for text representations appear needed.

An important step in chart image understanding is to identify the type of the input image so that corresponding interpretation can be performed. In this paper, we model the chart image classification as a multiple-instance learning problem. A chart image is treated as a bag containing a set of instances that are graphical symbols. For both training and recognition, shape detection is performed and general shape descriptors are used to form feature vectors. For the training images, the correlation factor (CF) of each shape is calculated for each chart type. The learnt CFs are then used to estimate the type of a new input image. Comparing with traditional multiple-instance learning algorithms, we allow negative examples to be less restrictive and hence easier to provide. Using our method, both the type and the data components of the chart image can be obtained in one-pass. The experimental results show that our approach works reasonably well

With the pervasive use of handheld digital devices such as camera phones and PDAs, people have started to capture images as a way of recording information. However, due to the non-planar geometric shapes of some documents such as thick bound book pages, rolled scripts, etc, the captured images are often warped. This causes problems for many document analysis and retrieval tasks. This paper proposes a dewarping approach that first reconstructs the surface using a shape-from-shading (SFS) technique and then flattens the warped surface using a physically-based deformable model. The SFS method is based on the viscosity framework and considers perspective projection under oblique light source. The recovered surface is represented as a triangular mesh and is restored to its planar state through a numerical integration process. We have tested the proposed approach on both synthetic and real images. The synthetic images are mainly used to evaluate the SFS method and the results are compared with those of a prior SFS method. The reconstructed surfaces of real document images are also compared with the actual shapes captured using a 3D scanner and the restored images are evaluated with those obtained from range-scanned surface shapes

A Voronoi diagram of image elements provides an intuitive and appealing definition of proximity, which has been suggested as an effective tool for the description of relations among the neighboring objects in a digital image. In this paper, an implementation algorithm based on direct calculation of the Freeman code of expanded contours is proposed for generating area Voronoi diagram of connected components. A closed convex polygon is utilized to bound each connected component, as an approximate representation, and the contour is represented using Freeman chain coding, from which we can compute the corresponding Freeman chain coding of its expanded contour directly, without recourse to the operation on pixels. While the contours iteratively expand outwards, the Voronoi diagram is constructed by the intersections of the expanded contours from different connected components. The experimental results show that our proposed approach significantly improves the speed of constructing area Voronoi diagram in digital images.

Shortage of manually labeled data is an obstacle to supervised relation extraction methods. In this paper we investigate a graph based semi-supervised learning algorithm, a label propagation (LP) algorithm, for relation extraction. It represents labeled and unlabeled examples and their distances as the nodes and the weights of edges of a graph, and tries to obtain a labeling function to satisfy two constraints: 1) it should be fixed on the labeled nodes, 2) it should be smooth on the whole graph. Experiment results on the ACE corpus showed that this LP algorithm achieves better performance than SVM when only very few labeled examples are available, and it also performs better than bootstrapping for the relation extraction task.

Syntactic knowledge is important for pronoun resolution. Traditionally, the syntactic information for pronoun resolution is represented in terms of features that have to be selected and defined heuristically. In the paper, we propose a kernel-based method that can automatically mine the syntactic information from the parse trees for pronoun resolution. Specifically, we utilize the parse trees directly as a structured feature and apply kernel functions to this feature, as well as other normal features, to learn the resolution classifier. In this way, our approach avoids the efforts of decoding the parse trees into the set of flat syntactic features. The experimental results show that our approach can bring significant performance improvement and is reliably effective for the pronoun resolution task.

This paper describes a new bipartite formulation for word-document co-clustering such that hyperclique patterns, strongly affiliated documents in this case, are guaranteed not to be split into different clusters. Our approach for pattern preserving clustering consists of three steps: mine maximal hyperclique patterns, form the bipartite, and partition it. With hyperclique patterns of documents preserved, the topic of each cluster can be represented by both the top words from that cluster and the documents in the patterns, which are expected to be more compact and representative than those in the standard bipartite formulation. Experiments with real-world datasets show that, with hyperclique patterns as starting points, we can improve the clustering results in terms of various external clustering criteria. Also, the partitioned bipartite with preserved topical sets of documents naturally lends itself to different functions in search engines

This paper presents an automatic orientation detection and categorization technique that is capable of detecting the orientation of multilingual documents with arbitrary skew and categorizing document images according to the underlying languages. We carry out orientation detection and categorization through document vectorization, which encodes document orientation and language information and converts each document image into an electronic document vector through the exploitation of the density and distribution of vertical component runs. For each language of interest, a pair of vector templates is first constructed through a training process. Orientation and category of the query image are then determined based on distances between the query document vector and the constructed vector templates. Experiments over 492 testing document images show that the average orientation detection and categorization rates reach up to 97.56% and 99.59%, respectively.

Scanning a document page from a thick bound volume often results in two kinds of distortions in the scanned image, i.e., shade along the "spine" of the book and warping in the shade area. In this paper, we propose an efficient restoration method based on the discovery of the 3D shape of a book surface from the shading information in a scanned document image. From a technical point of view, this shape from shading (SFS) problem in real-world environments is characterized by 1) a proximal and moving light source, 2) Lambertian reflection, 3) nonuniform albedo distribution, and 4) document skew. Taking all these factors into account, we first build practical models (consisting of a 3D geometric model and a 3D optical model) for the practical scanning conditions to reconstruct the 3D shape of the book surface. We next restore the scanned document image using this shape based on deshading and dewarping models. Finally, we evaluate the restoration results by comparing our estimated surface shape with the real shape as well as the OCR performance on original and restored document images. The results show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.

The comprehensibility aspect of rule discovery is of emerging interest in the realm of knowledge discovery in databases. Of the many cognitive and psychological factors relating the comprehensibility of knowledge, we focus on the use of human amenable concepts as a representation language in expressing classification rules. Existing work in neural logic networks (or neulonets) provides impetus for our research; its strength lies in its ability to learn and represent complex human logic in decision-making using symbolic-interpretable net rules. A novel technique is developed for neulonet learning by composing net rules using genetic programming. Coupled with a sequential covering approach for generating a list of neulonets, the straightforward extraction of human-like logic rules from each neulonet provides an alternate perspective to the greater extent of knowledge that can potentially be expressed and discovered, while the entire list of neulonets together constitute an effective classifier. We show how the sequential covering approach is analogous to association-based classification, leading to the development of an association-based neulonet classifier. Empirical study shows that associative classification integrated with the genetic construction of neulonets performs better than general association-based classifiers in terms of higher accuracies and smaller rule sets. This is due to the richness in logic expression inherent in the neulonet learning paradigm.

Shortage of manually sense-tagged data is an obstacle to supervised word sense disambiguation methods. In this paper we investigate a label propagation based semi-supervised learning algorithm for WSD, which combines labeled and unlabeled data in learning process to fully realize a global consistency assumption: similar examples should have similar labels. Our experimental results on benchmark corpora indicate that it consistently outperforms SVM when only very few labeled examples are available, and its performance is also better than monolingual bootstrapping, and comparable to bilingual bootstrapping.

In this paper we focus on how to improve pronoun resolution using the statistics-based semantic compatibility information. We investigate two unexplored issues that influence the effectiveness of such information: statistics source and learning framework. Specifically, we for the first time propose to utilize the web and the twin-candidate model, in addition to the previous combination of the corpus and the single-candidate model, to compute and apply the semantic information. Our study shows that the semantic compatibility obtained from the web can be effectively incorporated in the twin-candidate learning model and significantly improve the resolution of neutral pronouns.

When one scans a document page from a thick bound volume, the curvature of the page to be scanned results in two kinds of distortion in the scanned document images: i) shade along the 'spine' of the book; and ii) warping in the shade area. In this paper, we propose an efficient restoration method based on the discovery of the 3D shape of a book surface from the shading information in a scanned document image. We first build practical models namely a 3D geometric model and a 3D optical model for the practical scanning conditions to reconstruct the 3D shape of book surface. We next restore the scanned document image using this shape based on de-shading and de-warping models. Finally, we evaluate the restoration results by comparing the OCR (optical character recognition) performance on the original and restored document images. The experiments show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.

We present a framework for acquiring and restoring images of warped documents to their original planar shapes. Image capturing of warped documents often results in warped images. Most digital restoration approaches make use of 2D image processing methods which are dependent on the content of the images. Some recent work attempts to remove this constraint by working directly on the 3D shape of the warped documents. In our framework, we improve on this recent approach to make it more efficient and stable. First we capture the warped surface representation through a set of active lighting from a laser range scanner onto the document surface. This gives us an accurate 3D representation of the document shape. Next by using a stick constraint and a stable numerical integrator, we can digitally flatten the 3D model to a planar shape. This simple approach avoids potential instability problem and is fast. Restoration results for a few types of warped documents are demonstrated and a significant improvement over the original images can be observed.

This paper presents our recent work that aims at associating the recognition results of textual and graphical information contained in the scientific chart images. Text components are first located in the input image and then recognized using OCR. On the other hand, the graphical objects are segmented and form high level symbols. Both logical and semantic correspondence between text and graphical symbols are identified. The association of text and graphics allows us to capture the semantic meaning carried by scientific chart images in a more complete way. The result of scientific chart image understanding is presented using XML documents.

This paper addresses the problem of text extraction from name card images with fanciful design containing complicated color background and reverse contrast regions. The proposed method is based on special edge detection followed by grouping of edges based on size, orientation and color attributes. The difficulties from color complexity caused by background color variation or reverse contrast are overcome by edge detection and color histogram. The grouping of edges filters out the background textures or graphical objects. The final detected text grouping is then binarized for OCR. Experimental results show improved text extraction in comparison with a conventional method.

Voronoi diagram of image elements provides an intuitive and appealing definition of proximity, which has been suggested as an effective tool for the description of relations among the neighboring objects in a digital image. In this paper, a fast implementation algorithm is proposed for generating area Voronoi diagram of connected components in document images. A closed convex polygon is utilized to bound each connected component, and the contour is represented using Freeman chain coding, from which we can compute the corresponding Freeman chain coding of its expanded contour directly, without recourse to the operation on pixels. While the contours iteratively expand outwards, the Voronoi diagram is constructed by the intersections of the expanded contours from different connected components. The experimental results show that our proposed approach significantly improves the speed of constructing area Voronoi diagram.

This paper investigates the problem of blurring caused by motion during image capture of text documents. Motion blurring prevents proper optical character recognition of the document text contents. One area of such applications is to deblur name card images obtained from handheld cameras. In this paper, a complete motion deblurring procedure for document images has been proposed. The method handles both uniform linear motion blur and uniform acceleration motion blur. Experiments on synthetic and real-life blurred images prove the feasibility and reliability of this algorithm provided that the motion is not too irregular. The restoration procedure consumes only small amount of computation time.

The vast majority of the published skew estimation methods for scanned document images are for textual documents. These methods are based on the principle that the skew angles can be derived from the presence of the obvious text lines. The non-textual objects, such as line drawings, photographic inserts, scan artifacts including the dark bars around the borders and the center spine of bounded materials, and media contaminations are considered as "noises", thus are subject to elimination. Skew estimators that work in the presence of excessive noises are considered robust. This paper presents a skew estimation method that is based on the straight lines or edges. It uses the Muff Transform with a probe-line mapping scheme for feature identification. Various strategies for optimized line probing are devised. This method is applicable to both textual and graphical documents scanned with ordinary scanners or copiers under normal conditions. Selected images from the University of Washington English document image database I (UWDB-I) are used for its usability evaluation.

This paper describes a fast and effective component grouping algorithm for some layout analysis problems of textual document images. Based on an empirical function, it discovers neighborhood relations at the component level. Time-consuming algorithms that operate globally among components may obtain significant performance boost from the localization information that this grouping algorithm provides. Some of the applications using this grouping function are also demonstrated.

Recently, high resolution digital cameras have made the digitization process more flexible and convenient than traditional scanning technology. Therefore, document image analysis techniques need to be extended to handle images captured using ordinary hand-held cameras under an uncontrolled environment. In this paper, we propose an image restoration technique that corrects various warping distortions such as rolling curl, binding curl and fold distortions in camera images of geometrically distorted documents. Our goal is to restore these arbitrarily warped document images to a planar shape so as to facilitate OCR. Our method uses spline interpolation technique based on a ruled surface model constructed from text lines in the 2D document image. Comparisons on OCR performance using Microsoft imaging software with two existing methods show an encouraging improvement in terms of recall and precision. The OCR precision on the corrected images using our method is improved up to 21.8% comparing to original images with a resolution of 3 mega pixels.

The term weighting scheme, which is used to convert documents into vectors in the term spaces, is a vital step in automatic text categorization. The previous studies showed that term weighting schemes dominate the performance rather than the kernel functions of SVMs for the text categorization task. In this paper, we conducted experiments to compare various term weighting schemes with SVM on two widely-used benchmark data sets. We also presented a new term weighting scheme tf.rf for text categorization. The cross-scheme comparison was performed by using McNemar's tests. The controlled experimental results showed that the newly proposed tf.rf scheme is significantly better than other term weighting schemes. Compared with schemes related with tf factor alone, the idf factor does not improve or even decrease the term's discriminating power for text categorization. The binary and tf.chi representations significantly underperform the other term weighting schemes.

This paper addresses the problem of text extraction from name card images with fanciful design containing various graphical foreground and reverse contrast regions. The proposed method is to apply a neural network on canny edges with both spatial and relative features like sizes, color attributes and relative alignment features. By making use the alignment information, we can identify the text area from the character level rather than the conventional window block level. This alignment information is based on the human visual perception theory. Some post processing like color identification and binarization will be helpful to get a pure binary text image for OCR.

This paper presents an unsupervised word sense learning algorithm, which induces senses of target word by grouping its occurrences into a "natural" number of clusters based on the similarity of their contexts. For removing noisy words in feature set, feature selection is conducted by optimizing a cluster validation criterion subject to some constraint in an unsupervised manner. Gaussian mixture model and Minimum Description Length criterion are used to estimate cluster structure and cluster number. Experimental results show that our algorithm can find important feature subset, estimate model order (cluster number) and achieve better performance than another algorithm which requires cluster number to be provided.

In this paper, we propose a multi-criteriabased active learning approach and effectively apply it to named entity recognition. Active learning targets to minimize the human annotation efforts by selecting examples for labeling. To maximize the contribution of the selected examples, we consider the multiple criteria: informativeness, representativeness and diversity and propose measures to quantify them. More comprehensively, we incorporate all the criteria using two selection strategies, both of which result in less labeling cost than single-criterion-based method. The results of the named entity recognition in both MUC-6 and GENIA show that the labeling cost can be reduced by at least 80 % without degrading the performance. 1

Coreferential information of a candidate, such as the properties of its antecedents, is important for pronoun resolution because it reflects the salience of the candidate in the local discourse. Such information, however, is usually ignored in previous learning-based systems. In this paper we present a trainable model which incorporates coreferential information of candidates into pronoun resolution. Preliminary experiments show that our model will boost the resolution performance given the right antecedents of the candidates. We further discuss how to apply our model in real resolution where the antecedents of the candidate are found by a separate noun phrase resolution module. The experimental results show that our model still achieves better performance than the baseline.

This paper introduces a novel motif discovery algorithm based on the use of constraint mechanism and constraint rules. The key idea is to convert sets of similar substrings of the DNA sequences into patterns, as early as possible, using constraint mechanism or constraint rules. The advantages are two folds. Firstly, the approach generates limited number of patterns while still guaranteeing that the actual motifs are contained in the pattern set. Secondly, the procedure for deriving patterns is very cost-effective since it can be considered as that we use many "look ahead" to speed up the procedure. Therefore, the algorithm has the advantages of the high sensitivity of pattern-driven algorithms as well as the efficiency of sample-driven algorithms.

In this paper a new adaptation-based method is presented to reconstruct animatable facial models of human individuals from scan data. An anatomy-based generic control model serves as the starting point for our adaptation algorithm. Based on a series of measurements between the specified 3D landmarks, a global adaptation is carried out to align the generic control model to the scan surface. A local adaptation then deforms the geometry of the generic model to fit all of its vertices to the scan surface. The high-resolution geometry of the scan surface is represented as a displacement map which is generated using an offset-envelope mapping. Reconstruction of high-resolution geometry on the adapted generic mesh is achieved by hierarchical refinement using a surface subdivision scheme with resampling of the displacement map

International Conference on Information and Knowledge Management, Proceedings

In this paper, we present a novel approach to vectorize elliptic arcs for 3D pie chart recognition and data extraction. As a preprocessing step, a set of straight line segments are obtained first. By estimating the parameters based on the line segments, we are able to generate elliptic arc candidates. Curve fitting is then performed to reject false positives and further refine the parameters of each elliptic arc. The vectorized elliptic arcs are the key components in the 3D pie charts that allow us to determine the chart type and to extract data values embedded. Experiments using a group of testing images show that the proposed method works well and is computationally efficient.

Advances in 3D scanning technology have enabled automatic capture of complex 3D models such as human faces with highly detailed surfaces. However, the range data cannot be used easily for animatable face modeling due to the absence of functional animation structure and dense surface data. The paper presents an automatic facial model adaptation algorithm for reconstruction of animatable individualized 3D facial models from range data. A generic model that represents both the face shape and anatomical structure serves as the starting point for the adaptation algorithm. The global adaptation transforms the generic model to align it with the scan data in the 3D space based on measurements between a set of 3D landmarks. The local adaptation then deforms the skin mesh of the generic model to fit all of its vertices to the scan surface. The underlying muscle structure is automatically adapted and facial texture is transferred. The reconstructed 3D face resembles the shape and color of a real individual and can be animated immediately with muscle parameters.

Various iterative refinement clustering methods are dependent on the initial state of the model and are capable of obtaining one of their local optima only. Since the task of identifying the global optimization is NP-hard, the study of the initialization method towards a sub-optimization is of great value. This paper reviews the various cluster initialization methods in the literature by categorizing them into three major families, namely random sampling methods, distance optimization methods, and density estimation methods. In addition, using a set of quantitative measures, we assess their performance on a number of synthetic and real-life data sets. Our controlled benchmark identifies two distance optimization methods, namely SCS and KKZ, as complements of the k-means learning characteristics towards a better cluster separation in the output solution.

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

MOTIVATION: With an overwhelming amount of textual information in molecular biology and biomedicine, there is a need for effective and efficient literature mining and knowledge discovery that can help biologists to gather and make use of the knowledge encoded in text documents. In order to make organized and structured information available, automatically recognizing biomedical entity names becomes critical and is important for information retrieval, information extraction and automated knowledge acquisition. RESULTS: In this paper, we present a named entity recognition system in the biomedical domain, called PowerBioNE. In order to deal with the special phenomena of naming conventions in the biomedical domain, we propose various evidential features: (1) word formation pattern; (2) morphological pattern, such as prefix and suffix; (3) part-of-speech; (4) head noun trigger; (5) special verb trigger and (6) name alias feature. All the features are integrated effectively and efficiently through a hidden Markov model (HMM) and a HMM-based named entity recognizer. In addition, a k-Nearest Neighbor (k-NN) algorithm is proposed to resolve the data sparseness problem in our system. Finally, we present a pattern-based post-processing to automatically extract rules from the training data to deal with the cascaded entity name phenomenon. From our best knowledge, PowerBioNE is the first system which deals with the cascaded entity name phenomenon. Evaluation shows that our system achieves the F-measure of 66.6 and 62.2 on the 23 classes of GENIA V3.0 and V1.1, respectively. In particular, our system achieves the F-measure of 75.8 on the "protein" class of GENIA V3.0. For comparison, our system outperforms the best published result by 7.8 on GENIA V1.1, without help of any dictionaries. It also shows that our HMM and the k-NN algorithm outperform other models, such as back-off HMM, linear interpolated HMM, support vector machines, C4.5, C4.5 rules and RIPPER, by effectively capturing the local context dependency and resolving the data sparseness problem. Moreover, evaluation on GENIA V3.0 shows that the post-processing for the cascaded entity name phenomenon improves the F-measure by 3.9. Finally, error analysis shows that about half of the errors are caused by the strict annotation scheme and the annotation inconsistency in the GENIA corpus. This suggests that our system achieves an acceptable F-measure of 83.6 on the 23 classes of GENIA V3.0 and in particular 86.2 on the "protein" class, without help of any dictionaries. We think that a F-measure of 90 on the 23 classes of GENIA V3.0 and in particular 92 on the "protein" class, can be achieved through refining of the annotation scheme in the GENIA corpus, such as flexible annotation scheme and annotation consistency, and inclusion of a reasonable biomedical dictionary. AVAILABILITY: A demo system is available at http://textmining.i2r.a-star.edu.sg/NLS/demo.htm. Technology license is available upon the bilateral agreement.

An approach to searching for user-specified words in imaged Chinese documents, without the requirements of layout analysis and OCR processing of the entire documents, is proposed in this paper. A small number of Chinese characters that cannot be successfully bounded using connected component analysis due to larger gaps between elements within the characters are blacklisted. A suitable character that is not included in the blacklist is chosen from the user-specified word as the initial character to search for a matching candidate in the document. Once a matched candidate is found, the adjacent characters in the horizontal and vertical directions are examined for matching with other corresponding characters in the user-specified word, subject to the constraints of alignment (either horizontal or vertical direction) and size similarity. A weighted Hausdorff distance is proposed for the character matching. Experimental results show that the present method can effectively search the user-specified Chinese words from the document images with the format of either horizontal or vertical text lines, or both appearing on the same image.

With the rising popularity and importance of document images as an information source, information retrieval in document image databases has become a growing and challenging problem. In this paper, we propose an approach with the capability of matching partial word images to address two issues in document image retrieval: word spotting and similarity measurement between documents. First, each word image is represented by a primitive string. Then, an inexact string matching technique is utilized to measure the similarity between the two primitive strings generated from two word images. Based on the similarity, we can estimate how a word image is relevant to the other and, thereby, decide whether one is a portion of the other. To deal with various character fonts, we use a primitive string which is tolerant to serif and font differences to represent a word image. Using this technique of inexact string matching, our method is able to successfully handle the problem of heavily touching characters. Experimental results on a variety of document image databases confirm the feasibility, validity, and efficiency of our proposed approach in document image retrieval.

This paper introduces the Adaptive Resonance Theory under Constraint (ART-C 2A) learning paradigm based on ART 2A, which is capable of generating a user-defined number of recognition nodes through online estimation of an appropriate vigilance threshold. Empirical experiments compare the cluster validity and the learning efficiency of ART-C 2A with those of ART 2A, as well as three closely related clustering methods, namely online K-Means, batch K-Means, and SOM, in a quantitative manner. Besides retaining the online cluster creation capability of ART 2A, ART-C 2A gives the alternative clustering solution, which allows a direct control on the number of output clusters generated by the self-organizing process.

In this paper we propose a competition learning approach to coreference resolution. Traditionally, supervised machine learning approaches adopt the single-candidate model. Nevertheless the preference relationship between the antecedent candidates cannot be determined accurately in this model. By contrast, our approach adopts a twin-candidate learning model. Such a model can present the competition criterion for antecedent candidates reliably, and ensure that the most preferred candidate is selected. Furthermore, our approach applies a candidate filter to reduce the computational cost and data noises during training and resolution. The experimental results on MUC-6 and MUC-7 data set show that our approach can outperform those based on the single-candidate model.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Summary form only given. A compressed pattern matching method for searching keywords from the CCIT group 4-compressed document images, without explicit decompression, is presented. According to the CCIT Group 4 standards, each coded position indicates current pixel color is different from its previous pixel, except for the next coded positions of the pass mode. The changing elements from the compressed images are extracted and are then utilized to segment and bound the word objects and to measure the similarity of two word images. A two-stage matching strategy is constructed to measure the dissimilarity between the template image of the user's query word and the word extracted from document images. Experiments were conducted to verify the validity of the approach. The results show that the proposed approach was much faster than the traditional approach, because it avoids the pixel-level processing for analyzing the connected components and extracting word features.

To distinguish image features from high level noises, we propose to exploit the spatial correlations between wavelet coefficients by replacing the thresholding process with a diffusion process. The proposed smoothing method is more robust to noise when applied to highly corrupted document images.

Compared to binary images that most text extraction methods work on, gray scale images provide much more information for the extraction task. On the other hand complication also arises in determining the subject textual content from its background region (i.e. thresholding) before the actual text extraction process can begin. Differing from the usual sequence of processes where document images are binarized before the actual text extraction, this paper proposes a new method by first segmenting individual subject areas with the help of an irregular pyramid to be followed by the binarization process. This permits the focus of attention only on the appropriate subject areas for the binarization process before text recognition. Our method overcomes the difficulty in global binarization to find a single value to fit all. It also avoids the common problem in most local thresholding technique of finding a suitable window size. As shown in our experimented result, our method performed well in both text segmentation and binarization by varying the sequence of processing.

In this paper, we present a compressed pattern matching method for searching user queried words in the CCITT Group 4 compressed document images, without decompressing. The feature pixels composed of black changing elements and white changing elements are extracted directly from the CCITT Group 4 compressed document images. The connected components are labeled based on a line-by-line strategy according to the relative positions between the changing elements of the current coding line and the changing elements of the reference line. Word boxes are bounded by merging the connected components. A two-stage matching strategy is constructed to measure the dissimilarity between the template image of the user's query word and the words extracted from document images. Experimental results confirmed the validity of the proposed approach.

The nearest-neighbor based document skew detection methods do not require the presence of a predominant text area, and are not subject to skew angle limitation. However, the accuracy of these methods is not perfect in general. In this paper, we present an improved nearest-neighbor based approach to perform accurate document skew estimation. Size restriction is introduced to the detection of nearest-neighbor pairs. Then the chains with a largest possible number of nearest-neighbor pairs are selected, and their slopes are computed to give the skew angle of document image. Experimental results on various types of documents containing different linguistic scripts and diverse layouts show that the proposed approach has achieved an improved accuracy for estimating document image skew angle and has an advantage of being language independent.

In this paper, we propose a directional wavelet approach to remove images of interfering strokes coming from the back of a historical handwritten document due to seeping of ink during long period of storage. Our previous work required mapping of both sides of the document in order to identify the interfering strokes to be eliminated. Perfect mapping, however, is difficult due to document skews, differing resolutions, non-availability of the reverse side and warped pages during scanning. The new approach does not require double-sided mapping but instead uses a directional wavelet transform to distinguish the foreground and reverse side strokes. Experiments have shown that the directional wavelet operation effectively removes the interfering strokes. 1.

Skewscope is a software module designed to detect and analyze skew information of textual document images. Its skew detection algorithm, which is the major topic of this paper, directly computes the slopes of all unique pairs of components and draws skew information from their distribution profile. Its validity is examined by using the complete set of University of Washington English Document Image Database I as ground truth. Its utilization of both the foreground and the background components is also discussed.

Image warping is a common problem when one scans or photocopies a document page from a thick bound volume, resulting in shading and curved text lines in the spine area of the bound volume. This will not only impair readability, but will also reduce the OCR accuracy. Further to our earlier attempt to correct such images, this paper proposes a simpler connected component analysis and regression technique. Compared to our earlier method, the present system is computationally less expensive and is resolution independent too. The implementation of the new system and improvement of OCR accuracy are presented in this paper.

To distinguish image features from high level noises, we propose to exploit the spatial correlations between wavelet coefficients by replacing the thresholding process with a diffusion process. The proposed smoothing method is more robust to noise when applied to highly corrupted images.

Clustering of gene expression patterns is of great value for the understanding of the various molecular biological processes. While a number of algorithms have been applied to gene clustering, there are relatively few studies on the application of neural networks to this task. In addition, there is a lack of quantitative evaluation of the gene clustering results. This paper proposes Adaptive Resonance Theory under Constraint (ART-C) for efficient clustering of gene expression data. We illustrate that ART-C can effectively identify gene functional groupings through a case study on rat CNS data. Based on a set of quantitative evaluation measures, we compare the performance of ART-C with those of K-Means, SOM, and conventional ART. Our comparative studies on the yeast cell cycle and the human hematopoietic differentiation data sets show that ART-C produces reasonably good quantitative performance. More importantly, compared with K-Means and SOM, ART-C shows a significantly higher learning efficiency, which is crucial for knowledge discovery from large scale biological databases.

An important issue that needs to be addressed when using data mining tools is the validity of the rules outside of the data set from which they are generated. Rules are typically derived from the patterns in a particular data set. When a new situation occurs, the change in the set of rules obtained from the new data set could be significant. In this paper, we provide a novel model for understanding how the differences between two situations affect the changes of the rules, based on the concept of fine partitioned groups that we call caucuses. Using this model, we provide a simple technique called combination data set, to get a good estimate of the set of rules for a new situation. Our approach works independently of the core mining process and it can be easily implemented with all variations of rule mining techniques. Through experiments with real-life and synthetic data sets, we show the effectiveness of our technique in finding the correct set of rules under different situations.

This paper proposes a structure tensor based image restoration algorithm. Structure tensors capture the structural and textural distribution of similar pixels at each site. This property adapts the smoothing process to the contrast, orientation and spatial size of local image structures.

Perspective distortion always occurs while scanning thick, bound documents, resulting in two problems in the scanned grayscale image: (i) shadow along the 'spine' of the book; (ii) warping of words in the shadow area. In our previous paper, we proposed a restoration system to solve these two problems (see Zheng Zhang and Chew Lim Tan, Int. Conf. on Image Proc., p.1074-7, 2001). However the shape of the warped words was not fully restored, since we simply shifted and rotated the warped word. We now propose a better method to straighten a warped text line using polynomial regression. The implementation results comparing with those of the old system show a marked improvement in the restoration of the image.

In this paper, we illustrate a method to segment handwritten Chinese characters from destination addresses of mail pieces. Fast Hough transform is utilized to detect the reference lines preprinted on the mail piece. In the segmentation, subassemblies of Chinese characters are merged based on the structural features of Chinese characters and the subassemblies' topological relations, viz. upper–lower, inside–outside and left–right relations. The width of subassemblies and the spacing between neighboring subassemblies in the whole image of the destination address are analyzed to guide the merging of the left–right subassemblies. Experimental results with real mail piece images show that the proposed approach has achieved a promising performance for segmenting handwritten Chinese characters.

This paper addresses a problem of restoring handwritten archival documents by recovering their contents from the interfering handwriting on the reverse side caused by the seeping of ink. We present a novel method that works by first matching both sides of a document such that the interfering strokes are mapped with the corresponding strokes originating from the reverse side. This facilitates the identification of the foreground and interfering strokes. A wavelet reconstruction process then iteratively enhances the foreground strokes and smears the interfering strokes so as to strengthen the discriminating capability of an improved Canny edge detector against the interfering strokes. The method has been shown to restore the documents effectively with average precision and recall rates for foreground text extraction at 84 percent and 96 percent, respectively.

We propose a method for text retrieval from document images without the use of OCR. Documents are segmented into character objects. Image features, namely the vertical traverse density (VTD) and horizontal traverse density (HTD), are extracted. An n-gram-based document vector is constructed for each document based on these features. Text similarity between documents is then measured by calculating the dot product of the document vectors. Testing with seven corpora of imaged textual documents in English and Chinese as well as images from the UW1 (University of Washington 1) database confirms the validity of the proposed method.

It is crucial to use contextual information to improve the recognition accuracy of Chinese script in an offline, handwritten Chinese character-recognition system. However, with the increase in the number of candidates given by a character recognizer, contextual postprocessing using a word-based bigram is time-consuming. This article presents a novel contextual postprocessing method that integrates character-based bigram postprocessing with word-based bigram postprocessing in light of the complementary action between Chinese characters and Chinese words. On the basis of isolated character recognition, character-based bigram postprocessing using a forward-backward search is first executed on a big candidate set, which improves both the accuracy and efficiency of the candidate set (the cumulative accuracy of the top ten candidates is greatly boosted). Then, to further improve accuracy, word-based bigram postprocessing (WBP) is executed on a small candidate set. This method obtains high accuracy while paying attention to postprocessing speed at the same time. Experimental results for three Chinese scripts (about 66,000 characters in total) demonstrate the effectiveness of our method: character-based bigram postprocessing improves accuracy from 81.58% to 94.50%, and the cumulative accuracy of the top ten candidates rises from 94.33% to 98.25%. After WBP, 95.75% accuracy is achieved, which is equivalent to the accuracy of WBP executed on a big candidate set. However, our method is more than 100 times faster than that of WBP.

The National Archives of Singapore keeps a large volume of historical handwritten documents. One, common problem with the archives is that over the years, ink sipped through the pages of these documents such that characters on the reverse side become visible and interfere with the characters on the front side. This paper addresses this problem and develops a novel algorithm to extract clear textual images from the, interference. We achieve this by mapping images from both sides of a page such that interfering strokes seen on the front side are matched with the strokes originating from the reverse side so as to achieve a cancellation effect. The resultant image a further subjected to an improved Canny edge detection to eliminate remaining background interference. Experimental results have confirmed the validity of our proposed method.

The separation of overlapping text from graphics is a challenging problem in document image analysis. This paper proposes a specific method for detecting and extracting characters that are touching graphics. It is based on the observation that the constituent strokes of characters are usually short segments in comparison with those of graphics. It combines line continuation with the feature line width to decompose and reconstruct segments underlying the region of intersection. Experimental results showed that the proposed method improved the percentage of correctly detected text as well as the accuracy of character recognition significantly.

We propose a novel coplanar filter, which exploits the coplanarity of gray-level distribution of neighboring pixels, to pre-filter the document images. Experiments show that the proposed filter exhibits the following desired properties for document image binarization: 1) impulsive noise removal, 2) piecewise smoothing, and 3) sharp edge preservation.

A new method for gauging text similarity of image-based documents using word shape recognition is proposed in this paper. Image features are directly extracted instead of using OCR (optical character recognition). The proposed method forms so-called vertical bar patterns by detecting local extrema points in word units extracted by segmenting the document images. These vertical bar patterns form the feature vector of a document. The pair-wise similarity of document images is measured by calculating the scalar product of two document feature vectors. The proposed method is robust to changing fonts and styles, and is less affected by degradation of document qualities. To test the validity of the method, four corpora of document images were used and the ability of the method to retrieve relevant documents is reported.

This paper proposes a new algorithm to detect word groups in imaged documents, using an irregular pyramid. The uniqueness of this algorithm is its inclusion of strategic background information in the analysis, which most techniques have discarded. Both the foreground (i.e. text-area) and portions of the background (i.e. white-area) regions are examined. The fundamental aspect of the algorithm is based on the concept of "closeness", where text information within a group is closer to other text information within the group, in terms of spatial distance, compared to other text areas. The result produced by the algorithm is encouraging, with the ability to correctly group words of different sizes, fonts, arrangements and orientations.

An approach to word image matching based on weighted Hausdorff distance (WHD) is proposed in this paper to facilitate the detection and location of the user-specified words in the document images. Preprocessing such as eliminating the space between adjacent characters in the word images and scale normalization is first done before the WHD is utilized to measure the distance between the template image and the word image extracted from the document image. Experimental results in the application of detecting the user-specified words from both English and Chinese document images show that it is a promising approach for word image matching.

In this paper we present a well designed method that makes use of edge information to extract textual blocks from gray scale document images. It aims at detecting textual regions on heavy noise infected newspaper images and separate them from graphical regions. The algorithm traces the feature points in different entities and then groups those edge points of textual regions. From using the technology of line approximation and layout categorization, it can successfully retrieve directional placed text blocks. Finally feature based connected component merging was introduced to gather homogeneous textual regions together within the scope of its bounding rectangles. We can obtain correct page decomposition with efficient computation and reduced memory size by handling line segments instead of small pixels. The proposed method has been tested on a large group of newspaper images with multiple page layouts, promising results approved the effectiveness of our method.

Recovery, of document images scanned from thick bound volumes is necessary for the purpose of human reading and text retrieval. The main problem with scanning of bound volumes is that there always occurs perspective distortion. Stich distortion causes two sources of degradation for the scanned images - 1) shadow at the bookspine area, and 2) warping of the words in the shadow. In this paper, we have developed a restoration system to solve these two problems. First, the boundary between the shadow and the clean area is detected. Then the system applies a modified Niblack's method to remove the shadow. The system uses a connected component analysis to help improve the noise reduction and adjust the location and orientation of the warped word in the shadow area, i.e. the words within the boundary detected earlier. The implementation results for each step are presented. Our system will be used in the text retrieval projects for National Archives of Singapore and NUS Digital Library.

Hidden Markov models are a probabilistic modeling tool for time series data. It has been successfully applied to many areas, such as speech recognition, hand-written character recognition, etc. In this paper, we present a novel statistical approach using ergodic hidden Markov models to recognize scientific charts. We also present a newly developed feature extraction method for chart images. Unlike traditional primitive-based diagram recognition method, our approach need not recognize the graphic primitives in charts thus bypassing the recognition error problem caused by the inaccurate primitive extraction that is also a major obstacle to the construction of a general chart recognition system.

We present a novel method for processing double-sided historical handwritten documents using wavelets. The method is specially designed to remove interfering strokes from the reverse side due to ink seeping through pages after long periods of storage. The proposed method works by first matching both sides of a document page such that the interfering strokes are mapped with the originating strokes from the reverse side. Then with the rough identification of the foreground and the interfering strokes, a wavelet reconstruction process iteratively enhances the foreground strokes and smears the interfering strokes so as to strengthen the discriminating capability of an edge detector to extract a clear foreground image. This is an improvement of our earlier work.

We propose a novel coplanar filter, which exploits the coplanarity of the gray-level distribution of neighboring pixels, to pre-filter document images. Experiments show that the proposed filter exhibits the following desired properties for document image binarization: (1) impulsive noise removal; (2) piecewise smoothing; (3) sharp edge preservation.

We propose a word shape recognition method for retrieving image-based documents. Document images are segmented at the word level first. Then the proposed method detects local extrema points in word segments to form so-called vertical bar patterns. These vertical bar patterns form the feature vector of a document. The scalar product of two document feature vectors is calculated to measure the pairwise similarity of document images. The proposed method is robust to changing fonts and styles, and is less affected by degradation of document qualities. Three groups of words in different fonts and image qualities were used to test the validity of our method. Real-life document images were also used to test the method's ability of retrieving relevant documents.

The similarity measure of document images has a crucial role in the area of document image retrieval. A method of measuring the similarity of CCITT Group 4 compressed document images is proposed. The features are extracted directly from the changing elements of the compressed images. Weighted Hausdorff distance is utilized to assign all of the word objects from two document images to corresponding classes by an unsupervised classifier, whereas the possible stop words are excluded. Document vectors are built by the occurrence frequency of the word object classes, and the pair-wise similarity of two document images is represented by the scalar product of the document vectors. Five group articles relating to different domains are used to test the validity of the presented approach.

Perspective distortion always occurs while scanning thick, bound documents. This distortion mainly causes two sources of degradation for the scanned grayscale image: (i) shadow along the 'spine' of the book; and (ii) warp of the words in the shadow. We propose a restoration system to solve these two problems. Our system first produces a vertical projection profile to detect which side of the image the shadow lies on, and a runlength method is used to find the boundary between the shadow and the clean area. We then apply a modified Niblack method to remove the shadow. We use a connected component analysis to adjust the location and orientation of the warped word in the shadow area, ie, the words within the boundary detected earlier. The implementation results are presented. Our system will be used in a text retrieval project for the National Archives of Singapore.

There are several payment protocols mediated by trusted third party functioning as bank (or credit card) agents. But these protocols do not integrate all parts of the protocol together. These agents only establish payment intention but not handle settlement. These protocols rely on traditional payment systems and logistics pathways to complete transactions thus the efficiency is low. In addition, though the agents in these protocols took up some services of the traditional financial conglomerates, risk analysis of the agents is unintentionally neglected. Here we developed an integrative and instantaneous payment protocol-virtual card payment protocol. Broker in our protocol is a bank agent, a financial conglomerate, a logistics assistant, a creditor to score participants' performance and an arbitrator to solve dispute between customers and merchants. In this protocol, one transaction is divided into 3 parts and different parts of the transaction apply different security level. Settlement can be instantaneously handled totally online, without direct involvement of traditional payment systems. Logistics is integrated to process through Internet. We define this technique "pay-on-delivery-online". Since inconsistency in transactions causes customer to complain, we introduce complaint-processing to build trust among involved participants. To dissipate risk concentration in Broker, we propose an algorithm to quantify participants' trustworthiness by estimating participants' transaction history with statistic analysis. Since it assumes that the participants' performance can only be evaluated by those that have established business with them, the model is more objective than poll online and is like the credit scoring or behavioral scoring in real world banking systems.

Neural Logic Networks or Neulonets are hybrids of neural networks and expert systems capable of representing complex human logic in decision making. Each neulonet is composed of rudimentary net rules which themselves depict a wide variety of fundamental human logic rules. An early methodology employed in neulonet learning for pattern classification involved weight adjustments during back-propagation training which ultimately rendered the net rules incomprehensible. A new technique is now developed that allows the neulonet to learn by composing the net rules using genetic programming without the need to impose weight modifications, thereby maintaining the inherent logic of the net rules. Experimental results are presented to illustrate this new and exciting capability in capturing human decision logic from examples. The extraction and analysis of human logic net rules from an evolved neulonet will be discussed. These extracted net rules will be shown to provide an alternate perspective to the greater extent of knowledge that can be expressed and discovered. Comparisons will also be made to demonstrate the added advantage of using net rules, against the use of standard boolean logic of negation, disjunction and conjunction, in the realm of evolutionary computation.

Charts are common graphic representation for scientific data in technical and business papers. We present a robust system for detecting and recognizing bar charts. The system includes three stages, preprocessing, detection and recognition. The kernel algorithm in detection is newly developed modified probabilistic Hough transform algorithm for parallel lines clusters detection. The main algorithms in recognition are bar pattern reconstruction and text primitives grouping in the Hough space which are also original. The experiments show the system can also recognize slant bar charts, or even hand-drawn charts.

Given the large number and complexity of Chinese characters, pattern matching based on structural decomposition and analysis is believed to be necessary and essential to off-line character recognition. The paper proposes a model of stroke extraction for Chinese characters. One problem for stroke extraction is how to extract primary strokes. Another major problem is to solve the segmentation ambiguities at intersection points. We use the degree information and the stroke continuation property to tackle these two problems. The proposed model can be used to extract strokes from both printed and handwritten character images.

We propose a method for constructing a vector for a document image to represent its content to facilitate text retrieval. The method is based on an N-Gram algorithm for text similarity measure based on the frequency of occurrence of n-character strings appearing in the electronic text. Instead of using ASCII values, the present study investigates the use of character images to obtain the document vector and has found promising results for use in our news article retrieval project.

Supervised learning of Neocognitron is fulfilled by presenting training patterns, which map to specified features. However, the training patterns and many parameters are designed empirically and set manually in Fukushima's Neocognitron. In this paper, we use genetic algorithms (GAs) to tune the parameters of Neocognitron and search its reasonable training pattern sets. First of all, the correlation amongst the training patterns is considered as a critical factor affecting Neocognitron's performance, but it is ignored in the design of the original Neocognitron. Then, a GA-based supervised learning of the Neocognitron is proposed to tune the parameters and search training patterns. The results prove that the performance of a Neocognitron is sensitive to its training patterns, selectivity and receptive fields, and can be improved by this supervised learning on the basis of GAs and correlation analysis.

Goodness-of-fit is the most popular criterion for neural network time series forecasting. In the context of financial time series forecasting, we are not only concerned at how good the forecasts fit their targets, but we are more interested in profits. In order to increase the forecastability in terms of profit earning, we propose a profit based adjusted weight factor for backpropagation network training. Instead of using the traditional least squares error, we add a factor which contains the profit, direction, and time information to the error function. The results show that this new approach does improve the forecastability of neural network models, for the financial application domain.

This paper addresses a special problem with historical document images where handwritten characters from the reverse side appear as noise on the front side and even interfere with the front side characters. A novel method to extract clear textual images from interfering and overlapping areas of text is presented here. The proposed algorithm is interesting in that, with an observation that the edges of the seeping strokes from the reverse side are not as sharp as those on the front side, it adopts an edge detection approach to suppress unwanted background patterns. By further concentrating on the orientation of the strokes, other remaining long and strong noisy edges are removed by using an orientation filter and a size filter. The proposed method proves to perform well regardless of the intensity differences between the foreground writing and the interfering strokes. The segmentation results of real images are shown and evaluated.

This paper describes a document image analysis system using multiple agents working on a pyramid structure to separate text from graphics in the image. Text strings appear as different groupings of connected components at different image resolutions. As such, the pyramid structure, which is a multi-resolution image representation, provides a natural means of identifying and grouping of character strings in the document at different levels of resolution. The pyramid structure is also amenable to parallel processing, where multiple agents in the system can individually and concurrently look for groups of connected components at appropriate levels. The agent-based pyramid operations do not require expensive feature analysis among different connected components to detect text strings as found in other existing works.

This work is on the use of multiple attributes or features and spatial relationships, with the help of a user interface based on an iconic paradigm, to retrieve images represented by iconic pictures. An icon has texture, color and text attributes. Texture is represented by 3 statistical textural properties, namely, coarseness, contrast and directionality. For text, the vector space model is used. For color, a representation based on a modified color histogram method which is less storage-intensive is proposed. The final icon similarity is the combination of the attribute similarity values using a proven adaptive algorithm. 2-D strings and its variants are commonly used to represent spatial relationships and perform spatial reasoning. However, the basic methods do not perform similarity ranking. We extended the method to include similarity ranking by using different similarity functions for different spatial relationships and an efficient embedding algorithm. Furthermore, our method, on top of other benefits, solves the problem of query expressiveness which all methods based on 2-D string representations suffer from.

Presents the architecture of a hybrid neural network expert system shell. The system, structured around the concept of network elements, is aimed at preserving the semantic structure of the expert system rules while incorporating the learning capability of neural networks into the inferencing mechanism. Using this architecture, every rule of the knowledge base is represented by a one or two-layer neural network element. These network elements are dynamically linked up to form the rule-tree during the inferencing process. The system is also able to adjust its inference strategy according to different users and situations. An editor is also provided to enable easy maintenance of the neural network rule elements. The shell is housed in a user-friendly rule-based interface. Two applications that are built upon the abovementioned shell are discussed, they demonstrate the strengths of the network element architecture over conventional rule-based systems.