In this paper we present AMPNet, an acoustic abnormality detection model deployed at ACV Auctions to automatically identify engine faults of vehicles listed on the ACV Auctions platform. We investigate the problem of engine fault detection and discuss our approach of deep-learning based audio classification on a large-scale automobile dataset collected at ACV Auctions. Specifically, we discuss our data collection pipeline and its challenges, dataset preprocessing and training procedures, and deployment of our trained models into a production setting. We perform empirical evaluations of AMPNet and demonstrate that our framework is able to successfully capture various engine anomalies agnostic of vehicle type. Finally we demonstrate the effectiveness and impact of AMPNet in the real world, specifically showing a 20.85% reduction in vehicle arbitrations on ACV Auctions' live auction platform.

This work summarizes the results of the first Competition on Harvesting Raw Tables from Infographics (ICDAR 2019 CHART-Infographics). The complex process of automatic chart recognition is divided into multiple tasks for the purpose of this competition, including Chart Image Classification (Task 1), Text Detection and Recognition (Task 2), Text Role Classification (Task 3), Axis Analysis (Task 4), Legend Analysis (Task 5), Plot Element Detection and Classification (Task 6.a), Data Extraction (Task 6.b), and End-to-End Data Extraction (Task 7). We provided a large synthetic training set and evaluated submitted systems using newly proposed metrics on both synthetic charts and manually-annotated real charts taken from scientific literature. A total of 8 groups registered for the competition out of which 5 submitted results for tasks 1-5. The results show that some tasks can be performed highly accurately on synthetic data, but all systems did not perform as well on real world charts. The data, annotation tools, and evaluation scripts have been publicly released for academic use.

Online lecture videos are increasingly important e-learning materials for students. Automated content extraction from lecture videos facilitates information retrieval applications that improve access to the lecture material. A significant number of lecture videos include the speaker in the image. Speakers perform various semantically meaningful actions during the process of teaching. Among all the movements of the speaker, key actions such as writing or erasing potentially indicate important features directly related to the lecture content. In this paper, we present a methodology for lecture video content extraction using the speaker actions. Each lecture video is divided into small temporal units called action segments. Using a pose estimator, body and hands skeleton data are extracted and used to compute motion-based features describing each action segment. Then, the dominant speaker action of each of these segments is classified using Random forests and the motion-based features. With the temporal and spatial range of these actions, we implement an alternative way to draw key-frames of handwritten content from the video. In addition, for our fixed camera videos, we also use the skeleton data to compute a mask of the speaker writing locations for the subtraction of the background noise from the binarized key-frames. Our method has been tested on a publicly available lecture video dataset, and it shows reasonable recall and precision results, with a very good compression ratio which is better than previous methods based on content analysis.

The goal of a writer adaptive handwriting recognition system is to build a model that improves the recognition of a generic recognition model for a specific author. In this work, we show how structural representation learned from a generic writer-independent handwriting recognition model can be customized to individual authors. Convolutional Neural Network has shown outstanding performance in learning image-based representation that was used for classification. Additionally, they have been used along with Recurrent Neural Network (RNN) or its variations like, LSTM and GRU layers to analyze and understand sequences in handwriting recognition, sentence analysis, voice recognition etc. In most cases, the CNNs serve as a feature extractor instead of low-level hand-designed features that were used previously for the above-mentioned classification tasks. We design a method to reuse weights from layers trained on the IAM offline handwritten dataset to compute mid-level image representation for text in the Washington and Moore dataset. We show that despite differences in the writing style, fonts across these datasets, the transferred representation is able to capture a spatio-temporal representation leading to significantly improved recognition results. We hypothesize that the performance is solely not dependent on the number of samples and the model is evaluated with varying amount of fine-tuning samples showing promising results backing the hypothesis.

Information graphics, such as bar charts, graphs, plots etc. in scientific documents primarily facilitate better understanding of information. Graphics are a key component in technical documents as they are simplified representations of complex ideas. When the traditional optical character recognition (OCR) systems is used on digitized documents, we lose the ideas conveyed in these information graphics since OCRs typically work only on text. And although in more recent times, tools have been developed to extract information graphics from pdf files, they still do not intelligently interpret the contents of the extracted graphics. We therefore propose a method for identifying the intended messages of line plots using a Bayesian network. We accomplish this by first extracting a dense set of points in from a line plot and then represent the entire line plot as a sequence of trends. We then implement a Bayesian network for reasoning about the messages conveyed by the line plots and their trends. We validate our approach by performing experiments on a dataset obtained from computer science conference publications and evaluate the performance of the network against the messages generated by human end users. The resulting intended message gives holistic information about the line plot(s) as well as lower level information about the trends that make up the plot.

The challenges of assessing the "health" of online social media platforms and strategically growing them are recognized by many practitioners and researchers. For those platforms that primarily rely on user-generated content, the reach -- the degree of participation referring to the percentage and involvement of users -- is a key indicator of success. This paper lays a theoretical foundation for measuring engagement as a driver of reach that achieves growth via positive externality effects. The paper takes a game theoretic approach to quantifying engagement, viewing a platform's social capital as a cooperatively created value and finding a fair distribution of this value among the contributors. It introduces engagement capacity, a measure of the ability of users and user groups to engage peers, and formulates the Engaging Team Formation Problem (EngTFP) to identify the sets of users that "make a platform go". We show how engagement capacity can be useful in characterizing forum user behavior and in the reach maximization efforts. We also stress how engagement analysis differs from influence measurement. Computational investigations with Twitter and Health Forum data reveal the properties of engagement capacity and the utility of EngTFP.

Information graphics, such as graphs and plots, are used in technical documents to convey information to humans and to facilitate greater understanding. Usually, graphics are a key component in a technical document, as they enable the author to convey complex ideas in a simplified visual format. However, in an automatic text recognition system, which are typically used to digitize documents, the ideas conveyed in a graphical format are lost. We contend that the message or extracted information can be used to help better understand the ideas conveyed in the document. In scientific papers, line plots are the most commonly used graphic to represent experimental results in the form of correlation present between values represented on the axes. The contribution of our work is in the series of image processing algorithms that are used to automatically extract relevant information, including text and plot from graphics found in technical documents. We validate the approach by performing the experiments on a dataset of line plots obtained from scientific documents from computer science conference papers and evaluate the variation of a reconstructed curve from the original curve. Our algorithm achieves a classification accuracy of 91% across the dataset and successfully extracts the axes from 92% of line plots. Axes label extraction and line curve tracing are performed successfully in about half the line plots as well.

Word level handwritten CAPTCHA generation involves picking a handwritten word from a pre-existing database and cumulatively applying distortions and noise models. In principle, the addition of distortion and noise makes the CAPTCHA robust to automated attacks. However, the primary drawback of the word level CAPTCHA generation is that it limits us to words that already exist in our data set. If the primary building block of this approach was a character, we could move away from a lexicon based CAPTCHA generation and generate CAPTCHAs which are resistant to a dictionary based attack. In this paper, we propose a Sigma-Lognormal based approach to generate character level CAPTCHAs. Next, we increase the robustness of the model by applying ideas from accents in handwriting to our problem. Finally, we demonstrate the efficacy of our approach by simulating an attack by an automated word recognizer.

Writer Identification can be seen as a multi-class learning problem where number of writers are different classes. One of the fundamental approaches to solve a multi-class problemis by breaking it into binary classification tasks. In this work weare proposing a generic approach for multi-class classification using an ensemble of binary classifiers. We assign a distributedoutput representation to each class in the form of codewords andan ensemble of binary classifiers is created where each classifierpredicts one bit of the codeword. Actual label is determined using Belief Propagation algorithm on a graph constructed from the code matrix. We have performed experiments on a new publiclyavailable IBM-UB-1 dataset for the task of writer identification to show the efficacy of our method.

Writer identification is the process of determining the author of a handwritten specimen by utilizing characteristics inherent in the sample. In this work, we apply the concept of accents in handwriting to introduce a novel perspective for writer identification. Analogous to speech, accents in handwriting can be defined as distinctive writing quirks that are unique to a group of people sharing a common native script. Specifically, we postulate that a group of people with a common native script will share certain traits in their handwriting style that are exposed when they write in a different script. We propose a hierarchical framework for the writer identification task, wherein, we first identify the accent of the writer. In the next step, we perform writer identification based on the selected accent. This framework reduces the complexity of the classification task by reducing the number of classes at the prediction stage. Experiments are performed on the UNIPEN dataset and the results lend credibility to our model.

Accent in speech is defined as a distinctive mode of pronunciation that is unique to a geographical region. In a similar way, we define accent in handwriting as distinctive writing characteristics that are unique to a group of people sharing a common native script. In other words, we postulate that a group of people with a common native script will share certain traits in their handwriting that can be ascertained when they write in a different script. In this paper, we establish the existence of accents in handwriting using a hierarchical Bayesian framework. We then demonstrate that the unique trait in handwriting that arises out of the writer's native script is indigenous to that script, which is perceivable when writing in a different script. As a consequence, the ability to identify a person's native script based on the person's handwriting style in another script is introduced. We validated the approach by performing experiments on the UNIPEN dataset, and the experiments lend credibility to our model.

This paper describes a model based framework for detection and extraction of the contents of table cells from degraded handwritten document images that contain tables. Given the very poor quality of the target documents, the table cell detection problem is formulated conceptually as a two-step process. The first step is to identify the location of the table and extract the content of table cells given a model of the structure of the table present in the image. The second step is to identify the model of the table present in a document image from a list of given table models. A model-based representation for tables is introduced and is used for matching table candidates with the given model to identify and extract the contents of table cells. The approach for detecting potential table candidates is based on the detection of horizontal and vertical table line candidates. The table representation is a matrix of horizontal and vertical table line crossings, and the matching algorithm is formulated as a minimization problem where the optimal table candidate is obtained using the minimal distance between the candidate and model table matrices which is then used for extraction of the table cell contents. A similar approach is used to solve the model selection problem where the best fitting location in the document page for each of the candidate models is identified using the distance minimization approach along with a confidence score and the model with the highest confidence score is selected as the correct model. The approach was tested on document page images containing tables from the challenge set of the DARPA MADCAT handwritten document image data. Results indicate that the method is effective for both model selection as well as table cell content extraction.

In this paper we present a new dual mode, twin-folio structured English handwriting dataset IBM_UB_1. IBM_UB_1 is our first major release from a large multilingual handwriting corpus. Containing over 6000 pages of handwritten matter, this dataset can not only be used for unconstrained handwriting recognition, more importantly, the dataset's unique twin-folio structure presents a natural fit for research on writer identification, keyword spotting, indexing and various forms of handwritten document search and retrieval. We first describe two central characteristics of the dataset - the twin-folio structure and dual modality (online/offline) - and their relevance to current research problems. Secondly, we describe the dataset, its collection and construction, and provide key descriptive statistics. Finally, we evaluate the dataset on two different research domains - handwriting recognition and writer identification - and present related experimental results.

We propose a segmentation based online word recognition approach which uses a Conditional Random Field (CRF) driven beam search strategy. An efficient trie-lexicon directed, breadth-first beam search algorithm is employed in a combined segmentation-and-recognition framework to accomplish real-time recognition of online handwritten cursive English words. This framework is developed by building a candidate lattice of primitive segments obtained through over segmentation of the word pattern. The search space for the lattice is expanded by synchronously matching the lattice nodes to likely character patterns from a trie-dictionary constructed out of the target lexicon. The probable paths are evaluated by integrating character recognition scores with physical and spatial characteristics of the handwritten segments in a CRF (conditional random field) model and a beam search strategy is used to prune the set of likely paths. This approach has been benchmarked on the new IBM_UB_1 dataset as well as on the UNIPEN dataset for comparison.

This paper describes an online handwritten English cursive word recognition method using a segmentation-free Markov random field (MRF) model in combination with an offline recognition method which uses pseudo 2D bi-moment normalization (P2DBMN) and modified quadratic discriminant function (MQDF). It extracts feature points along the pen-tip trace from pen-down to pen-up and uses the feature point coordinates as unary features and the differences in coordinates between the neighboring feature points as binary features. Each character is modeled as a MRF and word MRFs are constructed by concatenating character MRFs according to a trie lexicon of words during recognition. Our method expands the search space using a character-synchronous beam search strategy to search the segmentation and recognition paths. This method restricts the search paths from the trie lexicon of words and preceding paths, as well as the lengths of feature points during path search. We also combine it with a P2DBMN-MQDF recognizer that is widely used for Chinese and Japanese character recognition.

Techniques and performance of text recognition systems and software has shown great improvement in recent years. OCRs now can read any machine printed document with good accuracy. However, the advancements are primarily for Latin scripts and even for such scripts performance is limited in case of handwritten documents. Little work has been done for cursive scripts such as Arabic and still there is a room for improvement both in terms of accuracy and techniques. This paper presents an algorithm to recognize handwritten Arabic text using an ensemble of biased classifiers in a hierarchical setting. We address the fundamental shortcomings of the traditional Machine Learning paradigms when applied to Arabic scripts. Experiments have been conducted on the AMA Arabic dataset to show the efficacy of our method.

Accent in handwriting can be defined as the influence of a writer's native script on his/her writing style in another script. In this paper, we approach the problem of detecting the existence of accents in handwriting. We approach this problem using two sets of writers, those who can write only in English, and the other set being multilingual writers who can also write in English. We learn the writing styles that are predominant in each set and use it as features in classification. Latent Dirichlet Allocation is used to learn the distribution over writing styles. Experimental results suggest the existence of accents in handwriting.

This paper presents a novel set of image enhancement algorithms for binary images of poorly scanned real world page documents. Problems that are targeted by the methods described include large blobs or clutter noise, salt-and-pepper noise and detection and removal of non-text objects such as form lines or rule-lines. The algorithms described are shown to be very effective in removing clutter noise and pepper noise as well as form lines and rule-lines. A region growing algorithm is also described to enhance the quality of the text and to fix the problems arising from the salt noise which leaves holes in the text and creates broken strokes. The methods were tested on 204 images from the challenge set of the DARPA MADCAT Arabic handwritten document image data. The results indicate that the methods described are robust and are capable of significantly improving the image quality for downstream OCR systems.

We present a method for figure-panel (subfigure) label detection and recognition in multi-panel figures extracted from biomedical articles. Figures in biomedical articles often comprise several subfigures that are identified by superimposed panel labels ('A', 'B', ...) which are referenced in the figure caption and discussion in the article body. Splitting such multi-panel figures into individual subfigures is a necessary step for improved multimodal biomedical information retrieval. Prior to feature extraction for indexing and retrieval of biomedical figures it is necessary to classify image content in each subfigure by its modality (X-ray, MRI, CT, etc.) and other relevant criteria. Subfigure labels are valuable in associating individual panels with relevant text in captions and discussion. We propose a 4-step panel label detection method based on Markov Random Field (MRF). Experiments on 515 multi-panel figures and analysis of the results show promising results. We present the successes and identify critical challenges.

In this paper, we describe a novel approach to Writer Identification in Offline handwriting using Latent Dirichlet Allocation. State-of-the-art methods for writer identification employ the traditional feature-classification paradigm which does not provide enough information about the handwriting attributes such as writing style which are key components in any forensic analysis of handwriting. This problem is also compounded due to lack of efficient rules for defining a particular writing style that can capture writer specific characteristics over a large dataset. We propose to address this issue by using a generative model in form of Latent Dirichlet Allocation(LDA) that automatically infers writing styles from handwritten document collection without any pre-defined set of rules. This information is then used to represent each writer as a distribution over multiple writing style for classifying any unknown writer sample. We describe our approach on two different feature sets consisting of contour angle features as well as structural and concavity features. Our experimental results show comparable performance with baseline systems and also demonstrate the efficacy of LDA for learning multiple handwriting styles.

Separating machine printed text and handwriting from overlapping text is a challenging problem in the document analysis field and no reliable algorithms have been developed thus far. In this paper, we propose a novel approach for separating handwriting from binary image of overlapped text. Instead of using fixed size training patches, we describe an aggregation method which uses shape context features to extract training samples automatically. We use a Markov Random Field (MRF) to model the overlapped text. The neighbor system is inherited from a coarsening procedure and the prior and likelihood of the MRF is learned based on a distance metric. Experimental results show that the proposed method can achieve 87.97% recall for handwriting and 91.44% recall for machine printed text.

In this paper, we describe an approach to segment handwritten text, machine printed text and noise from annotated machine printed documents. Three categories of word level features are extracted. We use a modified K-Means clustering algorithm for classification followed by a relabeling procedure using Markov Random Field(MRF) based on a concept of neighboring patches and Belief Propagation(BP) rules. Experimental results on an imbalanced data set show that our approach achieves an overall recall of 96.33%.

In this paper, we present a new text line extraction method for handwritten Arabic documents. The proposed technique is based on a generalized adaptive local connectivity map (ALCM) using a steerable directional filter. The algorithm is designed to solve the particularly complex problems seen in handwritten documents such as fluctuating, touching or crossing text lines. The proposed algorithm consists of three steps. Firstly, a steerable filter is used to probe and determine foreground intensity along multiple directions at each pixel while generating the ALCM. The ALCM is then binarized using an adaptive thresholding algorithm to get a rough estimate of the location of the text lines. In the second step, connected component analysis is used to classify text and non text patterns in the generated ALCM to refine the location of the text lines. Finally, the text lines are separated by superimposing the text line patterns in the ALCM on the original document image and extracting the connected components covered by the pattern mask. Analysis of experimental results on the DARPA MADCAT Arabic handwritten document data indicate that the method is robust and is capable of correctly isolating handwritten text lines even on challenging document images.

We propose a new algorithm for segmentation of off-line handwritten Arabic words. The algorithm segments the connected letters to smaller segments each of which contains no more than three letters. Each letter may be segmented to at most five pieces. In addition to improving the recognition of Arabic words, another potential application of the proposed segmentation method is to build lexicon of small size, consisting of no more than three letter combinations. Generally, it is very hard to generate lexicon for recognition of unconstraint handwritten Arabic documents due to the large number of words of Arabic language.The algorithm has been tested on over 6300 words from 45 different documents written by 18 writers. The system is able to segment more than 93% of the words into segments, each containing at most one letter, 6% of the words into segments that contains two letters and 3% of the words into segments that contains three letters.

We propose a novel hierarchical classification method for documents categorization in this paper. The approach consists of multiple levels of classification for different hierarchies. Regularized Least Square (RLS)binary classifiers are applied in the middle levels of the hierarchy to classify documents into smaller set of categories and K-nearest-neighbor (KNN) multi-class classifiers are used at the bottom to classify documents into final classes. Experiments on large-scale real world tax documents show that the proposed hierarchical approach outperforms traditional flat classification method.

Despite several decades of research in handwriting recognition, the goal of having computers access handwritten information from unconstrained document images is still elusive. Current handwriting recognition systems are only capable of recognizing words that are present in a restricted lexicon typically comprised of 10 to 1000 words. As the size of the lexicon grows, the recognition accuracy falls sharply and is reported to be around 30% for a10K word lexicon. The objective of this research is to raise the accuracy levels on unconstrained handwritten documents by reducing the size of lexicons. We present an innovative method of lexicon reduction by topic categorization of handwritten documents. After categorization of a document into a topic e.g. sports, science etc. we use smaller lexicons that include only words with high mutual information with that topic and hence increase performance of recognizers. In this paper we present different techniques and report results on a publicly available dataset.

A vector model based information retrieval of handwritten medical forms is presented in this paper. In order to improve the IR performance on the erroneous output of handwriting recognition (HR) systems, a variation of the vector model is made to estimate the number of occurrences of terms from word segmentation and recognition probabilities. IR Tests show that our approach outperforms the retrieval of ordinary HR text in terms of mean average precision (MAP), R-Precision, and interpolated 11-point precisions.

Partial Differential Equations are becoming one of the core tools for low-level image processing. They are especially functional in diffusion processes and variational models. In this paper, we exploit the regional smoothing that occurs in a nonlinear diffusion process and use this to enhance text in a degraded document image. The proposed smoothing method is robust when applied to either a highly corrupted text document or one with little degradation. The technique was tested on historical documents, carbon copies with highly varying grayscale backgrounds and on synthetic noisy documents. The PDE-based method far outperformed other industry-standard binarization techniques when compared quantitatively and qualitatively.

In order to improve the readability and the automatic recognition of handwritten document images, preprocessing steps are imperative. These steps in addition to conventional steps of noise removal and filtering include text normalization such as baseline correction, slant normalization and skew correction. These steps make the feature extraction process more reliable and effective. Recently Arabic handwriting recognition has received some attention from the research community. Due to the unique nature of the script, the conventional methods do not prove to be effective. In our work, we describe an orientation independent technique for baseline detection of Arabic words. In addition to that we describe, in the rest of the paper, our techniques for slant normalization, slope correction, line and word separation in handwritten Arabic documents. We show how the baseline can be exploited for slope and skew correction before proceeding with the steps of line and word separation.

In this paper, we propose a new feature ranking method in a recognition system, by introducing the concept of the effectiveness of the distinguishing power of features and considering the correlation among features. To find the subset of most important features, first, the best feature can be identified by its effective distinguishing power and put in an empty feature set. Then, each of the remaining features is ranked based on their effective distinguishing capacity contribution and the highest-ranked feature is added to the selected subset. This process is repeated till the performance of the system reaches its peak or the effective distinguishing contribution falls below a certain value. The application of this method to an existing handwriting recognition system showed strong support for our methodology of feature ranking.

OCR of Devanagari script presents a wide range of challenges that are not seen in Latin based scripts. This paper outlines the implementation of a neural network based Devanagari OCR. Experimental results on a standard data set are reported and analyzed.

A novel sequence classification method is proposed in the context of support vector machines (SVM). This method is driven by an intuitive similarity measure, namely ER/sup 2/, which directly tells the similarity of two sequences (1- or multi-dimensional). If sequence X is very similar to Y (for instance, the similarity by ER/sup 2/ is above 90%), it is safe to assign X to the same class as Y. ER/sup 2/ is plugged into standard SVM to speed up the decision-making of multi-class classification. The immediate application of the method is in the adaptive online handwriting recognition, where handwritten characters are represented by 2D sequences of X-, Y-coordinates. Experiments on the benchmark database UNIPEN show that the classification driven by ER/sup 2/ can be about three times faster than standard SVM while the classification accuracy is enhanced or comparable.

We propose a novel algorithm for the segmentation and prerecognition of offline handwritten Arabic text. Our character segmentation method over-segments each word, and then removes extra breakpoints using knowledge of letter shapes. On a test set of 200 images, 92.3% of the segmentation points were detected correctly, with 5.1% instances of over-segmentation. The prerecognition component annotates each detected letter with shape information, to be used for recognition in future work.

Traditional handwriting recognition algorithms rely heavily on small lexicons and clean word images. Unfortunately, emergency medical documents do not satisfy either of these conditions. This is a significant road-block that is hampering efforts to rapidly convert valuable offline healthcare handwriting data into digital content that can be efficiently mined for information. This paper describes a strategy whereby given an image representing a noisy handwritten word from a medical document, and a large lexicon consisting of English, medical and pharmacological words, symbols, abbreviations and acronyms, significantly reduces the size of the lexicon while keeping the unknown desired entry within the lexicon. The approach combines geometric interpretations of the word image along with contextual inference of concepts to reduce lexicons for word recognition. The data extracted can then be efficiently and securely disseminated for epidemiological and outbreak detection/analysis. Experimental results on NY State PCR forms are reported.

The recognition of unconstrained handwriting continues to be a difficult task for computers despite active research for several decades. This is because handwritten text offers great challenges such as: character and word segmentation, character recognition, variation between handwriting styles, different character size and orientation, no font constraints, the type of printing surface, as well as the background clarity. In this paper, we explore the gap in the ability in reading handwritten text between humans and computers to propose solutions for security problems in Web services. We present a new HIP algorithm that uses handwriting recognition task to distinguish between humans and computers. We propose methods to deform handwritten text images to make them indecipherable by computers and explore the cognitive factors that assist humans in reading and understanding. Experimental results on both humans and computers are presented and compared.

Page segmentation algorithms found in published literatures often rely on some predetermined parameters such as general font sizes, distances between text lines and document scan resolutions. Variations of these parameters in real document images greatly affect the performance of the algorithms. In this paper, we present a novel approach for document page segmentation using a multi-scale technique. An efficient implementation of a local connectivity algorithm transforms a document image into a parameter domain in which a parameter value at a pixel location represents a connectivity property for its neighboring foreground pixels in the original document image. Then a top-down approach with a linear search reveals the document regions at each scale levels as text block, text lines and graphics. We consider our algorithm a transform based multi-scale method. Our ongoing research shows that the algorithm is robust for variations of document parameters.

This paper presents an algorithm using adaptive local connectivity map for retrieving text lines from the complex handwritten documents such as handwritten historical manuscripts. The algorithm is designed for solving the particularly complex problems seen in handwritten documents. These problems include fluctuating text lines, touching or crossing text lines and low quality image that do not lend themselves easily to binarizations. The algorithm is based on connectivity features similar to local projection profiles, which can be directly extracted from gray scale images. The proposed technique is robust and has been tested on a set of complex historical handwritten documents such as Newton's and Galileo's manuscripts. A preliminary testing shows a successful location rate of above 95% for the test set.

The paper discusses a problem of combining recognition scores for different classes produced by one recognizer during one recognition attempt. This problem arises in identification problems which we define as 1:N classification problems with big or variable N. By using artificial example we show that intuitive solution of making identification decision based solely on the best matching score is frequently suboptimal. Paper presents reasons for such behavior, and draws parallels with score normalization technique used in speaker identification. Two examples of real life applications illustrate the possible benefits of properly combining recognition scores.

An intrusion is defined as any set of actions that compromise the integrity, confidentiality or availability of a resource. Intrusion detection is an important task for information infrastructure security. One major challenge in intrusion detection is that we have to identify the camouflaged intrusions from a huge amount of normal communication activities. Data mining is to identify valid, novel, potentially useful, and ultimately understandable patterns in massive data. It is demanding to apply data mining techniques to detect various intrusions. In the last several years, some exciting and important advances have been made in intrusion detection using data mining techniques. Research results have been published and some prototype systems have been established. Inspired by the huge demands from applications, the interactions and collaborations between the communities of security and data mining have been boosted substantially. This seminar will present an interdisciplinary survey of data mining techniques for intrusion detection so that the researchers from computer security and data mining communities can share the experiences and learn from each other. Some data mining based intrusion detection systems will also be reviewed briefly. Moreover, research challenges and problems will be discussed so that future collaborations may be stimulated. For data mining/database researchers and practitioners, the seminar will provide background knowledge and opportunities for applying data mining techniques to intrusion detection and computer security. For computer security researchers and practitioners, it provides knowledge on how data mining can benefit and enhance computer security. We will try to understand and appreciate the following technical issues.

Sequential pattern matching and retrieving is of real value. For example, finding stocks in the NASDAQ market whose closing prices are always about $β₀ higher than or β₁ times as that of a given company. The probelm reduces to linear pattern retrieval: given query X, find all sequence Y from database S so that Y = β₀ + β₁ with confidence C. In this paper, we novelly introduce SLR (Simple Linear Regression) model [5,7] to solve this problem. We extend 1-dimensional R^2 to ER^2 for multi-dimensional sequence matching, such as on-line handwritten signature. In addition, we develop SLR+FFT pruning techniques based on SLR to speed up retrieval without incurring any false dismissal. Experimental results show that the pruning ratio of SLR+FFT is efficient (can be above 99%). Experiments on real stocks discovered many interesting patterns. Preliminary test on on-line signature recognition using ER^2 as similarity measure also shows high accuracy.

We have developed a well performing algorithm for locating address blocks in postal parcel images. Both machine printed and handwritten addresses are processed by the algorithm. The algorithm is invariant to the image orientation and scale, and it works with high noise images. It could also serve as an additional step after other address block location algorithms.

In this paper we present a top-down, projection-profile based algorithm to separate text blocks from image blocks in a Devanagari document. We use a distinctive feature of Devanagari text, called Shirorekha (Header Line) to analyze the pattern produced by Devanagari text in the horizontal profile. The horizontal profile corresponding to a text block possesses certain regularity in frequency, orientation and shows spatial cohesion. The algorithm uses these features to identify text blocks in a document image containing both text and graphics.

A skew angle estimation approach based on the application of a fuzzy directional runlength is proposed for complex address images. The proposed technique was tested on a variety of USPS parcel images including both machine print and handwritten addresses. The testing results showed a successful rate more than 90% of the test set.

This paper describes the issues involved in the design of a system for evaluating improvements in the performance of a real-time address recognition system being used by the United States Postal Service for processing mail-piece images. Evaluation of the performance of recognition systems is normally carried out by measuring the performance of the system on a representative sample of images. Designing a comprehensive and valid testing scenario is a complex task that requires careful attention. Sampling live mail-stream to generate a deck of images representative of the general mail-stream for testing, truthing (generating reference data on a significant number of images), grading and evaluation, and designing tools to facilitate these functions are important topics that need to be addressed. This paper describes the efforts of the United States Postal Service and CEDAR towards developing an infrastructure for sampling, truthing and testing of mail-stream images.

Given sufficient resources, any classification task is possible with a high accuracy, but to achieve a particular task given finite resources, the problem is to utilize these resources intelligently. Cognitive studies in human vision associate multi-resolution features with high recognition accuracy. We show that classifier development using separability optimization is very similar to emulation of human cognition. The identification of key features leads to optimal resource utilization by the classifier. Evolving such classifiers is the focus of the paper. The resources required for classification can be identified in terms of amount of time required to develop a recognizer amount of processing power required and the number and kind of features extracted. Our digit recognition method strives not only to report high accuracy but also targets generation of simple solutions. The simplicity of a solution can be a measure of the resources utilized. Our methodology is termed as active based on the premise that once the complexity of a classification task is known an intelligent recognizer should incrementally increase the resources needed for classification.

We describe the construction of a model for off-line word recognizers based on over-segmentation of the input image and recognition of segment combinations as characters in a given lexicon word. One such recognizer, the Word Model Recognizer (WMR), is used extensively. Based on the proposed model it was possible to improve the performance of WMR.

Presents a method of building skeletal graphs for handwriting images, aiming at extraction of high-level structural features such as loops, turns, ends, and junctions. Block adjacency graphs are used as the base representation and transformed at locations where deformation occurs to obtain satisfactory skeletal graphs. Then the identification and ordering of structural features are considered based on skeletal graphs.

An efficient system which finds the best match between an input image and a lexicon is presented. To capture writing style of spacing between words and characters prime stroke analysis based on statistical methods is introduced. A method for estimating bound on number of characters without actual recognition is also presented. For system efficiency, before actual recognition, classified groups of word segments and eligible subset of lexicons are generated as hypotheses. The hypotheses are verified and ordered by a lexicon driven word recognizor. We have tested our approach in the street name recognition/interpretation for US mail stream. Experimental results and encouraging.

This paper concerns a study of information content in postal address fields for automatic address interpretation. Information provided by a combination of address components and information interaction among components is characterized in terms of Shannon's entropy. The efficiency of assignment strategies for determining a delivery point code can be compared by the propagation of uncertainty in address components. The quantity of redundancy between components can be computed from the information provided by these components. This information is useful in developing a strategy for selecting a useful component for recovering the value of an uncertain component. The uncertainty of a component based on another known component can be measured by conditional entropy. By ranking the uncertainty quantity, the effective processing flow for determining the value of a candidate component can be constructed.

84.6% of touching digit strings have only two digits touching, 12.3% have three digits touching and 3.1% have more than three digits touching. We present a multi-expert approach to recognize touching digit pairs (TDP) and touching digit triples (TDT). We combine holistic and traditional segmentation methods. 25,686 TDP training samples and 2,778 TDP testing samples collected from USPS mail are used in our experiment. The holistic method outperforms the traditional segmentation-based methods. The multi-expert combination has the best performance: a correct recognition rate of 91.1% on TDP.

The one-dimensional nature of contour representations presents interesting challenges for processing of images for handwritten word recognition. In this paper, we discuss the issues of determination of upper and lower contours of the word, determination of significant focal extrema on the contour, and determination of reference lines from contour representations of handwritten words.

The authors describe ongoing research on a system for rapid verification of unconstrained off-line handwritten phrases using perceptual holistic features of the handwritten phrase image. The system is used to verify handwritten street names automatically extracted from live US mail against recognition results of analytical classifiers. The system rejects errors with 98% accuracy at the 30% accept level, while consuming approximately 20 msec per image on the average on a 150 MHz SPARC 10.

A fast handwritten word recognition system for real time applications is presented. Preprocessing, segmentation and feature extraction are implemented using chain code representation. Dynamic matching between each character of a lexicon entry and segment(s) of input word image is used for ranking words in the lexicon. Speed of the entire recognition process is about 200 msec on a single SPARC-10 platform for lexicon size of 10. A top choice performance of 96% is achieved on a database of postal words captured at 212 dpi.

The performance of off-line handwritten word recognition algorithms declines with increasing lexicon size, but may be improved by serial combination of classifiers. The authors address some issues relevant to the design of serial classifier combinations. They present experimental results that show that the performance of a serial combination depends on not only the intrinsic recognition power of the classifiers but also the relative orthogonality of their features. A top-choice recognition rate of 83% is obtained for a lexicon of size 1700 by combining two analytical word classifiers that perform individually at 70%. Even higher recognition rates may be expected from a serial combination of two classifiers with less correlated features, such as a high-performance holistic classifier with an analytical classifier.

Commercial forms-reading systems for extraction of data from forms do not meet acceptable accuracy requirements on forms filled out by hand. In December 1993, NIST called industry and research organizations working in the area of handwriting recognition to participate in a test to determine the state of the art in the area. A database of form images containing actual responses received by the US Census Bureau was provided. The handwritten responses are very loosely constrained in terms of writing style, format of response and choice of text. The sizes of the lexicons provided are very large (about 50000 entries) and yet the coverage is incomplete (about 70%). In this paper we discuss the approach taken by CEDAR to automate the task of reading the census forms. The subtasks of field extraction and phrase recognition are described.