International audience

State-of-the-art methods for handwriting text recognition are based on deep learning approaches and language modeling that require large data sets during training. In practice, there are some applications where the system processes mono-writer documents, and would thus benefit from being trained on examples from that writer. However, this is not common to have numerous examples coming from just one writer. In this paper, we propose an approach to adapt both the optical model and the language model to a particular writer, from a generic system trained on large data sets with a variety of examples. We show the benefits of the optical and language model writer adaptation. Our approach reaches competitive results on the READ 2018 data set, which is dedicated to model adaptation to particular writers.

Deep learning approaches now provide state-of-the-art performance in many computer vision tasks such as handwriting recognition. However, the huge number of parameters of these models require big annotated training datasets which are difficult to obtain. Training neural networks with unlabeled data is one of the key problems to achieve significant progress in deep learning. In this article, we explore a new semi-supervised training strategy to train long-short term memory (LSTM) recurrent neural networks for isolated handwritten words recognition. The idea of our self-training strategy relies on the iteration of training Bidirectional LSTM recurrent neural network (BLSTM) using both labeled and unlabeled data. At each iteration the current trained network labels the unlabeled data and submit them to a very efficient "lexicon verification" rule. Verified unlabeled data are added to the labeled dataset at the end of each iteration. This verification stage has very low sensitivity to the lexicon size, and a full word coverage of the dataset is not necessary to make the semi-supervised method efficient. The strategy enables self-training with a single BLSTM and show promising results on the Rimes dataset.

We introduce a novel handwriting recognition approach based on sub-lexical units known as multigrams of characters, that are variable lengths characters sequences. A Hidden Semi Markov model is used to model the multigrams occurrences within the target language corpus. Decoding the training language corpus with this model provides an optimized multigram lexicon of reduced size with high coverage rate of OOV compared to the traditional word modeling approach. The handwriting recognition system is composed of two components: the optical model and the statistical n-grams of multigrams language model. The two models are combined together during the recognition process using a decoding technique based on Weighted Finite State Transducers (WFST). We experiment the approach on two Latin language datasets (the French RIMES and English IAM datasets) and we show that it outperforms words and character models language models for high Out Of Vocabulary (OOV) words rates, and that it performs similarly to these traditional models for low OOV rates, with the advantage of a reduced complexity.

In this article, we propose to benchmark the most popular methods for word spotting in handwritten documents. The benchmark includes a pure HMM approach, as well as hybrid discriminative methods MLP-HMM, CRF-HMM, RNN-HMM and BLSTM-CTC-HMM. This study enables us to observe the increase ratio of performance provided by each discriminative stage compared with the pure generative HMM approach. Moreover, we put forward the different abilities of all these discriminative stages from the simplest MLP to the most complex and current state of the art BLSTM-CTC. We also propose a more specific and original study on BLSTM-CTC, showing that when used as a lexicon-free recognizer, it can reach very interesting word-spotting performance.

This paper presents a pioneering attempt for developing a recurrent neural net based connectionist system for unconstrained Bengali offline handwriting recognition. The major challenge in configuring such a classification system for a complex script like Bengali is to effectively define the character classes. A novel way of defining character classes is introduced making the recognition problem suitable for using a recurrent model. Indeed, it has to deal with more than nine hundred character classes for which the occurrence probability is very skewed in the language. An off-the-shelf BLSTM-CTC recognizer is used. An open-source dataset is developed for unconstrained Bengali offline handwriting recognition. The dataset contains 2,338 handwritten text lines consisting of about 21,000 word. Experiment shows that with the new definition of character classes the BLSTM-CTC provides an impressive performance for unconstrained Bengali offline handwriting recognition. The character level recognition accuracy is 75.40% without doing any post-processing on the BLSTM-CTC output. Among the 24.60% character level errors, the substitution, deletion and insertion errors are 18.91%, 4.69% and 0.98%, respectively.

In this paper, we propose a novel system for keyword spotting in handwritten documents. Our approach proceeds in two steps: first a generic text line HMM provides a simple and flexible tool to localize the keyword and its character boundaries. In the second step, a SVM based verification system estimates and combines the character probabilities to provide keyword confidence scores which are further combined with the HMM score. The system has been evaluated on a public handwritten document database used for the 2011 ICDAR handwriting recognition competitions and shows that the verification stage improves the performance and outperforms some other state-of-the-art approaches.

This paper presents a novel approach for language identification in handwritten documents. The approach is based on script identification followed by character recognition. BLSTM-CTC based handwriting recognizers are used and the OCR output is fed to a statistical language identifier for detecting the language of the input handwritten document. Documents in two scripts (Latin and Bengali) and four languages (English, French, Bengali and Assamese) are considered for evaluation. Several alternative frameworks have been explored, effects of handwriting recognition and text length on language detection have been studied. It is observed that with some empirical restrictions it is very much possible to achieve more that 80% language detection accuracy and based on the current research practical systems can be designed.

Since 2006 the national library of France (BnF) has developed many mass digitization projects on its collections. The indexation of digital documents on Gallica (the digital library of the BnF) is done through their textual content obtained thanks to service providers that use Optical Character Recognition software (OCR). The modern technologies of OCR achieve good performances on modern documents produced with uniform layout and known fonts. However, for old documents, OCR results are of lower quality. The OCR quality assessment is a real challenge for the BnF. On the one hand, due to the sequential architecture of OCR treatments, the identification of OCR errors sources is intractable. On the other hand, besides the word confidence, no additional quality information is reported in OCR outputs. In this paper, we present a study on OCR performance estimation aiming to control the quality of word transcriptions achieved by OCR. This quality assessment process has to operate without any comparison with ground truthed data. In this respect, our methodology relies on cross alignment of the OCR results with those of a secondary OCR called reference OCR. This secondary OCR provides uncertain but useful information that will be used as uncertain groundtruth. OCR performance is estimated using support vector regression. This predictor uses some global features computed on the cross-alignment results. The experimentations reported show that our estimate describes more faithfully the quality of OCR outputs than average word confidence scores that are computed by OCR. The proposed methodology can be adapted easily to various corpora by tuning the system using a training dataset of documents that have similar properties to those to be treated.

In this paper, we introduce an iterative method for handwritten text line extraction. The proposed method improves the steerable filter approach by introducing an iterative scheme that iterates lines detection for various configurations of the filters, thus making the method auto adaptable with different types of scripts. We tested the method on different handwritten text datasets, used during earlier competitions organized at ICDAR or ICFHR, and using the same evaluation protocol. The tests carried out on the Open-Hart data set for Arabic scripts, as well as three other Latin and Greek scripts, show that state of the art performance are obtained without tuning any parameters.

This paper presents a Document Image Analysis (DIA) system able to extract homogeneous typed and handwritten text regions from complex layout documents of various types. The method is based on two connected component classification stages that successively discriminate text/non text and typed/handwritten shapes, followed by an original block segmentation method based on white rectangles detection. We present the results obtained by the system during the first competition round of the MAURDOR campaign.

In this paper, we describe a novel and simple technique for prediction of OCR results without using any OCR. The technique uses a bag of allographs to characterize textual components. Then a support vector regression (SVR) technique is used to build a predictor based on the bag of allographs. The performance of the system is evaluated on a corpus of historical documents. The proposed technique produces correct prediction of OCR results on training and test documents within the range of standard deviation of 4.18% and 6.54% respectively. The proposed system has been designed as a tool to assist selection of corpora in libraries and specify the typical performance that can be expected on the selection.

Document image binarization is still an active research area as shows the number of binarization techniques proposed since many decades. The binarization of degraded document images is still difficult and encourages the development of new algorithms. For the last decade, discrete conditional random fields have been successfully used for many domains such as automatic language analysis. In this paper, we propose a CRF based framework to explore the combination capabilities of this model by combining discrete outputs from several well known binarization algorithms. The framework uses two 1D CRF models on the horizontal and the vertical directions that are coupled for each pixel by the product of the marginal probabilities computed from the both models. Experiments are made on two datasets from the Document Image Binarization Contest (DIBCO) 2009 and 2011 and show best performances than most of the methods presented at DIBCO 2011.

This paper presents a method to detect table regions in document images by identifying the column and row line-separators and their properties. The method employs a run-length approach to identify the horizontal and vertical lines present in the input image. From each group of intersecting horizontal and vertical lines, a set of 26 low-level features are extracted and an SVM classifier is used to test if it belongs to a table or not. The performance of the method is evaluated on a heterogeneous corpus of French, English and Arabic documents that contain various types of table structures and compared with that of the Tesseract OCR system.

In this paper, we propose a novel system for word spotting and regular expression detection in Handwritten documents. The proposed approach is lexicon-free, i.e., able to spot arbitrary keywords that are not required to be known at the training stage. Furthermore, the proposed system is segmentation-free, i.e., text lines are not required to be segmented into words. The originalities of our approach is twofold. First we propose a new filler model which allows to speed-up the decoding process. Second, we extend the methodology to search for regular expressions. The system has been evaluated on a public handwritten document database used for the 2011 ICDAR handwriting recognition competitions.

Newspapers are documents made of news item and informative articles. They are not meant to be read iteratively: the reader can pick his items in any order he fancies. Ignoring this structural property, most digitized newspaper archives only offer access by issue or at best by page to their content. We have built a digitization workflow that automatically extracts newspaper articles from images, which allows indexing and retrieval of information at the article level. Our back-end system extracts the logical structure of the page to produce the informative units: the articles. Each image is labelled at the pixel level, through a machine learning based method, then the page logical structure is constructed up from there by the detection of structuring entities such as horizontal and vertical separators, titles and text lines. This logical structure is stored in a METS wrapper associated to the ALTO file produced by the system including the OCRed text. Our front-end system provides a web high definition visualisation of images, textual indexing and retrieval facilities, searching and reading at the article level. Articles transcriptions can be collaboratively corrected, which as a consequence allows for better indexing. We are currently testing our system on the archives of the Journal de Rouen, one of France eldest local newspaper. These 250 years of publication amount to 300 000 pages of very variable image quality and layout complexity. Test year 1808 can be consulted at plair.univ-rouen.fr.

In this paper, a novel rejection strategy is proposed to optimize the reliability of an handwritten word recognition system. The proposed approach is based on several steps. First, we combine the outputs of several HMM classifiers using the Dempster-Shafer theory (DST). Then, we take advantage of the expressivity of mass functions (the counter part of probability distributions in DST) to characterize the quality/reliability of the classification. Finally, we use this characterization to decide whether a test word is rejected or not. Experiments carried out on RIMES and IFN/ENIT datasets show that the proposed approach outperforms other state-of-the-art rejection methods.

We introduce quantization feature functions to represent continuous or large range discrete data into the symbolic CRF data representation. We show that doing this convertion in a simple way allows the CRF to automaticaly select discriminative features to achieve best performance. This system is evaluated on a segmentation task of degraded newspapers archives. The results obtained show the ability of the CRF model to deal with numerical features similarly as for symbolic representation thanks to the use of quantization feature functions. The segmentation task is achieved by the definition of a horizontal CRF model dedicated to pixel labelling.

This paper is focused on the optimization of the computational efficiency of a multi-stream word recognition system. The aim of this work is to optimize the multi-stream decoding step in order to reduce the recognition time and the complexity to allow combining a large number of streams. Two different multi-stream decoding strategies are compared based on two-level and HMM-recombination algorithms. Experiments carried out on public handwritten word databases show significant speed gains at decoding while keeping the same performances, in addition to new insights for combining a large number of streams.

The problem of writer identification in a multi-script environment is attempted using a two-dimensional (2D) autoregressive (AR) modeling technique. Each writer is represented by a set of 2D AR model coefficients. A method to estimate AR model coefficients is proposed. This method is applied to an image of text written by a specific writer so that AR coefficients are obtained to characterize the writer. For a given sample, AR coefficients are computed and its L2distance with each of the stored (writer) prototypes identifies the writer for the sample. The method has been tested on datasets of two different scripts, namely RIMES containing 382 French writers and ISI consisting of samples from 40 Bengali writers. Modeling of writing styles using different context patterns at different image resolution has been investigated. Experimental results show that the technique achieves results comparable with that of the previous approaches.

Generally, handwritten word recognition systems use script specific methodologies. In this paper, we present a unified approach for multi-lingual recognition of alphabetic scripts. The proposed system operates independently of the nature of the script using the multi-stream paradigm. The experiments have been carried out on a multi-script database composed of Arabic and Latin handwritten words from the IFN/ENIT and the IRONOFF public databases and show interesting recognition performances with only 1.5% of script confusion and an overall word recognition rate of 84.5% using a multi-script lexicon of 1142 words.

In this paper, we propose a multi-objective optimization method for SVM model selection using the well known NSGA-II algorithm. FA and FR rates are the two criteria used to find the optimal hyperparameters of a set of SVM classifiers. The proposed strategy is applied to a digit/outlier discrimination task embedded in a more global information extraction system that aims at locating and recognizing numerical fields in handwritten incoming mail documents. Experiments conducted on a large database of digits and outliers show clearly that our method compares favorably with the results obtained by a state-of-the- art mono-objective optimization technique using the classical Area Under ROC Curve criterion (AUC).

This paper presents a pioneering study on automatic dating of handwritten manuscripts. Analysis of handwriting style forms the core of the dating method. Initially, it is hypothesized that a manuscript can be dated, to a certain level of accuracy, by looking at the way it is written. The hypothesis is then verified with real samples of known dates. A general framework is proposed for machine dating of handwritten manuscripts. Experiments on a database containing manuscripts of Gustave Flaubert (1821- 1880), the famous French novelist reports about 62% accuracy when manuscripts are dated within a range of five calendar years with respect to their exact year of writing.

We present in this paper a new approach based on multi-stream hidden Markov models (HMM) for the recognition of off-line handwriting. Every word is presented by two HMM models: the first one is learned with features extracted from upper contour, the second with features extracted from lower contour. The combination of these two sources of information is studied using the multi-stream framework. We present experiment results obtained on a database composed of isolated words extracted from incoming mail documents.

This work relates to the implementation of a 2D conditional random field model in the context of document image analysis. Our model makes it possible to take variability into account and to integrate contextual knowledge, while taking benefit from machine learning techniques. Experiments on handwritten drafts of Flaubert show that these models provide interesting solutions.

This paper proposes an adaptive method for separation of foreground and background in low quality color document images. A connected component labelling is initially implemented to capture the spatially connected similar color pixels. Next, dominant background components are determined to divide the entire image into number of grids each representing local uniformity in illumination, background, etc. Finally foreground parts are located using local information around them. Several color images of old historical documents including manuscripts of high importance are used in the experiment. Apart from a qualitative evaluation, results are quantitatively compared with one popular foreground/background separation technique.

In this paper we present a method based on hidden Markov random fields and 2D dynamic programming image decoding, for segmenting pages of complex handwritten manuscripts such as novelist drafts. After a formal description of the theoretical framework and the principles of the decoding method, we describe the implementation of the model and the decoding method. Then we discuss the results obtained with this approach on the drafts of the French novelist Gustave Flaubert.

In this paper we describe the Bovary Project, a manuscripts digitization project of the famous French writer Gustave FLAUBERT first great work. This project has just begun at the end of 2002 and should end in 2006 by providing an online access to an hypertextual edition of "Madame Bovary" drafts set. We develop the global context of this project, the main objectives, the first studies and the considered outlooks for the project's carried out.

This communication deals with the Writer Identificationtask. Our previous work has shown the interest of usingthe graphemes as features for describing the individualproperties of Handwriting. We propose here to exploit thesame feature set but using an information retrievalparadigm to describe and compare the handwritten queryto each sample of handwriting in the database. Using thistechnique the image processing stage is performed onlyonce and before the retrieval process can take place, thusleading to a significant saving in the computation of eachquery response, compared to our initial proposition. Themethod has been tested on two handwritten databases.The first one has been collected from 88 different writersat PSI Lab. while the second one contains 39 writers fromthe original correspondence of Emile Zola, a famousFrench novelist of the last 19th century. We also analyzethe proposed method when using concatenation ofgraphemes (bi and tri-gramme) as features.

In this communication, we propose a method for the automatic extraction of numerical fields in handwritten documents. The approach exploits the known syntactic structure of the numerical field to extract, combined with a set of contextual morphological features to find the best label to each connected component. Applying an HMM based syntactic analyzer on the overall document allows to localize/extract fields of interest. Reported results on the extraction of zip codes, phone numbers and customer codes from handwritten incoming mail documents demonstrate the interest of the proposed approach.

Investigates the automatic reading of unconstrained omni-writer handwritten texts. This paper shows how to endow the reading system with adaptation faculties for each writer's handwriting. The adaptation principles are of major importance for making robust decisions when neither simple lexical nor syntactic rules can be used, e.g. for a free lexicon or for full text recognition. The first part of this paper defines the concept of writer's invariants. In the second part, we explain how the recognition system can be adapted to a particular handwriting by exploiting the graphical context defined by the writer's invariants. This adaptation is guaranteed, thanks to the writer's invariants, by activating interaction links over the whole text between the recognition procedures for word entities and those for letter entities.

In this paper, we present a mathematical model for evaluating codes and primitives in optical character recognition. The model is based on code efficiency, calculated from its average length and its transmitted information. This efficiency is obtained from entropies and conditional entropies which are estimated from probabilities of character recognition depending on the issued codes. This method is used to evaluate two sets of primitives in numeral recognition. Then, we propose a method of constructing binary decision trees for the recognition of handwritten numerals. This method is based on the mathematical model previously stated, which is used to process the transmitted information about the primitives. We demonstrate the performance of our system with experiments using real data.

The paper deals with the global recognition of a small lexicon of words, based on a pseudo segmentation stage introducing anchor points. We avoid the difficult problem of segmentating the word into letters and the complexity involved by such models to build possible letter graphs. We use two structural representations of the word, strokes and graphemes, each of them being analyzed using a Markov model. These simple models are individually optimized by a rigorous choice of the order for fitting the structural properties of the observed data using Akaike information criteria. The conditional probability to have a word model, given the observation sequence, is computed by taking into account the length of the sequence. Results of the study are presented on French cheque images.