Historical documents present many challenges for offline handwriting recognition systems, among them, the segmentation and labeling steps. Carefully annotated text lines are needed to train an HTR system. In some scenarios, transcripts are only available at the paragraph level with no text-line information. In this work, we demonstrate how to train an HTR system with few labeled data. Specifically, we train a deep convolutional recurrent neural network (CRNN) system on only 10% of manually labeled text-line data from a dataset and propose an incremental training procedure that covers the rest of the data. Performance is further increased by augmenting the training set with specially crafted multi scale data. We also propose a model-based normalization scheme which considers the variability in the writing scale at the recognition phase. We apply this approach to the publicly available READ dataset. Our system achieved the second best result during the ICDAR2017 competition [1].

Arabic handwritten documents present specific challenges due to the cursive nature of the writing and the presence of diacritical marks. Moreover, one of the largest labeled database of Arabic handwritten documents, the OpenHart-NIST database includes specific noise, namely guidelines, that has to be addressed. We propose several approaches to process these documents. First a guideline detection approach has been developed, based on K-means, that detects the documents that include guidelines. We then propose a series of preprocessing at text-line level to reduce the noise effects. For text-lines including guidelines, a guideline removal preprocessing is described and existing keystroke restoration approaches are assessed. In addition, we propose a preprocessing that combines noise removal and deskewing by removing line fragments from neighboring text lines, while searching for the principal orientation of the text-line. We provide recognition results, showing the significant improvement brought by the proposed processings.

Handwriting recognition systems usually rely on static dictionaries and language models. Full coverage of these dictionaries is generally not achieved when dealing with unrestricted document corpora due to the presence of Out-Of-Vocabulary words. In a previous work, dynamic dictionaries were built from Web resources and successfully applied to isolated word recognition. In the present work we extend this approach to text-line recognition. Line segmentation into words is needed to exploit dynamic dictionaries and it is performed using BLSTM classifiers to align filler models and word sequence outputs. Words are then classified based on the confidence score into anchor and non-anchor words (AWs and NAWs). AWs are equated to the BLSTM outputs and used as such. Dynamic dictionaries are built for NAWs by exploiting Web resources for their character sequence and for neighboring AWs. Text-lines are decoded again using dynamic dictionaries and re-estimated language model. We conduct experiments on the publicly available RIMES database and show that the introduction of the dynamic dictionary is beneficial. Equally important, we show that the gain increases as the proportion of OOVs increases.

Handwriting recognition systems rely on predefined dictionaries obtained from training data. Small and static dictionaries are usually exploited to obtain high in-vocabulary (IV) accuracy at the expense of coverage. Thus the recognition of out-of-vocabulary (OOV) words cannot be handled efficiently. To improve OOV recognition while keeping IV dictionaries small, we introduce a multi-step approach that exploits Web resources. After an initial IV-OOV sequence classification, external resources are used to create OOV sequence-adapted dynamic dictionaries. A final Viterbi-based decoding is performed over the dynamic dictionary to determine the most probable word for the OOV sequence. We validate our approach with experiments conducted on RIMES, a publicly available database. Results show that improvements are obtained compared to standard handwriting recognition, performed with a static dictionary. Both domain adapted and generic dynamic dictionaries are studied and we show that domain adaptation is beneficial.

This study aims at building an efficient word recognition system resulting from the combination of three handwriting recognizers. The main component of this combined system is an HMM-based recognizer which considers dynamic and contextual information for a better modeling of writing units. For modeling the contextual units, a state-tying process based on decision tree clustering is introduced. Decision trees are built according to a set of expert-based questions on how characters are written. Questions are divided into global questions, yielding larger clusters, and precise questions, yielding smaller ones. Such clustering enables us to reduce the total number of models and Gaussians densities by 10. We then apply this modeling to the recognition of handwritten words. Experiments are conducted on three publicly available databases based on Latin or Arabic languages: Rimes, IAM, and OpenHart. The results obtained show that contextual information embedded with dynamic modeling significantly improves recognition.

This article describes the isolated word recognizer presented by the authors to the ICDAR 2009 French handwriting recognition competition. The system is a combination of three isolated word recognizers based on different features and models. A novel n-best combination method is proposed and compared to standard combination methods. New results on the ICDAR 2009 test database are reported.

The problem addressed in this study is the offline recognition of handwritten Arabic city names. The names are assumed to belong to a fixed lexicon of about 1,000 entries. A state-of-the-art classical right-left hidden Markov model (HMM)-based recognizer (reference system) using the sliding window approach is developed. The feature set includes both baseline-independent and baseline-dependent features. The analysis of the errors made by the recognizer shows that the inclination, overlap, and shifted positions of diacritical marks are major sources of errors. In this paper, we propose coping with these problems. Our approach relies on the combination of three homogeneous HMM-based classifiers. All classifiers have the same topology as the reference system and differ only in the orientation of the sliding window. We compare three combination schemes of these classifiers at the decision level. Our reported results on the benchmark IFN/ENIT database of Arabic Tunisian city names give a recognition rate higher than 90 percent accuracy and demonstrate the superiority of the neural network-based combination. Our results also show that the combination of classifiers performs better than a single classifier dealing with slant-corrected images and that the approach is robust for a wide range of orientation angles.

The BioSecure Network of Excellence has collected a large multi- biometric publicly available database and organized the BioSecure Multimodal Evaluation Campaigns (BMEC) in 20072. This paper reports on the Talking Faces campaign. Open source reference systems were made available to participants and four laboratories submitted executable code to the organizer who performed tests on sequestered data. Several deliberate impostures were tested. It is demonstrated that forgeries are a real threat for such systems. A technological race is ongoing between deliberate impostors and system developers.

We present a new method that permits arbitrary region description in video sequence using a reduced number of bits allowing promising results in video compression. A robust unsupervised 3D (2D + t) segmentation is used to detect arbitrary regions for motion compensated video coding. The video sequence is first modeled using a Gaussian mixture model where each pixel, defined by its spatiotemporal position and its color vector is supposed to be generated by one of the mixture components. This permits to segment the video sequence into objects each one modeled by one Gaussian distribution. Grouping the mixtures in a binary tree defines a hierarchical representation of the video objects and a gradual segmentation. This segmentation is then used for region description in a motion compensated video coder. This provides a large improvement in motion bits budget. When compared to a H.264 video coder, promising results were obtained.

In this paper we present a two-stage system for the off-line recognition of cursive Arabic handwritten words. The proposed method is analytic without segmentation, and is able to cope with handwriting inclination and with shifted positions of diacritical marks. First, the recognition stage relies on 3 classifiers based on hidden Markov modelling (HMM). The second stage depends on the combination of these classifiers. The feature vectors used for recognition are related to pixel density distribution and to local pixel configurations. These vectors are extracted on word binary images by using a sliding window approach with different angles. We have experimented different combination schemes. The neural network-based combined system yields best performance on the IFN- ENIT benchmark data base of handwritten names of Tunisian villages/towns.

In this paper, we describe a 1D HMM offline handwriting recognition system employing an analytical approach. The system is supported by a set of robust language independent features extracted on binary images. Parameters such as lower and upper baselines are used to derive a subset of baseline dependent features. Thus, word variability due to lower and upper parts of words is better taken into account. In addition, the proposed system learns character models without character pre-segmentation. Experiments that have been conducted on the benchmark IFN/ENIT database of Tunisian handwritten country/village names, show the advantage of the proposed approach and of the baseline-dependant features.

State of the art Speech Recognition systems use statistical language modeling and in particular N-gram models to represent the language structure. The Arabic language has a rich morphology, which motivates the introduction of morphological constraints in the language model. Class-based N-gram models have shown satisfactory results, especially for language model adaptation and training from reduced datasets. They were also proven quite effective in their use of memory space. In this paper, we investigate a new morphological classbased language model. Morphological rules are used to derive the different words in a class from their stem. As morphological analyzer, a rule-based stemming method is proposed for the Arabic language. The language model has been evaluated on a database composed of articles from Lebanese newspaper Al-Nahar for the years 1998 and 1999. In addition, a linear interpolation between the N-gram model and the morphological model is also evaluated. Preliminary experiments detailed in this paper show satisfactory results.

Multilingual speech recognition pushes to us study the acoustic modeling of target language units using one or more source languages' units. This paper presents a study of manual and data driven association of two possible target units with source language's phonemes. The target units studied are words and phonemes. Algorithms for data-driven association are described. While phoneme-to-phoneme association is more practical, words' transcription provides better results. It has been shown that more precise and rich source models are more suitable to determine those association. Experiments are conducted with French as source language and Arabic as target language.

This paper introduces a unified framework for online adaptation of hidden Markov models (HMM) parameters to real-life conditions. Hence, it aims at improving the robustness of speech recognition systems. In addition, it describes some techniques developed to control the convergence of adaptation in unsupervised modes. Classically, two approaches have been used to adapt HMM parameters to new conditions, that is, Bayesian adaptation and spectral transformation-generally using linear regression. This paper lays out a unifying framework where both Bayesian adaptation and spectral transformation adaptation are seen as particular cases. In this sense, the framework attributes one transformation to each Gaussian distribution and partitions the latter automatically with respect to the adaptation data. Thus, the transformations of each class would share the same parameter vector. Consequently, the global transformation gets a data-driven freedom degree. The parameters of the global transformation are determined according to the maximum a posteriori (MAP) criterion using the original HMM a priori distributions. The general adaptation algorithm has been implemented within the CNET speech recognition system and the whole system evaluated on several field-telephone databases. The new adaptation method provides us with a systematic convergence in an online unsupervised mode of the speech recognition system toward a system enrolled with field data in a supervised mode.

Classical adaptation approaches are generally used for speaker or environment adaptation of speech recognition systems. In this paper, we use such techniques for the incremental training of client models in a speaker verification system. The initial model is trained on a very limited amount of data and then progressively updated with access data, using a segmental-EM procedure. In supervised mode (i.e. when access utterances are certified), the incremental approach yields equivalent performance to the batch one. We also investigate on the impact of various scenarios of impostor attacks during the incremental enrollment phase. All results are obtained with the Picassoft platform-the state-of-the-art speaker verification system developed in the PICASSO project.

Classical adaptation approaches generally allow a reliably trained model to match a particular condition. In this paper, we define an incremental version of the segmental-EM algorithm. This method permits one to incrementally enrich a model first trained with a limited amount of data. Resource memory constraints allow only the initial data statistics to be stored. The proposed method uses these statistics by fixing, within the segmental-EM algorithm applied on both initial and new data, the initial optimal paths in the model for the initial data. We proved theoretically that this is equivalent to the segmental MAP adaptation with specific choice of priors. Experiments on two speaker dependent telephone databases, showed that the approach permitted one to incrementally integrate new conditions of use. The performance was slightly less than that obtained with classical training over the whole data. As expected with the MAP interpretation of the algorithm, initial data characteristics influence largely the model evolution.

The problem of labelling speaker turns by automatically segmenting and clustering a continuous audio stream is addressed. A new clustering scheme is presented and evaluated using a clustering efficiency score which treats both agglomerative and divisive clustering strategies equally. Results show an efficiency of 70% can be obtained on both manually and automatically derived segments on the 1996 Hub4 development data. For the task of identifying potentially unknown anchor speakers within broadcast news shows, the frame classification error rate is very important. To reflect this, a frame-based cluster efficiency is defined and the results show a 90% frame-based efficiency can be achieved. Finally a frame-based comparison between the manually and automatically derived segment/cluster sets shows that approximately one third of the errors are introduced during segmentation and two-thirds during clustering.

In order to create a useful, gracefully functioning system for travel arrangements, we have first observed the task as it is accomplished by a human.We then imitated the human while making the user believe he was dialoguing with an automatic system.As we gradually built our system, we devised ways to assess progress and to detect errors.The following described the manner in which the Carnegie Mellon Communicator was built, data collected, and assessment begun using these criteria.

An acoustic mismatch between a given utterance and a model degrades the performance of the speech recognition process. We choose to model speech by hidden Markov models (HMMs) in the cepstrum domain and the mismatch by a parametric function. In order to reduce the mismatch, one has to estimate the parameters of this function. We present a frame synchronous estimation of these parameters. We show that the parameters can be computed recursively. Thanks to such methods, parameters variations can be tracked. We give general equations and study the particular case of an affine transform. Finally, we report recognition experiments carried out over both PSTN and cellular telephone network to show the efficiency of the method in a real context.

This paper deals with automatic speech recognition robustness for noisy wireless communications. We propose several solutions to improve speech recognition over the cellular network. Two architectures are derived for the recognizer. They are based on hidden Markov models (HMMs) adapted to adverse noise conditions. Then two more specific solutions aiming to alleviate GSM cellular network defects (holes and impulsive noise) are developed. Holes are detected and rejected. Impulsive noises are modeled using mixture density HMMs and a maximum likelihood criterion. These solutions allow a noticeable recognition error reduction. The last one seems to be promising.

For speech recognition based on hidden Markov modeling, parameter-tying, which consists of constraining some of the parameters of the model to share the same value, has emerged as a standard practice. An original algorithm is proposed that makes it possible to jointly estimate both the shared model parameters and the tying characteristics, using the maximum likelihood criterion. The proposed algorithm is based on a previously introduced extension of the classic expectation-maximization (EM) framework. The convergence properties of this class of algorithms are analyzed in detail. The method is evaluated on an isolated word recognition task using hidden Markov models (HMMs) with Gaussian observation densities and tying at the state level. Finally, the extension of this method to the case of mixture observation densities with tying at the mixture component level is discussed.

In this work, environment adaptation is studied in order to transform PSN speaker independent isolated words HMM to the GSM environment. Linear multiple regression (LMR) transformations associated with groups of HMM densities are used to adapt the densities. Both mean vectors and covariance matrices of the densities are adapted. It has been shown that a small amount of GSM data are sufficient to transform the PSN HMM in order to match the GSM environment and to achieve a performance equivalent to those of an HMM trained with a large amount of GSM data. The number of groups of Gaussian densities seems to have a small influence on the results. However, the minimum number of groups depends on the vocabulary size. Finally, this technique is compared to the Bayesian adaptation and the results show that similar performance can be obtained with both methods.

We propose using Hidden Markov Models (HMMs) associated with the cepstrum coefficients as a speech signal model in order to perform equalization or noise removal. The MUlti-path Stochastic Equalization (MUSE) framework allows one to process data at the frame level: it is an on-line adaptation of the model. More precisely, we apply this technique to perform bias removal in the cepstral domain in order to increase the robustness of automatic speech recognizers. Recognition experiments on two databases recorded on both PSN and GSM networks show the efficiency of the proposed method.

The paper compares, on a database recorded in a car, a number of signal analysis and speech enhancement techniques as well as some approaches to adapt speech recognition systems. It is shown that a new nonlinear spectral subtraction associated with Mel frequency cepstral coefficients (MFCC) is an adequate compromise for low-cost integration. The Lombard effect is analyzed and simulated. Such a simulation is used to derive realistic training utterances from noise-free utterances. Adapting a continuous-density hidden Markov model (CDHMM) to these artificially generated training samples yields a very high performance with respect to that achieved within the ESPRIT adverse environment recognition of speech (ARS) project, i.e., an average of 1% error for all driving conditions. Finally, the paper shows, both theoretically and experimentally, that whatever the noise estimation technique is, it is better to add this noise estimate to the reference clean models than to subtract it from the noisy data.>

Speech enhancement through Kalman filtering is compared to environment adaptation using spectral transformations. Important results on general characteristics of speech in noise and a possible correction to the Lombard effect are reported. It is shown that traditional speech enhancement offers little improvement to recognition. Best results are obtained with transformations applied to the reference models.>