With the advent of transformer based models, the use of fully automated ASR-pipelines in a real-world context has come close to a reality. However, for faithful transcription of conversational speech, there remain challenges both in terms of the content predicted by these models (hallucinations, unintended normalizations of disfluencies and transcriptions of background noises) and in terms of alignment accuracy. In this paper we present a hybrid ASR-pipeline which augments transformer models with other algorithms in order to transcribe conversational data. Through experiments on two French datasets, we show that: 1) VAD preprocessing can significantly improve transcription quality as well as word level temporal alignment, 2) prompting can reduce unintended normalizations of disfluencies, 3) heuristic-based detection of untranscribed sounds can further improve alignment quality. We conclude that our hybrid pipeline is an efficient way to improve and augment existing ASR-models.

We present an attention-based model for end-to-end handwriting recognition. Our system does not require any segmentation of the input paragraph. The model is inspired by the differentiable attention models presented recently for speech recognition, image captioning or translation. The main difference is the implementation of covert and overt attention with a multi-dimensional LSTM network. Our principal contribution towards handwriting recognition lies in the automatic transcription without a prior segmentation into lines, which was critical in previous approaches. Moreover, the system is able to learn the reading order, enabling it to handle bidirectional scripts such as Arabic. We carried out experiments on the well-known IAM Database and report encouraging results which bring hope to perform full paragraph transcription in the near future.

This paper presents an end-to-end system for automatic local climate zones classification of various types of urban environment. For that we perform fusion of multispectral images from Landsat-8 and Sentinel-2 satellites with site description extracted from OpenStreetMap layers. The proposed classification approach is based on a multi-level ensemble scheme that combines Convolutional Neural Networks, Random Forests and Gradient Boosting Machines.

Text line detection and localisation is a crucial step for full page document analysis, but still suffers from heterogeneity of real life documents. In this paper, we present a novel approach for text line localisation based on Convolutional Neural Networks and Multidimensional Long Short-Term Memory cells as a regressor in order to predict the coordinates of the text line bounding boxes directly from the pixel values. Targeting typically large images in document image analysis, we propose a new model using weight sharing over local blocks. We compare two strategies: directly predicting the four coordinates or predicting lower-left and upper-right points separately followed by matching. We evaluate our work on the highly unconstrained Maurdor dataset and show that our method outperforms both other machine learning and image processing methods.

This paper describes a simple approach to generate an efficient hybrid word/Part-of-Arabic-Word (PAW) Language Model (LM). More precisely, less frequent words in a full word vocabulary are decomposed into PAWs. The resulted PAWs are incorporated with the most frequent words to generate a hybrid word-PAW vocabulary which is used to estimate a hybrid flat n-gram statistical language model. For comparison purposes, language models with full PAW decomposition of the word vocabulary are generated. To assess the quality of the three types of LMs (i.e. full word, hybrid word/PAW and full PAW LMs), evaluation experiments are conducted under three different tasks using two benchmarking databases, namely Maurdor and Khatt. Results in terms of word error rate show that systems using the full PAW and the proposed hybrid LMs perform equally the same, and both of them, systematically, outperform systems using word LMs. However, systems using hybrid LMs require less memory than those using full PAW LMs.

The dropout technique is a data-driven regularization method for neural networks. It consists in randomly setting some activations from a given hidden layer to zero during training. Repeating the procedure for each training example, it is equivalent to sample a network from an exponential number of architectures that share weights. The goal of dropout is to prevent feature detectors to rely on each other. Dropout has successfully been applied to Deep MLPs and to convolutional neural networks, for various tasks of Speech Recognition and Computer Vision. We recently proposed a way to use dropout in MDLSTM-RNNs for handwritten word and line recognition. In this paper, we show that further improvement can be achieved by implementing dropout differently, more specifically by applying it at better positions relative to the LSTM units.

In recent years, Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs) trained with the Connectionist Temporal Classification (CTC) objective won many international handwriting recognition evaluations. The CTC algorithm is based on a forward-backward procedure, avoiding the need of a segmentation of the input before training. The network outputs are characters labels, and a special non-character label. On the other hand, in the hybrid Neural Network / Hidden Markov Models (NN/HMM) framework, networks are trained with framewise criteria to predict state labels. In this paper, we show that CTC training is close to forward-backward training of NN/HMMs, and can be extended to more standard HMM topologies. We apply this method to Multi-Layer Perceptrons (MLPs), and investigate the properties of CTC, namely the modeling of character by single labels and the role of the special label.

We present initial results on the use of Multi-Dimensional Long-Short Term Memory Recurrent Neural Networks (MDLSTM-RNN) in recognizing lines of handwritten Chinese text without explicit segmentation of the characters. In fact, most of Chinese text recognizers in the literature perform a pre-segmentation of text image into characters. This can be a drawback, as explicit segmentation is an extra step before recognizing the text, and the errors made at this stage have direct impact on the performance of the whole system. MDLSTM-RNN is now a state-of-the-art technology that provides the best performance on languages with Latin and Arabic characters, hence we propose to apply RNN on Chinese text recognition. Our results on the data from the Task 4 in ICDAR 2013 competition for handwritten Chinese recognition are comparable in performance with the best reported systems.

The detection of text lines, as a first processing step, is critical in all text recognition systems. State-of-the-art methods to locate lines of text are based on handcrafted heuristics fine-tuned by the image processing community's experience. They succeed under certain constraints; for instance the background has to be roughly uniform. We propose to use more “agnostic” Machine Learning-based approaches to address text line location. The main motivation is to be able to process either damaged documents, or flows of documents with a high variety of layouts and other characteristics. A new method is presented in this work, inspired by the latest generation of optical models used for text recognition, namely Recurrent Neural Networks. As these models are sequential, a column of text lines in our application plays here the same role as a line of characters in more traditional text recognition settings. A key advantage of the proposed method over other data-driven approaches is that compiling a training dataset does not require labeling line boundaries: only the number of lines are required for each paragraph. Experimental results show that our approach gives similar or better results than traditional handcrafted approaches, with little engineering efforts and less hyper-parameter tuning.

This paper describes the Arabic handwriting recognition systems proposed by A2iA to the NIST OpenHaRT2013 evaluation. These systems were based on an optical model using Long Short-Term Memory (LSTM) recurrent neural networks, trained to recognize the different forms of the Arabic characters directly from the image, without explicit feature extraction nor segmentation.Large vocabulary selection techniques and n-gram language modeling were used to provide a full paragraph recognition, without explicit word segmentation. Several recognition systems were also combined with the ROVER combination algorithm. The best system exceeded 80% of recognition rate.

Recurrent Neural Networks (RNN) have recently achieved the best performance in off-line Handwriting Text Recognition. At the same time, learning RNN by gradient descent leads to slow convergence, and training times are particularly long when the training database consists of full lines of text. In this paper, we propose an easy way to accelerate stochastic gradient descent in this set-up, and in the general context of learning to recognize sequences. The principle is called Curriculum Learning, or shaping. The idea is to first learn to recognize short sequences before training on all available training sequences. Experiments on three different handwritten text databases (Rimes, IAM, OpenHaRT) show that a simple implementation of this strategy can significantly speed up the training of RNN for Text Recognition, and even significantly improve performance in some cases.

This paper describes the system submitted by A2iA to the second Maurdor evaluation for multi-lingual text recognition. A system based on recurrent neural networks and weighted finite state transducers was used both for printed and handwritten recognition, in French, English and Arabic. To cope with the difficulty of the documents, multiple text line segmentations were considered. An automatic procedure was used to prepare annotated text lines needed for the training of the neural network. Language models were used to decode sequences of characters or words for French and English and also sequences of part-of-arabic words (PAWs) in case of Arabic. This system scored first at the second Maurdor evaluation for both printed and handwritten text recognition in French, English and Arabic.

Recurrent neural networks (RNNs) with Long Short-Term memory cells currently hold the best known results in unconstrained handwriting recognition. We show that their performance can be greatly improved using dropout - a recently proposed regularization method for deep architectures. While previous works showed that dropout gave superior performance in the context of convolutional networks, it had never been applied to RNNs. In our approach, dropout is carefully used in the network so that it does not affect the recurrent connections, hence the power of RNNs in modeling sequences is preserved. Extensive experiments on a broad range of handwritten databases confirm the effectiveness of dropout on deep architectures even when the network mainly consists of recurrent and shared connections.

This paper describes a complete system for hand-written information extraction in historical documents. The system was evaluated in real conditions and at a large scale (8 millions of snippets) on the tables of the 1930 US Census. The location of the table position was based on a registration algorithm using printed word anchors. The rows and columns were extracted for nine different fields. For each field, a recognizer based either on convolutional neural networks for small lexicon fields or recurrent neural networks for large lexicon fields were trained. This system yields very high results for data extraction, allowing to achieve more than 70% of automation rate at a error rate similar to human keyers for a complete identity field.

We compare the recently proposed Discriminative Restricted Boltzmann Machine (DRBM) to the classical Support Vector Machine (SVM) on a challenging classification task consisting in identifying weapon classes from audio signals. The three weapon classes considered in this work (mortar, rocket, and rocket‐propelled grenade), are difficult to reliably classify with standard techniques because they tend to have similar acoustic signatures. In addition, specificities of the data available in this study make it challenging to rigorously compare classifiers, and we address methodological issues arising from this situation. Experiments show good classification accuracy that could make these techniques suitable for fielding on autonomous devices. DRBMs appear to yield better accuracy than SVMs, and are less sensitive to the choice of signal preprocessing and model hyperparameters. This last property is especially appealing in such a task where the lack of data makes model validation difficult.

In document image classification, some classes of documents can be easily identified using pixel-level features, whereas some distinctions can only be made using semantics, which usually involves a full automatic text transcription. To be as much efficient as possible, the classification system should be able to avoid extracting high-level and time consuming features when they are not necessary to classify with confidence. We introduce here this issue of sample-dependent feature selection, which has not been addressed before as far as we know. We propose a method to tackle this problem, that can be generalized to any classifier that provides a confidence score along with its prediction. Empirical results using AdaBoost on three mail classification problems show that our approach allows to significantly improve classification efficiency (up to 40% CPU time off) without significant loss of accuracy in comparison to the baseline.

Simulations of cortical computation have often focused on networks built from simplified neuron models similar to rate models hypothesized for V1 simple cells. However, physiological research has revealed that even V1 simple cells have surprising complexity. Our computational simulations explore the effect of this complexity on the visual system's ability to solve simple tasks, such as the categorization of shapes and digits, after learning from a limited number of examples. We use recently proposed high-throughput methodology to explore what axes of modeling complexity are useful in these categorization tasks. We find that complex cell rate models learn to categorize objects better than simple cell models, and without incurring extra computational expense. We find that the squaring of linear filter responses leads to better performance. We find that several other components of physiologically derived models do not yield better performance.

In this paper, we consider the task of automatic handwritten mail classification and we investigate the relation between the transcription rate and the classification rate. Several configurations of a multi-word handwriting recognizer using different language models are tested and their word recognition rates on the documents to be classified are reported. For the document classification task, we have investigated three different classifiers (KNN, SVM, AdaBoost). All the experiments were conducted on the public database Rimes.

We present a comparative study of several SVM speaker verification (SV) systems based on sequence kernels: the GMM-supervectors kernel, the fisher kernel, the Generalized linear discriminant sequence (GLDS) kernel, our feature space normalized sequence (FSNS) kernel and a ldquonovelrdquo sequence kernel in SV, the correlation kernel. We also compare these SVM systems to the conventional generative UBM-GMM. We carry out experiments on the NIST'2005 SRE evaluation set. The results show that the FSNS system yields comparable performances to UBM-GMM and significantly outperforms GLDS. They also show that the GMM-supervectors system outperforms all the others. Finally, they show that the best performances are achieved by fusing the FSNS and the GMM-supervectors systems.

Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).

The generalized linear discriminant sequence (GLDS) kernel has been shown to provide very good performance and efficiency at the NIST Speaker Recognition Evaluations (SRE) in the last few years. This kernel is based on an explicit map of polynomial expansions of input frames which, because of practical limitations, have to be of a degree less or equal to three. In this paper, we consider an extension of the GLDS kernel to allow not only any polynomial degree but also any embedding, including infinite-dimensional ones associated with Mercer kernels (such as Gaussian kernels). It turns out that the resulting kernels belong to the family of posterior covariance kernels. However, their exact ldquokernelizedrdquo form involves the computation of the Gram matrix on background data, and may be intractable when the background corpus is very large (which is the case in speaker verification). To overcome this problem, we use a low-rank approximation of the Gram matrix to provide an approximate but tractable form of these kernels. We then present comparative experiments on NIST SRE 2005. The results show that our sequence kernel outperforms the GLDS one, and gives similar (individual) performances to the traditional universal background model-Gaussiam mixture model (UBM-GMM) system. As expected, the fusion of both improves the scores.

In speaker recognition, several recent studies attempt to integrate prior knowledge in order to make better distinction between speakers. This paper studies the relative speaker discriminative power of speech transient and steady zones. An automatic segmentation is used to localize these two types of zones. Experiments are carried out with the KIST 2003 speaker evaluation database. They show that transient frames, in the neighborhood of segment boundaries, are more speaker discriminative than the middle frames of long segments which correspond to the steady parts of phones. In addition, this study leads to a higher efficiency system (than the baseline one) by processing roughly three times less data during the scoring stage.

In all the efficient speaker recognition systems, the decision score is based on the average of the likelihood ratio computed on each frame of the sentence. Except for the non speech frames which are rejected, each one has the same weight in this summation. This paper deals with the study of the speaker relevance of each frame. An automatic segmentation provides quasi stationary segments of variable length; a weight is allocated to each frame in function of its segment position and a weighted mean of the likelihood ratio is then computed. Experiments are performed with NIST 2003 speaker evaluation database. They show that the frames near segment frontiers, that is to say the transient ones, are more speaker relevant than the middle frames of long segments which correspond to the steady parts of the phones.