Abstract Deep learning approaches have shown high performance for layout analysis of historical documents, provided that enough labeled data is available. This is not an issue for generic tasks such as image binarization, text graphics separation, or text line and text block detection but can become an impediment for more specialized tasks specific to one or a few books only. This paper addresses layout analysis of medieval books with rich and complex layouts, for which no labeled data is initially available. The proposed strategy consists of training an initial model with artificial data created to reflect the rules a deep neural network should learn. Then, the model is iteratively fine-tuned by mixing the artificial data with real data obtained by previous predictions, post-processed, and manually selected by an expert user. Such a strategy needs less human effort than manual ground truthing. The approach is qualitatively and quantitatively assessed and shows that the system converges to an accurate model that finally produces approximate ground truth stable and good enough to train a final model to solve the targeted task with high accuracy.

Abstract Using neural networks for semantic labeling has become a dominant technique for layout analysis of historical document images. However, to train or fine-tune appropriate models, large labeled datasets are needed. This paper addresses the case when only limited labeled data are available and promotes a novel approach using so-called controlled data to pre-train the networks. Two different strategies are proposed: The first addresses the real labeling task by using artificial data; the second uses real data to pre-train the networks with a pretext task. To assess these strategies, a large set of experiments has been carried out on a text line detection and classification task using different variants of U-Net. The observations, obtained from two different datasets, show that globally the approach reduces the training time while offering similar or better performance. Furthermore, the effect is bigger on lightweight network architectures.

It is widely accepted that deep neural networks are very efficient for detecting objects in images.They reach their limit when detecting multiple instances of long lines in low-resolution images.We present an original methodology for the recognition of vine lines in low-resolution satellite images.The method consists in combining an asymmetrical neural network with a sub-classifier.We first compare a traditional U-Net architecture with an asymmetrical U-Net architecture designed for precision agriculture.We then highlight the significant improvement in vine line detection when a Random Forest is added after the customized U-Net.This methodology addresses the complex task of dissociating vine lines from other agricultural objects.As a result, our experiments improve the precision from 0.83 to 0.94 over our optimized neural network.

In this work, we introduce a new architectural component to Neural Network (NN), i.e., trainable and spectrally initializable matrix transformations on feature maps. While previous literature has already demonstrated the possibility of adding static spectral transformations as feature processors, our focus is on more general trainable transforms. We study the transforms in various architectural configurations on four datasets of different nature: from medical (ColorectalHist, HAM10000) and natural (Flowers) images to historical documents (CB55). With rigorous experiments that control for the number of parameters and randomness, we show that networks utilizing the introduced matrix transformations outperform vanilla neural networks. The observed accuracy increases appreciably across all datasets. In addition, we show that the benefit of spectral initialization leads to significantly faster convergence, as opposed to randomly initialized matrix transformations. The transformations are implemented as auto-differentiable PyTorch modules that can be incorporated into any neural network architecture. The entire code base is open-source.

Text line localization in document images is still considered an open research task. The state-of-the-art methods in this regard that are only based on the classical image analysis techniques mostly have unsatisfactory performances especially when the document images i) contain significant degradations and different noise types and scanning defects, and ii) have touching and/or multi-skewed text lines or overlapping words/characters and non-uniform inter-line space. Moreover, localizing text in ancient handwritten Arabic document images is even more complex due to the morphological particularities related to the Arabic script. Thus, in this paper, we propose a hybrid method combining a deep network with classical document image analysis techniques for text line localization in ancient handwritten Arabic document images. The proposed method is firstly based on using the U-Net architecture to extract the main area covering the text core. Then, a modified RLSA combined with topological structural analysis are applied to localize whole text lines (including the ascender and descender components). To analyze the performance of the proposed method, a set of experiments has been conducted on many recent public and private datasets, and a thorough experimental evaluation has been carried out.

This paper introduces a new way for text-line extraction by integrating deep-learning based pre-classification and state-of-the-art segmentation methods. Text-line extraction in complex handwritten documents poses a significant challenge, even to the most modern computer vision algorithms. Historical manuscripts are a particularly hard class of documents as they present several forms of noise, such as degradation, bleed-through, interlinear glosses, and elaborated scripts. In this work, we propose a novel method which uses semantic segmentation at pixel level as intermediate task, followed by a text-line extraction step. We measured the performance of our method on a recent dataset of challenging medieval manuscripts and surpassed state-of-the-art results by reducing the error by 80.7%. Furthermore, we demonstrate the effectiveness of our approach on various other datasets written in different scripts. Hence, our contribution is two-fold. First, we demonstrate that semantic pixel segmentation can be used as strong denoising pre-processing step before performing text line extraction. Second, we introduce a novel, simple and robust algorithm that leverages the high-quality semantic segmentation to achieve a text-line extraction performance of 99.42% line IU on a challenging dataset.

On most document image transcription, indexing and retrieval systems, text line segmentation remains one of the most important preliminary task. Hence, the research community working in document image analysis is particularly interested in providing reliable text line segmentation methods. Recently, an increasing interest in using deep learning-based methods has been noted for solving various sub-fields and tasks related to the issues surrounding document image analysis. Thanks to the computer hardware and software evolution, several methods based on using deep architectures continue to outperform the pattern recognition issues and particularly those related to historical document image analysis. Thus, in this paper we present a novel deep learning-based method for text line segmentation of historical documents. The proposed method is based on using an adaptive U-Net architecture. Qualitative and numerical experiments are given using a large number of historical document images collected from the Tunisian national archives and different recent benchmarking datasets provided in the context of ICDAR and ICFHR competitions. Moreover, the results achieved are compared with those obtained using the state-of-the-art methods.

In recent years, different approaches for handwriting recognition that are based on graph representations have been proposed (e.g. graph-based keyword spotting or signature verification). This trend is mostly due to the availability of novel fast graph matching algorithms, as well as the inherent flexibility and expressivity of graph data structures when compared to vectorial representations. That is, graphs are able to directly adapt their size and structure to the size and complexity of the respective handwritten entities. However, the vast majority of the proposed approaches match the graphs from a global perspective only. In the present paper, we propose to match the underlying graphs from different local perspectives and combine the resulting assignments by means of Dynamic Time Warping. Moreover, we show that the proposed approach can be readily combined with global matchings. In an experimental evaluation, we employ the novel method in a signature verification scenario on two widely used benchmark datasets. On both datasets, we empirically confirm that the proposed approach outperforms state-of-the-art methods with respect to both accuracy and runtime.

Automatic analysis of scanned historical documents comprises a wide range of image analysis tasks, which are often challenging for machine learning due to a lack of human-annotated learning samples. With the advent of deep neural networks, a promising way to cope with the lack of training data is to pre-train models on images from a different domain and then fine-tune them on historical documents. In the current research, a typical example of such cross-domain transfer learning is the use of neural networks that have been pre-trained on the ImageNet database for object recognition. It remains a mostly open question whether or not this pre-training helps to analyse historical documents, which have fundamentally different image properties when compared with ImageNet. In this paper, we present a comprehensive empirical survey on the effect of ImageNet pre-training for diverse historical document analysis tasks, including character recognition, style classification, manuscript dating, semantic segmentation, and content-based retrieval. While we obtain mixed results for semantic segmentation at pixel-level, we observe a clear trend across different network architectures that ImageNet pre-training has a positive effect on classification as well as content-based retrieval.

In this paper, we present DIVAnnotation, an ancient document annotation tool which is freely available as open source. This software is easily modular thanks to the splitting of the different annotation steps through the use of a tabbed graphical user interface. State-of-the-art document image analysis methods are included through web services, thus allowing users to generate automatically annotations and correct them manually when needed. The annotations are stored into a highly structured TEI file which makes data access and manipulation simple. A Java library for managing TEI files generated by DIVAnnotation is also provided as open source.

Web Services are being adapted into the workflows of many Document Image Analysis researchers. However, so far, there is no common platform for providing access to algorithms in the community. DIVAServices aims to become this by providing a platform that is open to the whole community to provide their own methods as Web Services. In this paper we present updates and enhancements made to the existing DIVAServices platform. This includes a new computational backend, a revamped execution workflow based on asynchronous communication, and the possibility for methods to specify their outputs. Furthermore, we discuss the importance of an ecosystem for such platforms. We argue that only providing a RESTful API is not enough. Users need tools and services around the framework that support them in adapting the Web Services and we introduce some of the tools that we built around DIVAServices.

We introduce DeepDIVA: an infrastructure designed to enable quick and intuitive setup of reproducible experiments with a large range of useful analysis functionality. Reproducing scientific results can be a frustrating experience, not only in document image analysis but in machine learning in general. Using DeepDIVA a researcher can either reproduce a given experiment or share their own experiments with others. Moreover, the framework offers a large range of functions, such as boilerplate code, keeping track of experiments, hyper-parameter optimization, and visualization of data and results. To demonstrate the effectiveness of this framework, this paper presents case studies in the area of handwritten document analysis where researchers benefit from the integrated functionality. DeepDIVA is implemented in Python and uses the deep learning framework PyTorch. It is completely open source, and accessible as Web Service through DIVAServices.

For handwritten signature verification, signature images are typically represented with fixed-sized feature vectors capturing local and global properties of the handwriting. Graph-based representations offer a promising alternative, as they are flexible in size and model the global structure of the handwriting. However, they are only rarely used for signature verification, which may be due to the high computational complexity involved when matching two graphs. In this paper, we take a closer look at two recently presented structural methods for handwriting analysis, for which efficient matching methods are available: keypoint graphs with approximate graph edit distance and inkball models. Inkball models, in particular, have never been used for signature verification before. We investigate both approaches individually and propose a combined verification system, which demonstrates an excellent performance on the MCYT and GPDS benchmark data sets when compared with the state of the art.

Cross-depiction is the problem of identifying the same object even when it is depicted in a variety of manners.This is a common problem in handwritten historical document image analysis, for instance when the same letter or motif is depicted in several different ways. It is a simple task for humans yet conventional computer vision methods struggle to cope with it. In this paper we address this problem using state-of-the-art deep learning techniques on a dataset of historical watermarks containing images created with different methods of reproduction, such as hand tracing, rubbing, and radiography.To study the robustness of deep learning based approaches to the cross-depiction problem, we measure their performance on two different tasks: classification and similarity rankings. For the former we achieve a classification accuracy of 96 % using deep convolutional neural networks. For the latter we have a false positive rate at 95% recall of 0.11. These results outperform state-of-the-art methods by a significant margin.

This study presents a novel approach for Arabic video text recognition based on recurrent neural networks. In fact, embedded texts in videos represent a rich source of information for indexing and automatically annotating multimedia documents. However, video text recognition is a non‐trivial task due to many challenges like the variability of text patterns and the complexity of backgrounds. In the case of Arabic, the presence of diacritic marks, the cursive nature of the script and the non‐uniform intra/inter word distances, may introduce many additional challenges. The proposed system presents a segmentation‐free method that relies specifically on a multi‐dimensional long short‐term memory coupled with a connectionist temporal classification layer. It is shown that using an efficient pre‐processing step and a compact representation of Arabic character models brings robust performance and yields a low‐error rate than other recently published methods. The authors’ system is trained and evaluated using the public AcTiV‐R dataset under different evaluation protocols. The obtained results are very interesting. They also outperform current state‐of‐the‐art approaches on the public dataset ALIF in terms of recognition rates at both character and line levels.

This paper presents a page segmentation method for handwritten historical document images based on a Convolutional Neural Network (CNN). We consider page segmentation as a pixel labeling problem, i.e., each pixel is classified as one of the predefined classes. Traditional methods in this area rely on hand-crafted features carefully tuned considering prior knowledge. In contrast, we propose to learn features from raw image pixels using a CNN. While many researchers focus on developing deep CNN architectures to solve different problems, we train a simple CNN with only one convolution layer. We show that the simple architecture achieves competitive results against other deep architectures on different public datasets. Experiments also demonstrate the effectiveness and superiority of the proposed method compared to previous methods.

Graphs provide a powerful representation formalism for handwritten signatures, capturing local properties as well as their relations. Yet, although introduced early for signature verification, only a few current systems rely on graph-based representations. A possible reason is the high computational complexity involved for matching two general graphs. In this paper, we introduce a novel structural approach to offline signature verification using an efficient cubic-time approximation of graph edit distance. We put forward several ways of creating, normalizing, and comparing signature graphs built from keypoints and investigate their performance on three benchmark datasets. The experiments demonstrate a promising performance of the proposed structural approach when compared with the state of the art.

In this paper, we present a novel approach for initializing deep neural networks, i.e., by using Principal Component Analysis (PCA) to initialize neural layers. Usually, the initialization of the weights of a deep neural network is done in one of the three following ways: 1) with random values, 2) layer-wise, usually as Deep Belief Network or as auto-encoder, and 3) re-use of layers from another network (transfer learning). Therefore, typically, many training epochs are needed before meaningful weights are learned, or a rather similar dataset is required for seeding a fine-tuning of transfer learning. In this paper, we describe how to turn a PCA into an auto-encoder, by generating an encoder layer of the PCA parameters and furthermore adding a decoding layer. We analyze the initialization technique on real documents. First, we show that a PCA-based initialization is quick and leads to a very stable initialization. Furthermore, for the task of layout analysis we investigate the effectiveness of PCA-based initialization and show that it outperforms state-of-the-art random weight initialization methods.

This paper reports on the ICDAR2017 Competition on Layout Analysis for Challenging Medieval Manuscripts (HisDoc-Layout-Comp) and provides further details and discussions. In this competition we introduce a new challenging dataset and state-of-the-art benchmark results for pixel-labelling and text line segmentation. The DIVA-HisDB comprises medieval manuscripts with complex layout in contrast to previous datasets, where rectangular text blocks and only a few decorative elements exist. In particular, the images of this competition contain many interlinear and marginal glosses as well as texts in various sizes and decorated letters. This makes the distinction of the four target labels (text, comment, decoration, and background) more difficult. In addition, to reflect the needs of scholars in the humanities, we request multi-labeling of certain regions (decorated text as text and decoration). Furthermore, we measure not just the accuracy, but the Intersection over Union (IU) of pixel sets, which better reflects the real performance. Indeed, in our results we observe that the accuracy appears to be rather high, but the IU reveals, that there is still room for improvement. For the task of line segmentation, the recognition results are rather low (overall error higher than 5%). Noteworthy, a combination of the best layout analysis method with an adapted seam-carving based method achieves better results than the best contestant.

This paper describes the organisation and results of the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text held in the context of the 14th International Conference on Document Analysis and Recognition (ICDAR'2017), during November 10-15, 2017, Kyoto, Japan. This competition has used the freely available Arabic Printed Text Image (APTI) database. A first and second editions took place respectively in ICDAR'2011 and ICDAR'2013. In this edition, we propose four challenges. Six research groups are participating in the competition with thirteen systems. These systems are compared using the font, font-size, font and font-size, and character and word recognition rates. The systems were tested in a blind manner using the first 5000 images of APTI database set 6. A short description of the participating groups, their systems, the experimental setup, and the observed results are presented.

In this paper, we investigate fine-tuned features learned by deep neural networks in the context of layout analysis. Pre-training and fine-tuning are techniques used in deep neural networks to learn representations (features) of input. However, it is not clear if the fine-tuned features are all useful for a following classification task. We investigate this problem using feature selection. Firstly, features are learned by a deep neural network, where stacked autoencoders are used for pre-training and then the whole network is fine-tuned. Then, a feature selection method is used to select relevant features for classification. We observe that despite fine-tuning, a significant number of the features are still redundant or irrelevant for layout classification. Furthermore, features from the top layer of the stacked autoencoders are generally more relevant for classification than those from lower layers.

This paper describes the multi-resolution Arabic Text detection and recognition in Video Competition-AcTiVComp held in the context of the 14thInternational Conference on Document Analysis and Recognition (ICDAR' 2017), during November 9-15, 2017, in Kyoto, Japan. The main objective of this competition is to evaluate the performance of participants' algorithms for automatically detecting and recognizing Arabic texts in video frames using the freely available Arabic-Text-in-Video (AcTiV) dataset. A first edition was held in the framework of the 23rdInternational Conference on Pattern Recognition (ICPR'2016). Three groups with five systems are participating to the second edition of AcTiVComp. These systems are tested in a blind manner on a closed-subset of the AcTiV database, which is unknown to all participants. In addition to the experimental setup and observed results, we also provide a short description of the participating groups and their systems.

In historical manuscripts, humans can detect handwritten words, lines, and decorations with lightness even if they do not know the language or the script. Yet for automatic processing this task has proven elusive, especially in the case of handwritten documents with complex layouts, which is why semiautomatic methods that integrate the human user into the process are needed. In this paper, we introduce a user-centered segmentation method based on document graphs and scribbling interaction. The graphs capture a sparse representation of the document's structure that can then be edited by the user with a stylus on a touch-sensitive screen. We evaluate the proposed method on a newly introduced database of historical manuscripts with complex layout and demonstrate, first, that the document graphs are already close to the desired segmentation and, second, that scribbling allows a natural and efficient interaction.

In this paper, we present an efficient page segmentation method for historical document images. Many existing methods either rely on hand-crafted features or perform rather slow as they treat the problem as a pixel-level assignment problem. In order to create a feasible method for real applications, we propose to use superpixels as basic units of segmentation, and features are learned directly from pixels. An image is first oversegmented into superpixels with the simple linear iterative clustering (SLIC) algorithm. Then, each superpixel is represented by the features of its central pixel. The features are learned from pixel intensity values with stacked convolutional autoencoders in an unsupervised manner. A support vector machine (SVM) classifier is used to classify superpixels into four classes: periphery, background, text block, and decoration. Finally, the segmentation results are refined by a connected component based smoothing procedure. Experiments on three public datasets demonstrate that compared to our previous method, the proposed method is much faster and achieves comparable segmentation results. Additionally, much fewer pixels are used for classifier training.

Ground truth is both - indispensable for training and evaluating document analysis methods, and yet very tedious to create manually. This especially holds true for complex historical manuscripts that exhibit challenging layouts with interfering and overlapping handwriting. In this paper, we propose a novel semi-automatic system to support layout annotations in such a scenario based on document graphs and a pen-based scribbling interaction. On the one hand, document graphs provide a sparse page representation that is already close to the desired ground truth and on the other hand, scribbling facilitates an efficient and convenient pen-based interaction with the graph. The performance of the system is demonstrated in the context of a newly introduced database of historical manuscripts with complex layouts.

Document Image Analysis (DIA) systems become ever more advanced, but also more complex -- computationally, and logically. This increases the difficulty of integrating existing state-of-the-art approaches into new research or into practical workflows. The current approach to sharing software is publishing source code -- leaving the burden to the integrator -- or creating a Software Development Kit (SDK) which is often restricted to one programming language. We present DIVAServices a framework for sharing and accessing DIA methods within the research community and beyond. Using a RESTful web service architecture we provide access to the methods, leading to only one system on which the binaries of methods need to be maintained. All it takes for a developer to use an algorithm is a simple HTTP request with the image data and parameters for the method and they will receive the computed results in a format that allows for seamless integration into any kind of workflow or for further processing. Furthermore, DIVAServices is open-source, enabling other research groups or libraries to host their own instance in their environment. Using this framework, future DIA systems can be built on the shoulders of well tested algorithms, accessible to everyone.

Text detection in videos is a challenging problem due to variety of text specificities, presence of complex background and anti-aliasing/compression artifacts. In this paper, we present an approach for horizontally aligned artificial text detection in Arabic news video. The novelty of this method revolves around the combination of two techniques: an adapted version of the Stroke Width Transform (SWT) algorithm and a convolutional auto-encoder (CAE). First, the SWT extracts text candidates' components. They are then filtered and grouped using geometric constraints and Stroke Width information. Second, the CAE is used as an unsupervised feature learning method to discriminate the obtained textline candidates as text or non-text. We assess the proposed approach on the public Arabic-Text-in-Video database (AcTiV-DB) using different evaluation protocols including data from several TV channels. Experiments indicate that the use of learned features significantly improves the text detection results.

In this paper, we present a Conditional Random Field (CRF) model to deal with the problem of segmenting handwritten historical document images into different regions. We consider page segmentation as a pixel-labeling problem, i.e., each pixel is assigned to one of a set of labels. Features are learned from pixel intensity values with stacked convolutional autoencoders in an unsupervised manner. The features are used for the purpose of initial classification with a multilayer perceptron. Then a CRF model is introduced for modeling the local and contextual information jointly in order to improve the segmentation. For the purpose of decreasing the time complexity, we perform labeling at superpixel level. In the CRF model, graph nodes are represented by superpixels. The label of each pixel is determined by the label of the superpixel to which it belongs. Experiments on three public datasets demonstrate that, compared to previous methods, the proposed method achieves more accurate segmentation results and is much faster.

This paper presents a novel, highly-adaptable Java framework N-light-N, for the work with deep neural networks, especially with CAEs. While the most popular deep learning libraries focus on fast processing and high performance, they only implement the main-stream network architectures and network units. In recent research in the document domain, however, we have shown that modified networks, units, and training processes significantly improve the performance in various tasks. To enable the document research community with such capabilities, in this paper we introduce a novel, publicly available Deep Learning framework which is easy to use, adapt, and extend. Furthermore, we present successful applications for three tasks, including two in the domain of handwritten historical documents, and show how the framework can be used for adaptation, optimization, and deeper analysis.

This paper introduces a publicly available historical manuscript database DIVA-HisDB for the evaluation of several Document Image Analysis (DIA) tasks. The database consists of 150 annotated pages of three different medieval manuscripts with challenging layouts. Furthermore, we provide a layout analysis ground-truth which has been iterated on, reviewed, and refined by an expert in medieval studies. DIVA-HisDB and the ground truth can be used for training and evaluating DIA tasks, such as layout analysis, text line segmentation, binarization and writer identification. Layout analysis results of several representative baseline technologies are also presented in order to help researchers evaluate their methods and advance the frontiers of complex historical manuscripts analysis. An optimized state-of-the-art Convolutional Auto-Encoder (CAE) performs with around 95% accuracy, demonstrating that for this challenging layout there is much room for improvement. Finally, we show that existing text line segmentation methods fail due to interlinear and marginal text elements.

This paper describes the AcTiVComp: detection and recognition of Arabic Text in Video competition in conjunction with the 23rd International Conference on Pattern Recognition (ICPR). The main objective of this competition is to evaluate the performance of participants' algorithms to automatically locate and/or recognize overlay text lines in Arabic video frames using the freely available AcTiV dataset. In this first edition of AcTiVComp, four groups with five systems are participating to the competition. In the detection challenge, the systems are compared based on the standard assessment metrics (i.e. recall, precision and F-score). The recognition results evaluation is based on the recognition rates at the character, word and line levels. The systems were tested in a blind manner on the closed-test set of the AcTiV dataset which is unknown to all participants. In addition to the test results, we also provide a short description of the participating groups and their systems.

In this paper, we present an unsupervised feature learning method for page segmentation of historical handwritten documents available as color images. We consider page segmentation as a pixel labeling problem, i.e., each pixel is classified as either periphery, background, text block, or decoration. Traditional methods in this area rely on carefully hand-crafted features or large amounts of prior knowledge. In contrast, we apply convolutional autoencoders to learn features directly from pixel intensity values. Then, using these features to train an SVM, we achieve high quality segmentation without any assumption of specific topologies and shapes. Experiments on three public datasets demonstrate the effectiveness and superiority of the proposed approach.

We present a novel method for adding realistic degradations to historical document images in order to generate more training data. Degradation patches are extracted from other documents and applied to the target document in the gradient domain. Working in the gradient domain has not been done for this purpose in document images analysis so far. It has the advantage to prevent color inconsistencies and allows to efficiently avoid border effects. This paper contains the detailed description of our novel method, with a focus on the mathematical aspect of the transition to and from the gradient domain. Furthermore, we perform quantitative experiments where we investigate the effects of using synthetically generated training data on historical documents with different kind of degradations.

Recently, promising results have been reported on video text detection and recognition. Most of the proposed methods are tested on private datasets with non-uniform evaluation metrics. We report here on the development of a publicly accessible annotated video dataset designed to assess the performance of different artificial Arabic text detection, tracking and recognition systems. The dataset includes 80 videos (more than 850,000 frames) collected from 4 different Arabic news channels. An attempt was made to ensure maximum diversities of the textual content in terms of size, position and background. This data is accompanied by detailed annotations for each textbox. We also present a region-based text detection approach in addition to a set of evaluation protocols on which the performance of different systems can be measured.

Automated reading of historical handwriting is needed to search and browse ancient manuscripts in digital libraries based on their textual content. In this paper, we present a combined system for text localization and transcription in page images. It includes flexible learning-based methods for layout analysis and handwriting recognition, which were developed in the context of the Swiss research project HisDoc. A comprehensive experimental evaluation is provided for the medieval Parzival database, demonstrating a promising word recognition accuracy of 93.0% with closed vocabulary. In order to harmonize the evaluation of the two document analysis tasks, we introduce a novel evaluation measure for text line extraction that takes substitution, deletion, as well as insertion errors into account.

In this paper we present a physical structure detection method for historical handwritten document images. We considered layout analysis as a pixel labeling problem. By classifying each pixel as either periphery, background, text block, or decoration, we achieve high quality segmentation without any assumption of specific topologies and shapes. Various color and texture features such as color variance, smoothness, Laplacian, Local Binary Patterns, and Gabor Dominant Orientation Histogram are used for classification. Some of these features have so far not got many attentions for document image layout analysis. By applying an Improved Fast Correlation-Based Filter feature selection algorithm, the redundant and irrelevant features are removed. Finally, the segmentation results are refined by a smoothing post-processing procedure. The proposed method is demonstrated by experiments conducted on three different historical handwritten document image datasets. Experiments show the benefit of combining various color and texture features for classification. The results also show the advantage of using a feature selection method to choose optimal feature subset. By applying the proposed method we achieve superior accuracy compared with earlier work on several datasets, e.g., We achieved 93% accuracy compared with 91% of the previous method on the Parzival dataset which contains about 100 million pixels.

Classification of the content of a scanned document as either printed or handwritten is typically tackled as a segmentation problem of pages into text lines or words. However these methods are not applicable on documents where handwritten annotations overlay printed text. In this paper we propose to treat the task as a pixel classification task, i.e., To classify individual foreground pixels into either printed or handwritten pixels. Our method uses various features of diverse nature taking the surrounding window into account. The influence of the features and their parameters are investigated and optimized on a validation set. Each foreground pixel is then classified by a multilayer perceptron using feature vectors based on a pixel neighborhood. Finally, a post-processing step corrects typical misclassifications, i.e., It removes outliers based on several heuristics. We evaluated our method on printed documents with real handwritten annotations and reached an accuracy of 96.10% on the test set. This is significantly higher than a previously published methods based on local features.

In this paper we propose a novel hybrid feature selection method for historical Document Image Analysis (DIA). Adapted greedy forward selection and genetic selection are used in a cascading way. We apply the proposed method to the task of historical document layout analysis on three handwritten datasets of diverse nature. The documents contain complex layouts, different handwriting styles, and several results of decay. The task is to segment each page into four areas: periphery, background, text block, and decoration. The proposed method selected significantly less features and resulted in significantly lower error rates than using all features. Compared to several conventional feature selection methods, the proposed method is competitive with respect to the number of selected features and the resultant error rates. In addition, we found that some features, e.g., Gradient, Laplacian, and local binary patterns (LBP), are selected by most of the feature selection methods and we give some explanations. This finding suggests a clue for the layout analysis on handwritten documents in general.

In this paper we present a novel text line segmentation method for historical manuscript images. We use a pyramidal approach where at the first level, pixels are classified into: text, background, decoration, and out of page, at the second level, text regions are split into text line and non text line. Color and texture features based on Local Binary Patterns and Gabor Dominant Orientation are used for classification. By applying a modified Fast Correlation-Based Filter feature selection algorithm, redundant and irrelevant features are removed. Finally, the text line segmentation results are refined by a smoothing post-processing procedure. Unlike other projection profile or connected components methods, the proposed algorithm does not use any script-specific knowledge and is applicable to color images. The proposed algorithm is evaluated on three historical manuscript image datasets of diverse nature and achieved an average precision of 91% and recall of 84%. Experiments also show that the proposed algorithm is robust with respect to changes of the writing style, page layout, and noise on the image.

This paper proposes a novel text line extraction method for historical documents. The method works in two steps. In the first step, layout analysis is performed to recognize the physical structure of a given document using a classification technique, more precisely the pixels of a coloured document image are classified into five classes: text-block, core-text-line, decoration, background, and periphery. This layout recognition is achieved by a cascade of two Dynamic Multilayer Perceptron (DMLP) classifiers and works without binarisation. In the second step, an algorithm takes the layout recognition results as an input, extracts the text lines, and groups them into blocks using the connected components approach. Finally, the algorithm refines the boundaries of the text lines using the binary image and the layout recognition results. Our system is evaluated on three historical manuscripts with a test set of 49 pages. The best obtained hit rate for text lines is 96.3%.

Text line segmentation is one of the main parts of document image analysis, it provides crucial information for automated reading, word spotting, alignment between image and transcription, or indexing of documents. Yet it remains an open problem for handwritten historical documents because of complex layouts on the one hand, such as curved and touching text lines, and binarization problems on the other hand, caused by ornaments, wrinkles, stains, holes, etc. In this paper, we propose a binarization-free clustering method for text line segmentation that is not only able to cope with touching text lines, but also with complex baseline curvature. Avoiding the assumption of straight baselines, small interest point clusters are grouped into text lines based on their local orientation. Experiments conducted on artificially distorted images of the Saint Gall database show promising results.

This paper describes the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text held in the context of the 12th International Conference on Document Analysis and Recognition (ICDAR'2013), during August 25-28, 2013, Washington DC, United States of America. This competition has used the freely available Arabic Printed Text Image (APTI) database. A first edition took place in ICDAR'2011. In this edition, four groups with six systems are participating in the competition. The systems are compared using the recognition rates at character and word levels. The systems were tested in a blind manner using set 6 of APTI database. A short description of the participating groups, their systems, the experimental setup, and the observed results are presented.

This paper presents a comparison between three classifiers based on Support Vector Machines, Multi-Layer Perceptrons and Gaussian Mixture Models respectively to detect physical structure of historical documents. Each classifier segments a scaled image of historical document into four classes, i.e., areas of periphery, background, text and decoration. We evaluate them on three data sets of historical documents. Depending on data sets, the best classification rates obtained vary from 90.35% to 97.47%.

In this paper we propose an approach to address the gesture segmentation issue, an important concern strongly related to the gesture recognition field. Gesture segmentation has two main goals: first, detecting when a gesture begins and ends, second, understanding whether a gesture is meant to be meaningful for the machine or is a non-command gesture (such as gesticulation). This work proposes a novel hands-free, always-available approach for the gesture segmentation and recognition in which the user can communicate directly to the system through a wearable and "intimate" interface based on electromyography signals (EMG). The system addresses the well-known "gorilla-arm" problem recognizing subtle gestures and segmenting them through motionless gestures. We report experimental results indicating that the system is able to reliably detect and recognize subtle gestures, with minimal training across users with different muscle volumes, representing a consistent gesture segmentation approach. Finally, the usability tests showed that the system is easy to use and the subjects felt quickly confident with its utilization.

This paper presents OCD Dolores, an environment that aims at recovering the logical structures from documents by interactively inferring their models. Dolores is based on OCD, an XML canonical document format used to represent structured electronic content efficiently. The relevance of our restructuring system is assessed through a deep evaluation of Dolores' logical labeling capacities.

In this paper we propose a novel interaction approach, based on gestures, aiming to recognize and enhance interactions between augmented human beings and augmented environments. We explain how this model, we called ARAMIS, allows interaction designers to fill the gap between real and virtual worlds augmenting the human itself. An enhanced interaction is achieved exploiting a hybrid approach. This approach is defined hybrid since it is the combination of several complementary techniques: wearable and pervasive computing paradigms, brute force, fuzzy and ML methods, virtual and real worlds, optical and non-optical sensing technologies. A framework implementing this concept has been developed. Finally, in order to validate our approach, we present a first prototype implemented according to the ARAMIS concept.

This paper describes a generic layout analysis system for historical documents. It presents the architecture of a pyramidal approach using three analysis levels. Each level consists of a classifier using machine learning techniques where the output of the upper level is used as a feature in the lower level. The current implementation uses a so called Dynamic Multi-Layer perceptron (DMLP), which is a natural extension of MLP classifiers. The system is evaluated on medieval documents for which a multi-layer model is used to discriminate among 10 classes organized hierarchically.

This paper describes the adaptation of a previously developed document recognition framework called PLANET (Physical Layout Analysis of complex structured Arabic documents using artificial neural NETs) into a ground truthing system for complex Arabic document images [8]. PLANET is a layout analysis tool for Arabic documents with complex structures allowing incremental learning in an interactive environment. Artificial neural nets drive the classification of homogeneous text blocks. We have observed that when users use PLANET for ground truthing, the number of interactive corrections is quite large. In order to reduce user intervention and to make use of PLANET as a ground truthing system we have adapted its architecture.

This paper describes the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text held in the context of the 11$^{th}$ International Conference on Document Analysis and Recognition (ICDAR2011), during September 18-21, 2011, Beijing, China. This first competition used the freely available Arabic Printed Text Image (APTI) database. Several research groups have started using the APTI database and this year, 2 groups with 3 systems are participating in the competition. The systems are compared using the recognition rates at the character and word levels. The systems were tested on one test dataset which is unknown to all participants (set 6 of APTI database). The systems are compared on the most important characteristic of classification systems, the recognition rate. A short description of the participating groups, their systems, the experimental setup, and the observed results are presented.

In this paper, we describe a multimodal approach for human-smart environment interaction. The input interaction is based on three modalities: deictic gestures, symbolic gestures and isolated-words. The deictic gesture is interpreted using the PTAMM (Parallel Tracking and Multiple Mapping) method exploiting a camera handheld or worn on the user arm. The PTAMM algorithm tracks in real-time the position and orientation of the hand in the environment. This information is used to point real or virtual objects, previously added to the environment, using the optical camera axis. Symbolic hand-gestures and isolated voice commands are recognized and used to interact with the pointed target. Haptic and acoustic feedbacks are provided to the user in order to improve the quality of the interaction. A complete prototype has been realized and a first usability evaluation, assessed with the help of 10 users has shown positive results.

PDF documents are widely used but the extraction and the manipulation and of their structured content is not an easy task. It requires sophisticated pre-processing and reverse engineering techniques to get such achievements. In this paper, we present an improvement of XED in order to handle unresolved issues related to the analysis of Arabic documents. A set of rules were proposed and implemented to enhance the extraction of Arabic content, by taking care of the different Arabic fonts, through mapping the un-interpreted Unicode values to the other interpreted sets as well as applying a reverse algorithm whenever needed. We finally expose concrete evaluations for the improvement of XED.

Medieval manuscript layouts are quite complex. Additionally to their main text flow, which can spread over one or several columns, such manuscripts contain also other textual elements such as insertions, annotations, and corrections. They are often richly decorated with ornaments, illustrations, and drop capitals making their layout even more complex. In this paper we propose a generic layout model to represent their physical structure.To achieve this goal we propose to use four layers in order to distinguish between the different graphical elements. In this paper we show how this model is used to represent automatic segmentation results and how it allows a quantitative measure of their accuracy.

A known difficulty of Arabic text recognition is in the large variability of printed representation from one font to the other. In this paper, we present a comparative study between two strategies for the recognition of multi-font Arabic text. The first strategy is to use a global recognition system working independently on all the fonts. The second strategy is to use a so-called cascade built from a font identification system followed by font-dependent systems. In order to reach a fair comparison, the feature extraction and the modeling algorithms based on HMMs are kept as similar as possible between both approaches. The evaluation is carried out on the large and publicly available APTI (Arabic Printed Text Image) database with 10 different fonts. The results are showing a clear advantage of performance for the cascading approach. However, the cascading system is more costly in terms of cpu and memory.

Medieval manuscript layouts are quite complex. They contain textual elements such as insertions, annotations, and corrections. They may be richly decorated with ornaments, illustrations, and decorative initials making their layout even more complex. In this paper we describe a semi-automatic tool which annotates medieval manuscripts using our generic format. This format allows to represent the physical structure of such manuscripts. Our semi-automatic tool is composed of two parts. The first part achieves a layout analysis which automatically segments manuscripts into text blocks and text lines. That is, a Multi-Layer Perceptron (MLP) identifies layout elements by using color features, it extracts the textual content image of the manuscript. Then, a segmentation based on Connected Component (CC) is performed on the textual content in order to retrieve text blocks and lines. The second part provides an interactive interface allowing the user to customize the automatic analysis, to visualize its results, and to correct them. Our tool is still a prototype, nevertheless, first experiments give encouraging results. Thus, in this paper, we show how to generate a ground truth for medieval manuscripts layouts.

We analyze in this paper the impact of sub-models choice for automatic Arabic printed text recognition based on Hidden Markov Models (HMM). In our approach, sub-models correspond to characters shapes assembled to compose words models. One of the peculiarities of Arabic writing is to present various character shapes according to their position in the word. With 28 basic characters, there are over 120 different shapes. Ideally, there should be one sub model for each different shape. However, some shapes are less frequent than others and, as training databases are finite, the learning process leads to less reliable models for the infrequent shapes. We show in this paper that an optimal set of models has then to be found looking for the trade-off between having more models capturing the intricacies of shapes and grouping the models of similar shapes with other. We propose in this paper different sets of sub-models that have been evaluated using the Arabic Printed Text Image (APTI) Database freely available for the scientific community.

We present in this paper a new approach for Arabic font recognition. Our proposal is to use a fixed-length sliding window for the feature extraction and to model feature distributions with Gaussian Mixture Models (GMMs). This approach presents a double advantage. First, we do not need to perform a priori segmentation into characters, which is a difficult task for arabic text. Second, we use versatile and powerful GMMs able to model finely distributions of features in large multi-dimensional input spaces. We report on the evaluation of our system on the APTI (Arabic Printed Text Image) database using 10 different fonts and 10 font sizes. Considering the variability of the different font shapes and the fact that our system is independent of the font size, the obtained results are convincing and compare well with competing systems.

Revealing and being able to manipulate the structured content of PDF documents is a difficult task, requiring pre-processing and reverse engineering techniques. In this paper, we present OCD, an optimized, easy-to-process and canonical format for representing structured electronic documents. The system and methods used for reverse engineering PDF documents into the OCD format are presented as well as the techniques to optimize it. We finally expose concrete evaluations of our OCD format compactness and restructuring performances.

In this paper, we propose a comparative study between the affixal approach and the analytical approach for off-line Arabic decomposable word recognition. The analytical approach is based on the modeling of alphabetical letters. The affixal approach is based on the modeling of the linguistic entity namely prefix, infix, suffix and root. The experimental results obtained by these two last approaches are presented on the basis of the printed decomposable word data set in mono-font nature by varying the character sizes. We achieve then our paper by the current improvements of our works concerning the Arabic multi-font, multi-style and multi-size word recognition.

We report on the creation of a database composed of images of Arabic Printed words. The purpose of this database is the large-scale benchmarking of open-vocabulary, multi-font, multi-size and multi-style text recognition systems in Arabic. The challenges that are addressed by the database are in the variability of the sizes, fonts and style used to generate the images. A focus is also given on low-resolution images where anti-aliasing is generating noise on the characters to recognize. The database is synthetically generated using a lexicon of 113psila284 words, 10 Arabic fonts, 10 font sizes and 4 font styles. The database contains 45psila313psila600 single word images totaling to more than 250 million characters. Ground truth annotation is provided for each image. The database is called APTI for Arabic Printed Text Images.

This article proposes an evaluation framework to benchmark the performance of multimodal fusion engines. The paper first introduces different concepts and techniques associated with multimodal fusion engines and further surveys recent implementations. It then discusses the importance of evaluation as a mean to assess fusion engines, not only from the user perspective, but also at a performance level. The article further proposes a benchmark and a formalism to build testbeds for assessing multimodal fusion engines. In its last section, our current fusion engine and the associated system HephaisTK are evaluated thanks to the evaluation framework proposed in this article. The article concludes with a discussion on the proposed quantitative evaluation, suggestions to build useful testbeds, and proposes some future improvements.

This article introduces HephaisTK, a toolkit for rapid prototyping of multimodal interfaces. After briefly discussing the state of the art, the architecture traits of the toolkit are displayed, along with the major features of HephaisTK: agent-based architecture, ability to plug in easily new input recognizers, fusion engine and configuration by means of a SMUIML XML file. Finally, applications created with the HephaisTK toolkit are discussed.

In this paper, we report on the development of an efficient user authentication system based on a combined acquisition of online pen and speech signals. The novelty of our approach is in the simultaneous recording of these two modalities, simply asking the user to utter what she/he is writing. The main benefit of this multimodal approach is a better accuracy at no extra costs in terms of access time or inconvenience. Another benefit comes from an increased difficulty for forgers willing to perform imitation attacks as two signals need to be reproduced. We are comparing here two potential scenarios of use. The first one is calledspokensignatureswhere the user signs and says the content of the signature. The second scenario is based onspokenhandwritingwhere the user is prompted to write and read the content of sentences randomly extracted from a text. Data according to these two scenarios have been recorded from a set of 70 users. In the first part of this paper, we describe the acquisition procedure, and we comment on the viability and usability of such simultaneous recordings. Our conclusions are supported by a short survey performed with the users. In the second part, we present the authentication systems that we have developed for both scenarios. More specifically, our strategy was to model independently both streams of data and to perform a fusion at the score level. Starting from a state-of-the-art-modeling algorithm based on Gaussian Mixture Models trained with an Expectation-Maximization procedure, we report on several significant improvements that are brought. As a general observation, the use of both modalities outperforms significantly the modalities used alone.

Physical and logical structure recovering from electronic documents is still an open issue. In this paper, we propose a flexible and efficient approach for recovering document structures from PDF files. After a brief introduction of the PDF format and its major features, we report about our evaluation of different existing tools and works for PDF content extraction and analysis. To overcome the weaknesses of these systems, we propose a new analysis strategy, based on an intermediate representation, called XCDF, which enables representing physical structures in a canonical way. This paper then describes the PDF reverse engineering workflow and focuses on the document logical restructuring. Finally, the paper concludes with potential future improvements.

This paper reviews the challenges associated with the development of tangible and multimodal interfaces and exposes our experiences with the development of three different software architectures to rapidly prototype such interfaces. The article first reviews the state of the art, and further compares existing systems with our approaches. Finally, the article stresses the major issues associated with the development of toolkits allowing the creation of multimodal and tangible interfaces, and presents our future objectives.

XML and its associated languages, namely DTD, XML Schema and XSLT, have tremendous importance for lots of applications even if their semantics is often hard to understand and incomplete. In this paper, we concentrate on transformation languages and propose a new one in XML syntax and focusing on strong specifications. Since our language is completely defined by formal semantics, conceptual drawbacks have been avoided and complexity has been reduced. Thus, static type checking could easily be provided. Finally, we combine our transformation language with our own schema language in order to perform static typing.

We report on our developments towards building a novel user authentication system using combined acquisition of online handwritten signature and speech modalities. In our approach, signatures are recorded by asking the user to say what she/he is writing, leading to the so-called spoken signatures. We have built a verification system composed of two Gaussian mixture models (GMMs) sub-systems that model independently the pen and voice signal. We report on results obtained with two algorithms used for training the GMMs, respectively expectation maximization and maximum a posteriori adaptation. Different algorithms are also compared for fusing the scores of each modality. The evaluations are conducted on spoken signatures taken from the MyIDea multimodal database, accordingly to the protocols provided with the database. Results are in favor of using MAP adaptation with a simple weighted sum fusion. Results show also clearly the impact of time variability and of skilled versus unskilled forgeries attacks.

We are proposing a novel and efficient user authentication system using combined acquisition of online handwriting and speech signals. In our approach, signals are recorded by asking the user to say what she or he is simultaneously writing. This methodology has the clear advantage of acquiring two sources of biometric information at no extra cost in terms of time or inconvenience. We have built a straightforward verification system to model these signals using statistical models. It is composed of two Gaussian mixture models (GMMs) sub-systems that takes as input features extracted from the pen and voice signals. The system is evaluated on Myldea, a realistic multimodal biometric database. Results show that the use of both speech and handwriting modalities outperforms significantly these modalities used alone. We also report on the evaluations of different training algorithms and fusion strategies.

The sound of phonographic records is contained in the radial position of the groove (and sometimes also its depth). The sound is usually extracted by a mechanical device (the stylus of the record player). Many old records are in such bad shape that no mechanical contact is possible. An alternative is to take a picture of the record and to estimate the radial groove position, using image processing. In this paper, we present the technique to extract the radial position of the groove and therefore the sound of the records.

The paper describes a method by which one could use the documents captured from low-resolution handheld devices to retrieve the originals of those documents from a document store. The method considers conjunctively two complementary feature sets. First, the geometrical distribution of the color in the document's 2D image plane is preferred. Secondly, the shallow layout features is considered due to the poor resolution of the captured documents. We propose in this article to fuse those two complementary feature sets in order to improve document identification performance. Finally, in order to test the influence of merging strategies on document identification performance, a synergic method is proposed and evaluated relative to a similar method in which feature sets are simply considered sequentially.

This paper proposes a multi-signature document identification method that works robustly with low-resolution documents captured from handheld devices. The proposed method is based on the extraction of a visual signature containing both (a) the color content distribution in the image plane of the document, i.e. the color signature, and (b) the shallow layout structure of the document, i.e. the layout signature. The color distribution is first considered, in order to filter documents with very dissimilar colors, and the identification is finally done on the remaining set using the layout signature. An evaluation, that compares our color and layout-based method with the layout signature alone, is finally presented.

Logical structure analysis is an important phase in the process of document image understanding. In this paper we propose a learning-based method to label logical components on Arabic newspaper documents. The labeling is driven by artificial neural nets. Each one is specialized in a document class. The first prototype of LUNET has been tested on a set of Arabic newspapers of three document classes. Some promising experimental results are reported.

Relationships that link static documents discussed during meetings to the corresponding speech transcripts can be of various kinds. The most important ones, thematic links, quotations and references are presented in this paper. Thematic links are detected via a thematic alignment process. However, quotations extraction is based on the detection of segments of documents that are quoted in the speech transcript. References, made by speakers to documents, are performed via a matching process between referring expressions detected in the speech transcript, and corresponding documents logical blocks. Finally, a framework that combines these links and an evaluation of the links complementarity are presented.

This article presents Xed, a reverse engineering tool for PDF documents, which extracts the original document layout structure. Xed mixes electronic extraction methods with state-of-the-art document analysis techniques and outputs the layout structure in a hierarchical canonical form, i.e. which is universal and independent of the document type. This article first reviews the major traps and tricks of the PDF format. It then introduces the architecture of Xed along with its main modules, and, in particular, the document physical structure extraction algorithm. Later on, a canonical format is proposed and discussed with an example. Finally the results of a practical evaluation are presented, followed by an outline of future works on the logical structure extraction.

In this paper, we describe a new method for a simultaneous thematic segmentation of the meeting dialogs and the documents discussed or visible throughout the meeting. This bi-modal method is suitable for multimodal applications that are centered on documents, such as meetings and lectures, where documents can be aligned with meeting dialogs. Bringing into play this alignment, our bi-modal segmentation method first transforms its results into a set of nodes in a 2D graph space, where the two axes represent respectively the document units and the meeting dialogs units. Secondly, via a clustering method, the most connected regions in the constituted bi-graph are detected. Finally, the denser clusters are projected on the two axes. The two sequences of segments, obtained on both axes, represent the thematic structure of the document and of the meeting dialogs respectively. We present in this article this bi-modal segmentation technique and its performance compared with two mono-modal segmentation methods. Categories and Subject Descriptors H.3.1 [Content Analysis and Indexing] indexing methods;

In this paper, we present (a) a method for identifying documents captured from low-resolution devices such as web-cams, digital cameras or mobile phones and (b) a technique for extracting their textual content without performing OCR. The first method associates a hierarchically structured visual signature to the low-resolution document image and further matches it with the visual signatures of the original high-resolution document images, stored in PDF form in a repository. The matching algorithm follows the signature hierarchy, which speeds-up the search by guiding it towards fruitful solution spaces. In a second step, the content of the original PDF document is extracted, structured, and matched with its corresponding high-resolution visual signature. Finally, the matched content is attached to the low-resolution document image's visual signature, which greatly enriches the document's content and indexing. We present in this article both these identification and extraction methods and evaluate them on various documents, resolutions and lighting conditions, using different capture devices.

The optical retrieval and storage technique called VisualAudio provides a way to extract sound information from an analog disk without any mechanical contact. The process is straightforward: we take a picture of each side of the disk using a dedicated analog camera, we store the film as our working copy, and when needed, we scan the film and process the image in order to extract the sound. This technique can be used to retrieve the sound from old records that are in such a bad shape that no regular turntable can be used. A working prototype has been built and has retrieved the sound from several records. We present the signal processing issues of this system and, in particular, a signal-to-noise ratio analysis.

In the context of a multimodal application, the article proposes an image-based method for bridging the gap between document excerpts and video extracts. The approach, called document image alignment, takes advantage of the observable events related to documents that are visible during meetings. In particular, the article presents a new method for detecting slide changes in slideshows, its evaluation, and a preliminary work on document identification.

This article proposes a multimodal approach for segmenting meeting recordings. This bi-modal method takes advantages of the alignment of speech transcript with documents, in the context of meetings or lectures, where documents are discussed. The method first displays the alignment results as a set of nodes in a 2D space, where the two axes represent respectively the documents content and the speech transcript. The most connected regions in this graph are detected using a clustering method. The final clusters are then projected on the speech axis. Finally, the obtained sequence of segments is considered as the thematic structure of the speech transcript. In this article, we present our bi-modal method and compare it with two other mono-modal thematic segmentation methods.

We present in this article a method for detecting similarity links between documents' content and speech recordings' content. This process, further called thematic alignment, is a novel research area that combines both document and speech analysis. This alignment will a) provide temporal indexes to documents, which are non-temporal data, and b) help discovering hidden thematic structures. This article first introduces a multi-layered document structure and quickly introduces the traditional speech structure. Further, it presents a simple similarity measure and various multi-level simple alignments between those two structures. Later, the meeting corpus is presented, as well as an evaluation of the implemented alignments. Finally, we present our future works on multi-alignments and thematic structure discovery.

The aim of layout analysis is to extract the geometricstructure from a document image. It consists of labelinghomogenous regions of a document image. This paperdescribes the performance of segmentation algorithmsand their adaptation in order to treat complex structuredArabic documents such as newspapers. Experimentaltests have been carried out on four different phases ofnewspaper image analysis: thread recognition, framerecognition, image text separation, text line recognition,and line merging into blocks. Some promisingexperimental results are reported.

Indexing large newspaper archives requires automatic page decomposition algorithms with high accuracy. In this paper, we present our approach to an automatic page decomposition algorithm developed for the First International Newspaper Segmentation Contest. Our approach decomposes the newspaper image into image regions, horizontal and vertical lines, text regions and title areas. Experimental results are obtained from the data set of the contest.

The paper presents ongoing work on the design of a Web-based framework for cooperative document understanding. The authors begin by exposing their motivations for designing a new document understanding environment. They then describe the different levels of cooperation they intend to support and how Web technologies can help in this respect. Finally, the authors present Edelweiss, the framework we currently being developing based on this approach.

Interoperability and reusability are becoming challenging problems in document engineering that cannot be solved satisfactorily with converters and filters. This paper advocates a new approach for logical structure recovery by means of document recognition techniques that operate on synthetic images, such as those rendered by a PostScript interpreter. Differences with respect to scanned images are discussed on the basis of a sample application.

A new statistical approach based on global typographical features is proposed to the widely neglected problem of font recognition. It aims at the identification of the typeface, weight, slope and size of the text from an image block without any knowledge of the content of that text. The recognition is based on a multivariate Bayesian classifier and operates on a given set of known fonts. The effectiveness of the adopted approach has been experimented on a set of 280 fonts. Font recognition accuracies of about 97 percent were reached on high-quality images. In addition, rates higher than 99.9 percent were obtained for weight and slope detection. Experiments have also shown the system robustness to document language and text content and its sensitivity to text length.

Assisted document recognition systems have to integrate automatic recognition, manual edition and incremental learning in a single interactive environment. This paper raises the question of the organization of these three kinds of operations. When an analyzer has the ability to improve with use, there is a tradeoff between the benefits of enhancing the accuracy of automatic analysis, and the additional time spent in interacting for feedback communication. The global cost depends then on the sequence of processed entities, and on the relevance of the learning transactions. Notations are introduced to describe the evolution of a recognition session, and possible organization strategies are discussed. Then a cost model is presented to allow the comparison between different organization schemes. We describe some concrete experiments of cost measures with the ApOFIS font identification tool and the ScanWorX OCR; the first results show that a user-driven approach can potentially save substantial effort in the recognition process, in comparison with machine-driven systems.

We present and discuss a novel approach to modeling logical structures of documents, based on a statistical representation of patterns in a document class. An efficient and error tolerant recognition heuristics adapted to the model is proposed. The statistical approach permits easily automated and incremental learning of the model. The approach has been partially evaluated on a prototype. A discussion of the results achieved by the prototype is finally made.

This paper focuses on the design of a software architecture that can support the programming challenges of CIDRE. This is in itself an important research issue, because in document image analysis as in many other domains, it is a real challenge to integrate coherently all the existing knowledge; that's why the effort should not only be put on inventing new algorithms for specific subtasks, but also on finding a relevant framework to combine various software components. Section 2 addresses the architectural problems raised by the specification of the desired system. Our proposition is presented in section 3, in terms of the integration of existing software components. We show then in section 4 how the proposed framework was successfully used for a restricted application, and what kind of problems are expected in a generalization of the prototype.

We present a segmentation method guided by a generic layout description expressed in a new language. The proposed language allows to describe a page as superposed layers that may be used to separate the main text body from other components, for example figures. The language's novelty resides in the fact that, instead of describing directly the global topology of generic pages according to their regions, generic separators are described and used as region boundary delimiters. Separators may be declared as white spaces or threads. By doing this, the problem of document segmentation into regions has become a problem of separator determination, solved by analyzing lines and white spaces contained in documents.

A font recognition system allowing the identification of font families, weights, slopes and sizes with an accuracy of 99% for weights and slopes and 96% for families and sizes, has been developed. Our system uses a knowledge base of 240 fonts models, which have been created from a training set of text images written with these different fonts. In this paper, a study of image degradations effects on the system performances, is presented. The evaluation that has been carried out, shows that the system is robust against natural degradations such as those introduced by scanning and photocopying, but its performances decrease with very degraded document images. In order to avoid this weakness, a degradation modeling strategy has been adopted, allowing an automatic adaptation of the system to these degradations. The adaptation is derived from statistical analysis of features behavior against degradations and is performed by specific transformations applied to the system knowledge base. Some promising results are reported.