In recent decades, Recommender Systems (RS) have undergone significant advancements, particularly in popular domains like movies, music, and product recommendations. Yet, progress has been notably slower in leveraging these systems for personal information management and knowledge assistance. In addition to challenges that complicate the adoption of RS in this domain (such as privacy concerns, heterogeneous recommendation items, and frequent context switching), a significant barrier to progress in this area has been the absence of a standardized benchmark for researchers to evaluate their approaches. In response to this gap, this paper presents a benchmark built upon a publicly available dataset of Real-Life Knowledge Work in Context (RLKWiC). This benchmark focuses on evaluating context-based entity recommendation, a use case for leveraging RS to support knowledge workers in their daily digital tasks. By providing this benchmark, it is aimed to facilitate and accelerate research efforts in enhancing personal knowledge assistance through RS.

The RDF Mapping Language (RML) allows to map semi-structured data to RDF knowledge graphs. Besides CSV, JSON and XML, this also includes the mapping of spreadsheet tables. Since spreadsheets have a complex data model and can become rather messy, their mapping creation tends to be very time consuming. In order to reduce such efforts, this paper presents Spread2RML which predicts RML mappings on messy spreadsheets. This is done with an extensible set of RML object map templates which are applied for each column based on heuristics. In our evaluation, three datasets are used ranging from very messy synthetic data to spreadsheets from data.gov which are less messy. We obtained first promising results especially with regard to our approach being fully automatic and dealing with rather messy data.

Text document classification is an important task for diverse natural language processing based applications. Traditional machine learning approaches mainly focused on reducing dimensionality of textual data to perform classification. This although improved the overall classification accuracy, the classifiers still faced sparsity problem due to lack of better data representation techniques. Deep learning based text document classification, on the other hand, benefitted greatly from the invention of word embeddings that have solved the sparsity problem and researchers focus mainly remained on the development of deep architectures. Deeper architectures, however, learn some redundant features that limit the performance of deep learning based solutions. In this paper, we propose a two stage text document classification methodology which combines traditional feature engineering with automatic feature engineering (using deep learning). The proposed methodology comprises a filter based feature selection (FSE) algorithm followed by a deep convolutional neural network. This methodology is evaluated on the two most commonly used public datasets, i.e., 20 Newsgroups data and BBC news data. Evaluation results reveal that the proposed methodology outperforms the state-of-the-art of both the (traditional) machine learning and deep learning based text document classification methodologies with a significant margin of 7.7% on 20 Newsgroups and 6.6% on BBC news datasets.

This paper presents a novel two-stream approach for document image classification. The proposed approach leverages textual and visual modalities to classify document images into ten categories, including letter, memo, news article, etc. In order to alleviate dependency of textual stream on performance of underlying OCR (which is the case with general content based document image classifiers), we utilize a filter based feature-ranking algorithm. This algorithm ranks the features of each class based on their ability to discriminate document images and selects a set of top 'K' features that are retained for further processing. In parallel, the visual stream uses deep CNN models to extract structural features of document images.Finally, textual and visual streams are concatenated together using an average ensembling method. Experimental results reveal that the proposed approach outperforms the state-of-the-art system with a significant margin of 4.5% on publicly available Tobacco-3482 dataset.

In this era of advanced technology and automation, information extraction has become a very common practice for the analysis of data. A technique known as Optical Character Recognition (OCR) is used for recognition of text. The purpose is to extract textual data for automatic information analysis or natural language processing of document images. However, in the field of cheminformatics where it is required to recognize 2D molecular structures as they are published in research journals or patent documents, OCR is not adequate for processing, as chemical compounds can be represented both in textual as well as in graphical format. The digital representation of an image based chemical structure allows not only patent analysis teams to provide customize insights but also cheminformatic research groups to enhance their molecular structure databases, which further can be used for querying structure as well as sub-structural patterns. Some tools have been made for extraction and processing of image-based molecular structures. Optical Structure Recognition Application (OSRA) being one of the tools that partially fulfill the task of recognizing chemical structural in document images into chemical formats (SMILES, SDF, or MOL). However, it has few problems such as poor character recognition, false structure extraction, and slow processing. In this paper, we have developed a prototype Chemical Structure Recognition (CSR) system using modern and advanced image processing open-source libraries, which allows us to extract structural information of a chemical structure embedded in the form of a digital raster image. The CSR system is capable of processing chemical information contained in chemical structure image and generates the SMILES or MOL representation. For performance evaluation, we have used two different data sets to measure the potential of the CSR system. It yields better results than OSRA that depict accurate recognition, fast extraction, and correctness of great significance.

In the context of historical documents, where labeled training data is especially expensive to acquire, the prospect of using unlabeled training data to improve and speed up the overall training is desirable. The most common way to use unlabeled data is unsupervised pretraining, which has been successfully applied to various CNN and RNN architectures in different domains. There is however not sufficient work for its application in the field of OCR. In this paper we investigate multiple architectures and how unlabeled data could be applied to them. We show that in combination with Connectionist Temporal Classification (CTC), a reconstruction objective has no apparent synergistic effect, with both objectives learning different representations. We therefore investigate the use of an LSTM-based Seq2Seq OCR architecture which shows promise regarding unsupervised pretraining.

This paper presents a novel method for the analysis of tabular structures in document images using the potential of deformable convolutional networks. In order to assess the suitability of the model to the task of table structure recognition, most of the prior methods have been tested on the smaller ICDAR-13 table structure recognition dataset comprising of just 156 tables. We curated a new image-based table structure recognition dataset, TabStructDB2, comprising of 1081 tables densely labeled with row and column information. Instead of collecting new images for this purpose, we leveraged the famous Page-Object Detection dataset from ICDAR-17, and added structural information for all the tabular regions present in the dataset. This new publicly available dataset will enable the development of more sophisticated table structure recognition techniques in the future. We performed extensive evaluation on the two datasets (ICDAR-13 and TabStructDB) including cross-dataset testing in order to evaluate the efficacy of the proposed approach. We achieved state-of-the-art results with deformable models on ICDAR-13 with an average F-Measure of 92.98% (89.42% for rows and 96.55% for columns) and report baseline results on TabStructDB for guiding future research efforts with an F-Measure of 93.72% (91.26% for rows and 95.59% for columns). Despite promising results, structural analysis of tables with arbitrary layouts is still far from achievable at this point.

Based on the recent advancements in the domain of semantic segmentation, Fully-Convolutional Networks (FCN) have been successfully applied for the task of table structure recognition in the past. We analyze the efficacy of semantic segmentation networks for this purpose and simplify the problem by proposing prediction tiling based on the consistency assumption which holds for tabular structures. For an image of dimensions H × W, we predict a single column for the rows (ŷrowϵ H) and a predict a single row for the columns (ŷrowϵ W). We use a dual-headed architecture where initial feature maps (from the encoder-decoder model) are shared while the last two layers generate class specific (row/column) predictions. This allows us to generate predictions using a single model for both rows and columns simultaneously, where previous methods relied on two separate models for inference. With the proposed method, we were able to achieve state-of-the-art results on ICDAR-13 image-based table structure recognition dataset with an average F-Measure of 92.39% (91.90% and 92.88% F-Measure for rows and columns respectively). With the proposed method, we were able to achieve state-of-the-art results on ICDAR-13. The obtained results advocate that constraining the problem space in the case of FCN by imposing valid constraints can lead to significant performance gains.

Optical Character Recognition (OCR) has achieved its state-of-the-art performance with the use of Deep Learning for character recognition. Deep Learning techniques need large amount of data along with ground truth. Out of the available data, small portion of it has to be used for validation purpose as well. Preparing ground truth for historical documents is expensive and hence availability of data is of utmost concern. Jenckel et al. jenckel came up with an idea of using all the available data for training the OCR model and for the purpose of validation, they generated the input image from Softmax layer of the OCR model; using the decoder setup which can be used to compare with the original input image to validate the OCR model. In this paper, we have explored the possibilities of using Generative Adversial Networks (GANs) [6] for generating the image directly from the text obtained from OCR model instead of using the Softmax layer which is not always accessible for all the Deep Learning based OCR models. Using text directly to generate the input image back gives us the advantage to use this pipeline for any OCR models even whose Softmax layer is not accessible. In the results section, we have shown that the current state of using GANs for unsupervised OCR model evaluation.

A growing number of applications users daily interact with have to operate in (near) real-time: chatbots, digital companions, knowledge work support systems - just to name a few. To perform the services desired by the user, these systems have to analyze user activity logs or explicit user input extremely fast. In particular, text content (e.g. in form of text snippets) needs to be processed in an information extraction task. Regarding the aforementioned temporal requirements, this has to be accomplished in just a few milliseconds, which limits the number of methods that can be applied. Practically, only very fast methods remain, which on the other hand deliver worse results than slower but more sophisticated Natural Language Processing (NLP) pipelines. In this paper, we investigate and propose methods for real-time capable Named Entity Recognition (NER). As a first improvement step, we address word variations induced by inflection, for example present in the German language. Our approach is ontology-based and makes use of several language information sources like Wiktionary. We evaluated it using the German Wikipedia (about 9.4B characters), for which the whole NER process took considerably less than an hour. Since precision and recall are higher than with comparably fast methods, we conclude that the quality gap between high speed methods and sophisticated NLP pipelines can be narrowed a bit more without losing real-time capable runtime performance.

Subjective visual interpretation is a challenging yet important topic in computer vision. Many approaches reduce this problem to the prediction of adjective- or attribute-labels from images. However,most of these do not take attribute semantics into account, or only process the image in a holistic manner. Furthermore, there is alack of relevant datasets with fine-grained subjective labels. In this paper, we propose the Focus-Aspect-Value (FAV) model to structure the process of capturing subjectivity in image processing,and introduce a novel dataset following this way of modeling. We run experiments on this dataset to compare several deep learning methods and find that incorporating context information based on tensor multiplication outperforms the default way of information fusion (concatenation).

Text alignment is an important step for analyzing historical archives. For analysis, it is important to align the text with their corresponding document images. It is a time and labor intensive work for many paleographers. In this paper, we have presented an end-to-end semi-automatic and interactive text alignment system for historical document. The presented system consists of five main sequential steps: binarization, automatic text-line extraction, interactive error correction in extracted text-line, automatic text alignment, and interactive error correction in aligned text. The anyOCR system [1] is used for the first three steps. Afterwards, text alignment is done automatically by the system using Oriented Fast and Rotated Brief (ORB) local image feature descriptors. The ORB features are matched by k-Nearest-Neighbor (KNN). Finally, the system provides an interactive user interface for rectifying wrong text alignment. The results are discussed in the evaluation section.

Layout analysis, mainly including binarization and text-line extraction, is one of the most important performance determining steps of an OCR system for complex medieval historical document images, which contain noise, distortions and irregular layouts. In this paper, we present a novel text-line error correction technique which include a VGG Net to classify non-text-line and adversarial network approach to obtain the layout bounding mask. The presented text-line error correction technique are applied to a collection of 15th century Latin documents, which achieved more than 75% accuracy for segmentation techniques.

Although the performance of the state-of-the-art OCR systems is very high, they can still introduce errors due to various reasons, and when it comes to historical documents with old manuscripts the performance of such systems gets even worse. That is why Post-OCR error correction has been an open problem for many years. Many state-of-the-art approaches have been introduced through the recent years. This paper contributes to the field of Post-OCR Error Correction by introducing two novel deep learning approaches to improve the accuracy of OCR systems, and a post processing technique that can further enhance the quality of the output results. These approaches are based on Neural Machine Translation (NMT) and were motivated by the great success that deep learning introduced to the field of Natural Language Processing. Finally, we will compare the state-of-the-art approaches in Post-OCR Error Correction with the newly introduced systems and discuss the results.

In this contemporaneous world, it is an obligation for any organization working with documents to end up with the insipid task of classifying truckload of documents, which is the nascent stage of venturing into the realm of information retrieval and data mining. But classification of such humongous documents into multiple classes, calls for a lot of time and labor. Hence a system which could classify these documents with acceptable accuracy would be of an unfathomable help in document engineering. We have created multiple classifiers for document classification and compared their accuracy on raw and processed data. We have garnered data used in a corporate organization as well as publicly available data for comparison. Data is processed by removing the stop-words and stemming is implemented to produce root words. Multiple traditional machine learning techniques like Naive Bayes, Logistic Regression, Support Vector Machine, Random forest Classifier and Multi-Layer Perceptron are used for classification of documents. Classifiers are applied on raw and processed data separately and their accuracy is noted. Along with this, Deep learning technique such as Convolution Neural Network is also used to classify the data and its accuracy is compared with that of traditional machine learning techniques. We are also exploring hierarchical classifiers for classification of classes and subclasses. The system classifies the data faster and with better accuracy than if done manually. The results are discussed in the results and evaluation section.

End-to-end Optical Character Recognition (OCR) systems are heavily used to convert document images into machine-readable text. Commercial and open-source OCR systems (like Abbyy, OCRopus, Tesseract etc.) have traditionally been optimized for contemporary documents like books, letters, memos, and other end-user documents. However, these systems are difficult to use equally well for digitizing historical document images, which contain degradations like non-uniform shading, bleed-through, and irregular layout; such degradations usually do not exist in contemporary document images.

This paper presents a novel Academic Community Explorer (ACE) which performs syntactic, semantic and pragmatic document analysis of scientific publications. Firstly, ACE uses syntactic structure to extract relevant information from a scientific document. Secondly, semantic analysis is performed to derive an article based co-authorship and citation network. Finally, ACE uses these document based networks to build a complete community network for pragmatic analysis. Furthermore, scientometric analysis is performed to extract the pragmatics by analyzing authors and publication community networks through micro and macro indicators. Two novel micro indicators Senti-Index, reflecting the sentiment present in citations and, Overlap index, reflecting community behavior have been introduced. This is a step in the direction of automatic qualitative assessment of scientific documents. In addition, ACE provides a rich visualization interface which helps in exploratory analysis of the community to identify hidden patterns, e.g, isolated small groups in the community which collaborate and cite each other frequently. A feasibility study is performed on the corpus of ICDAR publications from 1993-2015 to show the insights and benefits of the ACE framework. The results reveals that ICDAR is a highly collaborative community which has most likely arrived at its 'phase transition' stage with 70% of the community closely connected to each other.

Currently an intensive amount of research is going on in the field of digitizing historical archives for converting scanned document images into searchable full text. This paper presents the "anyOCR" system which mainly emphasize the techniques requires for digitizing a historical archive with high accuracy. It is an open-source system for the research community who can easily apply the anyOCR system for digitizing historical archives. The anyOCR system supports a complete document processing pipeline, which includes layout analysis, training OCR models and text line prediction, with an addition of intelligent and interactive layout and OCR error corrections web applications. The anyOCR system can also be used for contemporary document images containing diverse, simple to complex, layouts. This paper describes the current state of the anyOCR system, its architecture, as well as its major features. This paper also provides information about the availability, documentation, and tutorials of the anyOCR system.

This paper presents a novel approach, called AirScript, for creating, recognizing and visualizing documents in air. We present a novel algorithm, called 2-DifViz, that converts the hand movements in air (captured by a Myo-armband worn by a user) into a sequence of x, y coordinates on a 2D Cartesian plane, and visualizes them on a canvas. Existing sensor-based approaches either do not provide visual feedback or represent the recognized characters using prefixed templates. In contrast, AirScript stands out by giving freedom of movement to the user, as well as by providing a real-time visual feedback of the written characters, making the interaction natural. AirScript provides a recognition module to predict the content of the document created in air. To do so, we present a novel approach based on deep learning, which uses the sensor data and the visualizations created by 2-DifViz. The recognition module consists of a Convolutional Neural Network (CNN). and two Gated Recurrent Unit (GRU) Networks. The output from these three networks is fused to get the final prediction about the characters written in air. AirScript can be used in highly sophisticated environments like a smart classroom, a smart factory or a smart laboratory, where it would enable people to annotate pieces of texts wherever they want without any reference surface. We have evaluated AirScript against various well-known learning models (HMM, KNN, SVM, etc.) on the data of 12 participants. Evaluation results show that the recognition module of AirScript largely outperforms all of these models by achieving an accuracy of 91.7% in a person independent evaluation and a 96.7% accuracy in a person dependent evaluation.

Tables are an easy way to represent information in a structural form. Table recognition is important for the extraction of such information from document images. Usually, modern OCR systems provide textual information coming from tables without recognizing actual table structure. However, recognition of table structure is important to get the contextual meaning of the contents. Table structure recognition in heterogeneous documents is challenging due to a variety of table layouts. It becomes harder where no physical rulings are present in a table. This work proposes a novel learning based methodology for the recognition of table contents in heterogeneous document images. Textual contents of documents are classified as table or non-table elements using a pre-trained neural network model. The output of the neural network is further enhanced by applying a contextual post processing on each element to correct the classifications errors if any. The system is trained using a subset of UNLV and UW3 document images and depicted more than 97% accuracy on a test set in detection of table and non-table elements.

Understanding of technical documents, like manuals, is one of the most important steps in automatic reporting and/or troubleshooting of defects. The majority of the relevant information exists in tabular structure. There are some solutions for extracting tabular structures from text. However, it is still a big issue to extract tabular information from images and, on top of that, from complex and nested tables. This paper aims to propose classification and information extraction methods for complex tabular structures in document images. These are hybrid approaches using both image layout and OCRed text. The proposed methods outperform on a real-world technical documents dataset from a German railway company (Deutsche Bahn AG) as compared to other state-of-the-art approaches. As a result, the proposed approaches won the competition held by Deutsche Bahn AG in 2016 against other participating research groups and companies.

This paper presents a novel end-to-end system for table understanding in document images called DeepDeSRT. In particular, the contribution of DeepDeSRT is two-fold. First, it presents a deep learning-based solution for table detection in document images. Secondly, it proposes a novel deep learning-based approach for table structure recognition, i.e. identifying rows, columns, and cell positions in the detected tables. In contrast to existing rule-based methods, which rely on heuristics or additional PDF metadata (like, for example, print instructions, character bounding boxes, or line segments), the presented system is data-driven and does not need any heuristics or metadata to detect as well as to recognize tabular structures in document images. Furthermore, in contrast to most existing table detection and structure recognition methods, which are applicable only to PDFs, DeepDeSRT processes document images, which makes it equally suitable for born-digital PDFs (as they can automatically be converted into images) as well as even harder problems, e.g. scanned documents. To gauge the performance of DeepDeSRT, the system is evaluated on the publicly available ICDAR 2013 table competition dataset containing 67 documents with 238 pages overall. Evaluation results reveal that DeepDeSRT outperforms state-of-the-art methods for table detection and structure recognition and achieves F1-measures of 96.77% and 91.44% for table detection and structure recognition, respectively. Additionally, DeepDeSRT is evaluated on a closed dataset from a real use case of a major European aviation company comprising documents which are highly unlike those in ICDAR 2013. Tested on a randomly selected sample from this dataset, DeepDeSRT achieves high detection accuracy for tables which demonstrates the sound generalization capabilities of our system.

Documents are routinely captured by digital cameras in today's age owing to the availability of high quality cameras in smart phones. However, recognition of camera-captured documents is substantially more challenging as compared to traditional flat bed scanned documents due to the distortions introduced by the cameras. One of the major performancelimiting artifacts is the motion and out-of-focus blur that is often induced in the document during the capturing process. Existing approaches try to detect presence of blur in the document to inform the user for re-capturing the image. This paper reports, for the first time, an Optical Character Recognition (OCR) system that can directly recognize blurred documents on which the stateof-the-art OCR systems are unable to provide usable results. Our presented system is based on the Long Short-Term Memory (LSTM) networks and has shown promising character recognition results on both the motion-blurred and out-of-focus blurred images. One important feature of this work is that the LSTM networks have been applied directly to the gray-scale document images to avoid error-prone binarization of blurred documents. Experiments are conducted on publicly available SmartDoc-QA dataset that contains a wide variety of image blur degradations. Our presented system achieves 12.3% character error rate on the test documents, which is an over three-fold reduction in the error rate (38.9%) of the best-performing contemporary OCR system (ABBYY Fine Reader) on the same data.

Consuming news over online media has witnessed rapid growth in recent years, especially with the increasing popularity of social media. However, the ease and speed with which users can access and share information online facilitated the dissemination of false or unverified information. One way of assessing the credibility of online news stories is by examining the attached images. These images could be fake, manipulated or not belonging to the context of the accompanying news story. Previous attempts to news verification provided the user with a set of related images for manual inspection. In this work, we present a semi-automatic approach to assist news-consumers in instantaneously assessing the credibility of information in hypertext news articles by means of meta-data and feature analysis of images in the articles. In the first phase, we use a hybrid approach including image and text clustering techniques for checking the authenticity of an image. In the second phase, we use a hierarchical feature analysis technique for checking the alteration in an image, where different sets of features, such as edges and SURF, are used. In contrast to recently reported manual news verification, our presented work shows a quantitative measurement on a custom dataset. Results revealed an accuracy of 72.7% for checking the authenticity of attached images with a dataset of 55 articles. Finding alterations in images resulted in an accuracy of 88% for a dataset of 50 images.

Digitizing historical documents is crucial in preserving the literary heritage. With the availability of low cost capturing devices, libraries and institutes all over the world have old literature preserved in the form of scanned documents. However, searching through these scanned images is still a tedious job as one is unable to search through them. Contemporary machine learning approaches have been applied successfully to recognize text in both printed and handwriting form, however, these approaches require a lot of transcribed training data in order to obtain satisfactory performance. Transcribing the documents manually is a laborious and costly task, requiring many man-hours and language-specific expertise. This paper presents a generic iterative training framework to address this issue. The proposed framework is not only applicable to historical documents, but for present-day documents as well, where manually transcribed training data is unavailable. Starting with the minimal information available, the proposed approach iteratively corrects the training and generalization errors. Specifically, we have used a segmentation-based OCR method to train on individual symbols and then use the semi-corrected recognized text lines as the ground-truth data for segmentation-free sequence learning, which learns to correct the errors in the ground-truth by incorporating context-aware processing. The proposed approach is applied to a collection of 15th century Latin documents. The iterative procedure using segmentation-free OCR was able to reduce the initial character error of about 23% (obtained from segmentation-based OCR) to less than 7% in few iterations.

The area of image captioning i.e. the automatic generation of short textual descriptions of images has experienced much progress recently. However, image captioning approaches often only focus on describing the content of the image without any emotional or sentimental dimension which is common in human captions. This paper presents an approach for image captioning designed specifically to incorporate emotions and feelings into the caption generation process. The presented approach consists of a Deep Convolutional Neural Network (CNN) for detecting Adjective Noun Pairs in the image and a novel graphical network architecture called "Concept And Syntax Transition (CAST)" network for generating sentences from these detected concepts.

This paper presents a deep Convolutional Neural Network (CNN) based approach for document image classification. One of the main requirement of deep CNN architecture is that they need huge number of samples for training. To overcome this problem we adopt a deep CNN which is trained using big image dataset containing millions of samples i.e., ImageNet. The proposed work outperforms both the traditional structure similarity methods and the CNN based approaches proposed earlier. The accuracy of the proposed approach with merely 20 images per class outperforms the state-of-the-art by achieving classification accuracy of 68.25%. The best results on Tobbacoo-3428 dataset show that our proposed method outperforms the state-of-the-art method by a significant margin and achieved a median accuracy of 77.6% with 100 samples per class used for training and validation.

Atomic segmentation of cursive scripts into constituent characters is one of the most challenging problems in pattern recognition. To avoid segmentation in cursive script, concrete shapes are considered as recognizable units. Therefore, the objective of this work is to find out the alternate recognizable units in Pashto cursive script. These alternatives are ligatures and primary ligatures. However, we need sound statistical analysis to find the appropriate numbers of ligatures and primary ligatures in Pashto script. In this work, a corpus of 2, 313, 736 Pashto words are extracted from a large scale diversified web sources, and total of 19, 268 unique ligatures have been identified in Pashto cursive script. Analysis shows that only 7000 ligatures represent 91% portion of overall corpus of the Pashto unique words. Similarly, about 7, 681 primary ligatures are also identified which represent the basic shapes of all the ligatures.

Most of the traditional document image classification techniques concentrate on document segmentation and OCR analysis, in spite of so many complexities and limitations involved. Recently, many of the document image classification problems are easily solved just by adapting standard computer vision approaches for natural image retrieval and classification, that are referred as visual appearance based document classification techniques. These approaches have reported better results as compared to the traditional approaches on proprietary datasets. However, so far these approaches are not compared with each other and, despite having potential, they are not evaluated on distorted camera-captured documents, which is one of the challenging requirements in our present commercial document analysis projects. In this paper, we present simple and effective descriptions of different visual appearance based document image classification techniques. We compare their performance on various standard and publicly available datasets, that are differ in degree of image degradations and content variations. We also demonstrate their advantages and limitations. Additionally, we make the implemented versions of these method publicly available to research community for usage and further testing on other domains.

The World Wide Web is arguably the most important source of digital documents nowadays. These documents mainly consist of unstructured and semi-structured data comprising a wealth of information at the disposal of the DAR (Document Analysis and Recognition) community. Contextualization plays an important role in understanding the content of those documents. In this paper, we present an approach to early contextualization of textual data in HTML documents. It combines automatic as well as semiautomatic annotation of named entities with user interaction to support contextualization of the content of digital documents as early as in the authoring stage of their life cycle. We also present the results of an online experimental evaluation involving 120 human test subjects. They show that our approach successfully managed to produce semantically annotated versions of unstructured textual content, which contain reliable contextual information, thus facilitating the task of later document analysis stages.

The growing amounts of multimedia data being made available and shared via the Internet pose an increasing problem for law enforcement to investigate the distribution and possession of child sexual abuse (CSA) media. In this paper we address the automatic detection of CSA material in image and video data by multi-modal feature description. Instead of analyzing hash sums or file names, we propose the content-based analysis on visual and, in case of videos, also audio features. To this end, we apply multiple low level features as well as SentiBank, a novel mid-level representation of visual content. In collaboration with police partners and European cyber crime units, we conducted experiments on several datasets, including real world CSA media. Our quantitative evaluation reveals the challenging nature of child pornography detection, especially in the joint presence of non-illegal pornographic data, rendering skin detection, a popular feature for detecting pornography, less discriminative. Further, the utilization of SentiBank features shows high potential for detection and explainability of such content. Overall, multi-modal feature fusion can achieve an improved detection accuracy, reducing equal error rate from 17% to 10% for images and from 16% to 8% for videos as compared to best single feature performance for the challenging task of classifying CSA content from adult media.

In this paper a novel method for automatic ground truth generation of camera captured document images is proposed. Currently, no dataset is available for camera captured documents. It is very difficult to build these datasets manually, as it is very laborious and costly. The proposed method is fully automatic, allowing building the very large scale (i.e., millions of images) labeled camera captured documents dataset, without any human intervention. Evaluation of samples generated by the proposed approach shows that 99.98% of the images are correctly labeled. Novelty of the proposed approach lies in the use of document image retrieval for automatic labeling, especially for camera captured documents, which contain different distortions specific to camera, e.g., blur, occlusion, perspective distortion, etc.

Traditionally, stamps are considered as a seal of authenticity for documents. For automatic processing and verification, segmentation of stamps from documents is pivotal. Existing methods for stamp extraction mostly employ color and/or shape based techniques, thereby limiting their applicability to only colored and specific shape stamps. In this paper, a novel, generic method based on part-based features is presented for segmentation of stamps from document images. The proposed method can segment black, colored, unseen, arbitrary shaped, textual, as well as graphical stamps. The proposed method is evaluated on a publicly available dataset for stamp detection and verification and achieved recall and precision of 73% and 83% respectively, for black stamps which were not addressed in the past.

Automatically identifying that a certain page in a set of documents is printed with a different printer than the rest of the documents can give an important clue for a possible forgery attempt. Different printers vary in their produced printing quality, which is especially noticeable at the edges of printed characters. In this paper, a system using the difference in edge roughness to distinguish laser printed ages from inkjet printed pages is presented. Several feature extraction methods have been developed and evaluated for that purpose. In contrast to previous work, this system uses unsupervised anomaly detection to detect documents printed by a different printing technique than the majority of the documents among a set. This approach has the advantage that no prior training using genuine documents has to be done. Furthermore, we created a dataset featuring 1200 document images from different domains (invoices, contracts, scientific papers) printed by 7 different inkjet and 13 laser printers. Results show that the presented feature extraction method achieves the best outlier rank score in comparison to state-of-the-art features.

This paper presents a novel signature verification system based on local features of signatures. The proposed system uses Fast Retina Key points (FREAK) which represent local features and are inspired by the human visual system, particularly the retina. To locate local points of interest in signatures, two local key point detectors, i.e., Features from Accelerated Segment Test (FAST) and Speeded-up Robust Features (SURF), have been used and their performance comparison in terms of Equal Error Rate (EER) and time is presented. The proposed system has been evaluated on publicly available dataset of forensic signature verification competition, 4NSigComp2010, which contains genuine, forged, and disguised signatures. The proposed system achieved an EER of 30%, which is considerably very low when compared against all the participants of the said competition. In addition to EER, the proposed system requires only 0.6 seconds on average to verify a 3000*1500 scanned signature. This shows that the proposed system has a potential and suitability for forensic signature verification as well as real time applications.

The purpose of writing this paper is three-fold. First, it presents a novel local / part-based automatic system for forensic signature verification involving disguised signatures. Disguised signatures are written by authentic authors but with the intention of later denial. The proposed system reaches an equal error rate of 3.36% in classifying disguised and genuine signatures. Second, it compares the performance of the proposed system with various state-of-the-art signature verification systems on the same data, i.e., the publicly available dataset of 4NSigComp2010 signature verification competition. Third, it presents a performance comparison of the proposed system with human forensic handwriting examiners. It is important as it highlights the potential of the proposed system to assist humans in solving real world forensic signature verification cases.

With the rise of email communication, enterprises strive to manage incoming documents from all input channels for achieving customer satisfaction. Their overall goal is to reduce request processing time and to increase processing quality. Previously, we proposed the approach of process-driven document analysis (DA) using the concepts of Attentive Tasks (ATs) and the Specialist Board (SB). The ATs formalize information expectations of the processes toward an incoming document, whereas the SB describes all available DA methods. In this paper, we propose to apply continuous partial order planning (CPOP) for guiding DA with the goal of optimal extraction accuracy and runtime. To our knowledge, this approach provides a novel method for integrating knowledge management with DA, in particular for processes. Since planning has not been applied to this field yet, we explore learning the suitability function (SF) and the adaptation of the DA plan. First evaluations indicate the applicability of the approach and preferences for calibration.

We present a new system that assists people's reading activity by combining a wearable eye tracker, a see-through head mounted display, and an image based document retrieval engine. An image based document retrieval engine is used for identification of the reading document, whereas an eye tracker is used to detect which part of the document the reader is currently reading. The reader can refer to the glossary of the latest viewed key word by looking at the see-through head mounted display. This novel document reading assist application, which is the integration of a document retrieval system into an everyday reading scenario for the first time, enriches people's reading life. In this paper, we i) investigate the performance of the state-of-the-art image based document retrieval method using a wearable camera, ii) propose a method for identification of the word the reader is attendant, and iii) conduct pilot studies for evaluation of the system in this reading context. The results show the potential of a document retrieval system in combination with a gaze based user-oriented system.

In this paper we propose a novel part-based method for the extraction of text touching graphic components. The Speeded Up Robust Features (SURF) are used to localize the text components and distinguish them from graphics. We introduce several post-processing steps to finally detect the text. We have tested our method on a publicly available data set of architectural floor plans and on real geographical maps. On floor plans we have located more than 95% of the text components which were not identified as text beforehand because they were touching graphic components.

This paper presents an automatic system for analyzing and labeling architectural floor plans. In order to detect the locations of the rooms, the proposed systems extracts both, structural and semantic information from given floor plans. Furthermore, OCR is applied on the text layer to retrieve the meaningful room labeling. Finally, a novel post-processing is proposed to split rooms into several sub-regions if several semantic rooms share the same physical room. Our fully automatic system is evaluated on a publicly available dataset of architectural floor plans. In our experiments, we could clearly outperform other state-of-the-art approaches for room detection.

smartFIX is a product portfolio for knowledge based extraction of data from any document format. The system automatically determines the document type and extracts all relevant data for the respective business process. Data that is unreliably recognized is forwarded to a verification workplace for manual checking. In general, users have no difficulties to interpret the document data and wonder why the system needs additional input. For that reason, we implemented an explanation component that is used to justify extraction results, thus, increasing confidence of users. The component is using a semantic log making it possible to provide understandable explanations. We illustrate the benefits of that kind of technology in contrast to the current smartFIX Log Viewer by means of a preliminary user experiment.

Recognition of scene text using a hand-held camera is emerging as a hot topic of research. In this paper, we investigate the use of a head-mounted eye-tracker for scene text recognition. An eye-tracker detects the position of the user's gaze. Using gaze information of the user, we can provide the user with more information about his region/object of interest in a ubiquitous manner. Therefore, we can realize a service such as the user gazes at a certain word and soon obtain the related information of the word by combining a word recognition system with eye-tracking technology. Such a service is useful since the user has to do nothing but gazes at interested words. With a view to realize the service, we experimentally evaluate the effectiveness of using the eye-tracker for word recognition. The initial results show the recognition accuracy was around 70% in our word recognition experiment and the average computational time was less than one second per a query image.

In this paper we propose KOIOS++, which automatically processes natural language queries provided by handwritten input. The system integrates several recent achievements in the area of handwriting recognition, natural language processing, information retrieval, and human computer interaction. It uses a knowledge base described by the resource description framework (RDF). Our generic approach first generates a lexicon as background information for the handwritten text recognition. After recognizing a handwritten query, several output hypotheses are sent to a natural language processing system in order to generate a structured query (SPARQL query). Subsequently, the query is applied to the given knowledge base and a result graph visualizes the retrieved information. At all stages, the user can easily adjust the intermediate results if there is any undesired outcome. The system is implemented as a web-service and therefore works for handwritten input on digital paper as well as on input on Pen-enabled interactive surfaces. Furthermore, we build on the generic RDF-representation of semantic knowledge which is also used by the linked open data (LOD) initiative. As such, our system works well in various scenarios. We have implemented prototypes for querying company knowledge bases, the DBPedia1, the DBLP computer science bibliography2, and a knowledge base of the DAS 2012.

This paper proposes a framework for the integration of handwriting recognition into natural user interfaces. As more and more pen-enabled touch displays are available, we make use of the distinction between touch actions and pen actions. Furthermore, we apply a recently introduced mode detection approach to distinguish between handwritten strokes and graphics drawn with the pen. These ideas are implemented in the Touch & Write SDK which can be used for various applications. In order to evaluate the effectiveness of our approach, we have conducted experiments for an annotation scenario. We asked several users to mark and label several objects in videos. We have measured the labeling time when using our novel user interaction system and compared it to the time needed when using common labeling tools. Furthermore, we compare our handwritten input paradigm to other existing systems. It turns out that the annotation is performed much faster when using our method and the user experience is also much better.

In this paper we present a framework for real-time online signature verification scenarios. The proposed framework is based on state-of-the-art feature extraction and Gaussian Mixture Model (GMM) classification. While our signature verification library is generally applicable to any input device using digital pens, we have implemented verification scenarios using the Anoto digital pen. As such our automated signature verification framework becomes an interesting commodity for industry, because the Anoto SDK is easy to apply and the GMM-based classification can be seamlessly integrated. The novelty of this work is the application of our framework that takes real-time online signature verification to every scenario where digital pens may potentially be used. In this paper we describe several scenarios where our framework has been applied, including signatures in financial contracts or ordering processes. We also propose a general approach to integrate the GMM-descriptions into electronic ID-cards in order to also store behavioral biometrics on these cards. In experiments we have measured the performance of the signature verification system when skilled forgeries were present. The interest shown by our partner financial institutions and the results of our initial evaluations indicate that our signature verification framework suits exactly the demands of our clients.

Computational models of visual attention use image features to identify salient locations in an image that are likely to attract human attention. Attention models have been quite effectively used for various object detection tasks. However, their use for scene text detection is under-investigated. As a general observation, scene text often conveys important information and is usually prominent or salient in the scene itself. In this paper, we evaluate four state-of-the-art attention models for their response to scene text. Initial results indicate that saliency maps produced by these attention models can be used for aiding scene text detection algorithms by suppressing non-text regions.

The increasing amount of email data has led many companies to new challenges with their employees now having to deal with information overload while managing multiple communication channels at the same time, e.g., email, mail, and phone. Moreover, emails can contain attachments, i.e., files with additional information. Most existing approaches for reducing email processing time require significant domain specific customization efforts to achieve good performance and lack attachment handling. We aim at providing a more domain independent approach by integrating the process context and using the information expectations of a process to guide the document analysis (DA) schedule for emails and their attachments. We rely on the concepts of Attentive Tasks (ATs) and Specialist Board (SB). ATs are templates that describe all relevant and expected information about a process currently waiting for input. The SB provides a machine readable description of DA methods, so-called specialists, that extract all relevant information for further processes. We present our approach and demonstrate the benefits for a domain specific application, i.e., a financial institution.

This paper proposes a novel complete system for automated floor plan analysis. Besides applying and improving state-of-the-art processing methods, we introduce novel preprocessing methods, e.g., the differentiation between thick, medium, and thin lines and the removal of components outside the convex hull of the outer walls. Especially the latter method increases the performance of the final system. In our experiments on a reference data set we compare our approach to other approaches available in the literature. We show that our system outperforms previous systems. The final room recognition accuracy is 79% that is 10% higher than the 69% achieved by a state-of-the-art approach from the literature.

In this paper, we propose an improved method for text/graphics segmentation. Text/graphics separation is a crucial preprocessing step in document analysis before further analysis and recognition can be applied. Our proposed system extends the method of Tombre et al. with a number of improvements to make it more suitable for architectural floor plans. A crucial novel preprocessing step is the detection and removal of walls before the actual segmentation. Furthermore, text components are then extracted by analyzing connected components and even considering text overlapping with graphics. Finally, a smearing approach is used to remove noise and extract the final text components. Evaluation results over the series of 90 floor plans which has also been used in reference work shows that our method has a recall of almost 99% and a precision greater then 97%.

smartFIX is a product portfolio for knowledge-based extraction of data from any document format. smartFIX automatically determines the document type and extracts all relevant data for the respective business process. Data that is uncertainly recognized is forwarded to a verification workplace for manual checking. In general, users have no difficulties to interpret the document data and wonder why the system needs additional input. For that reason, we will integrate an explanation component that will be used to justify uncertain extraction results, thus, increasing confidence of users. The component will be based on semantic technologies in general and on a semantic log in particular. The log will contain all process relevant information enabling the explanation facility to generate customized and understandable explanations. In this paper, we will discuss the benefits of that kind of technology with reference to DAS.

Time-stamps and URLs overlaid artificially on images add useful meta information which can be used for automatic indexing of images and videos. In this paper, we propose a method based on an attention-based model of visual saliency to extract overlaid text and time-stamps that are rendered on images. Our model of visual saliency is based on a Bayesian framework and works very well for the task of time-stamp detection and segmentation as is evident by overall object recall of 80% and precision of 70%. Our method produces a clean text segmented binarized image, which can be used for recognition directly by an OCR system. Furthermore, our technique is robust against variation of font styles and color of time-stamp and overlaid text.

Recognition of text in natural scene images is becoming a prominent research area due to the widespread availablity of imaging devices in low-cost consumer products like mobile phones. To evaluate the performance of recent algorithms in detecting and recognizing text from complex images, the ICDAR 2011 Robust Reading Competition was organized. Challenge 2 of the competition dealt specifically with detecting/recognizing text in natural scene images. This paper presents an overview of the approaches that the participants used, the evaluation measure, and the dataset used in the Challenge 2 of the contest. We also report the performance of all participating methods for text localization and word recognition tasks and compare their results using standard methods of area precision/recall and edit distance.

This paper proposes a new approach for drawing mode detection in online handwriting. The system classifies groups of ink traces into several categories. The main contributions of this work are as follows. First, we improve and optimize several state-of-the-art recognizers by adding new features and applying feature selections. Second, we use several classifiers for the recognition. Third, we perform multiple classifier combination strategies for combining the outputs. Finally, a large experimental evaluation on two data sets is performed: the publicly available Touch&Write database which has been acquired on a pen-enabled multi-touch surface, and the publicly available IAMonDo-database which serves as a benchmark. In our experiments on the IAM-OnDo-database we achieved a recognition rate of 97%, which is much higher than other results reported in the literature. On the more balanced multi-touch surface data set we achieved a recognition rate of close to 98%.

Whenever the question arises how a product, a personality, a technology or some other specific entity is perceived by the public, the blogosphere is a very good source of information. This is what usually interests business users from marketing or PR. Modern search services offer a rich set of tools to monitor or track the blogosphere as a whole, but the analysis with respect to a certain domain is very limited. In this paper we lay some foundations to aggregate blog articles of a specific domain from multiple search services, to analyse the social authorities of articles and blogs, and to monitor the attention articles of the domain receive over time. These are the building blocks required for a monitoring application that presents users the currently most interesting articles. This methodology can be instantiated and combined with additional textual analysis methods to create highly automated business intelligence applications.

In the growing mobile computing sector two trends are more and more wide-spread and gain further momentum: location-sensing technologies and mobile Internet access. In this paper we describe Crumblr, an application for semi-automatic capturing, aggregating and sharing spatial content in mobile environments. While existing Web 2.0 services focus mostly on points of interest or on routes, our application combines these two entities and additionally enriches them with contextual data. The result is the generation of an implicit network among people, linked via spatial information. This allows us to provide them with personalized recommendations of places, routes and related other people.

This paper proposes a first attempt to include real semantic information into the process of handwriting recognition. We take advantage of the fact that the main topic of handwritten notes is often known beforehand like in annotation or reviewing tasks. Using state-of-the-art technologies from the knowledge management research area it is possible to store a semantic representation of the user's knowledge in a Personal Information Model (PIMO). This PIMO stores the relations between semantic concepts and documents on the computer. In this paper we extract texts from related documents and concepts of the PIMO. The vocabulary of these texts is then used to aid the recognizer. In our multi-writer experiments, a significant improvement of the recognition accuracy by 8% on the text line level has been achieved.

In this paper we present a novel rear-projection tabletop called Touch & Write. It combines the FTIR technology for touching with the Anoto-technology for handwriting. This allows an implicit switch between the modes object manipulation, and content editing. Our system incorporates real-time gesture and handwriting recognition. Drawn objects and written concepts can be converted to digital information immediately. We introduce a functional application, the LeCoOnt concept mapping software makes use of the full capability of the Touch & Write table. Touching actions are used for arranging the concepts like sheets on a normal table, and to recognizes guestures like zooming. Pen-actions are used for drawing, connecting concepts, and handwriting. The handwritten strokes are automatically recognized and converted into a machine-readable string. This system provides a reliable alternative to common approaches which try to reconstruct the information from photographs.

Table spotting and structural analysis are just a small fraction of tasks relevant when speaking of table analysis. Today, quite a large number of different approaches facing these tasks have been described in literature or are available as part of commercial OCR systems that claim to deal with tables on the scanned documents and to treat them accordingly.

This paper deals with the transfer of knowledge on invoice document layout and extraction strategies, collected by users of the invoice recognition software smartFIX over several years of productive use, to other user's systems. The results of a project analyzing this 'treasure' of knowledge and putting it to use in the smartFIX system are presented. The evaluation shows that this transfer of knowledge using state-of-the-art techniques in transfer learning achieves significantly higher initial recognition rates than the unaugmented system, delivering instant economic advantages by reducing accountant personnel workload.

We examine two basic sources for implicit relevance feedback on the segment level for search personalization: eye tracking and display time. A controlled study has been conducted where 32 participants had to view documents in front of an eye tracker, query a search engine, and give explicit relevance ratings for the results. We examined the performance of the basic implicit feedback methods with respect to improved ranking and compared their performance to a pseudo relevance feedback baseline on the segment level and the original ranking of a Web search engine.

We describe an approach for creating precise personalized document classifiers based on the user's attention. The general idea is to observe which parts of a document the user was interested in just before he or she comes to a classification decision. Having information about this manual classification decision and the document parts the decision was based on, we can learn precise classifiers. For observing the user's focus point of attention we use an unobtrusive eye tracking device and apply an algorithm for reading behavior detection. On this basis, we can extract terms characterizing the text parts interesting to the user and employ them for describing the class the document was assigned to by the user. Having learned classifiers in that way, new documents can be classified automatically using techniques of passage-based retrieval. We prove the very strong improvement of incorporating the user's visual attention by a case study that evaluates an attention-based term extraction method.

Today, printing and reverse printing (scanning, OCR, logical labeling etc.) technologies have become quite mature and thus allow for an easy transition of documents between physical and electronic world. However, there is no technology today which supports the lossless interpretation of paper-based user interaction with direct effects upon the electronic representation of that document. The HyperPrinting environment tries to fill in this gap and thus accounts for the personal favors of a majority of office workers: Not only managers and knowledge workers prefer to read longer documents, articles or news from paper in contrast to a computer monitor or handheld computer. With the help of HyperPrinting, users can annotate, send notes or initiate tasks and it thus offers a completely new paradigm in the usage and treatment of paper documents. As a side-effect, the use of HyperPrinting builds up a document repository which is not only searchable by full text but also by meta-information, which in turn is depending on the selected user scenario.

We examine the effect of incorporating gaze-based attention feedback from the user on personalizing the search process. Employing eye tracking data, we keep track of document parts the user read in some way. We use this information on the subdocument level as implicit feedback for query expansion and reranking.

Automatic processing of office documents, such as orders, invoices, or offers entails a significant potential for saving costs. Because such domains have a high percentage of special vocabulary, purely statistical approaches fail in automatic classification. The inherent structure and short text messages require specific approaches. We propose a rule-based method to classify mixed stacks of documents into a set of hierarchically organized classes. Rules are learned by extracting patterns of different types from a document sample. The paper focuses on the architecture and on the learning process, presents comparing results to other techniques, and gives an outlook on how to further improve the system.

Knowledge management approaches for weakly-structured, adhoc knowledge work processes need to be lightweight, i.e., they cannot rely on high upfront modeling efforts. This paper presents TaskNavigator, a novel prototype to support weakly-structured processes by integrating a standard task list application with a state-of-the-art document classification system. The resulting system allows for a task-oriented view on office workers' personal knowledge spaces in order to realize a proactive and contextsensitive information support during daily, knowledge-intensive tasks. Moreover, TaskNavigator supports process know-how reuse by proactively suggesting similar tasks or relevant process models, based on textual similarities. Finally, we report on a feasibility test and a case study that have been conducted in order to evaluate the system in the context of daily research task management and software requirements analysis.

After we developed a model free table recognition system we had the desire to automatically register the effect of minor changes to parameters upon the overall performance quality of our system in order to tune parameters. Therefore we developed a complete benchmarking environment, containing a user front-end to acquire ground truth data as well as mechanisms to evaluate the quality of the recognition results. The tasks involved in the analysis systems were the locating of table regions, identification of cells and mapping of cells to rows and columns. This paper presents our approach towards the comparison of recognition results with the ground truth. The established definitions of recall and precision did not meet our requirements, as we wanted to register even smallest improvements (or changes in general) in the results, even when both results were imperfect. We therefore extended the measures recall and precision in order to deal with recognition probabilities of objects rather than just with Boolean values.

The decomposition of a document into segments such as text regions and graphics is a significant part of the document analysis process. The basic requirement for rating and improvement of page segmentation algorithms is systematic evaluation. The approaches known from the literature have the disadvantage that manually generated reference data (zoning ground truth) are needed for the evaluation task. The effort and cost of the creation of these data are very high. This paper describes the evaluation system SEE and presents an assessment of its quality. The system requires the OCR generated text and the original text of the document in correct reading order (text ground truth) as input. No manually generated zoning ground truth is needed. The implicit structure information that is contained in the text ground truth is used for the evaluation of the automatic zoning. Therefore, an assignment of the corresponding text regions in the text ground truth and those in the OCR generated text (matches) is sought. A fault tolerant string matching algorithm underlies a method, able to tolerate OCR errors in the text. The segmentation errors are determined as a result of the evaluation of the matching. Subsequently, the edit operations which are necessary for the correction of the recognized segmentation errors are computed to estimate the correction costs. Furthermore, SEE provides a version of the OCR generated text, which is corrected from the detected page segmentation errors.

In this paper I will try to explain the nature of documentunderstanding in all of its dimensions. Therefore I willfirst describe the characteristics of data, knowledge, andinformation in order to describe their synergetic inter-weaving.After that I will try to structure the inherentcomplexity of sub-problems of document understandingwhich may not be solved serially, but rather are attributesof individual documents. Thus, this paper focuses onsystem engineering challenges. However, I will showsome recent work done on the different topics and givesome insights in the individual techniques we chose atDFKI.

This paper summarizes the core idea of the T-Recs table recognition system, an integrated system covering block-segmentation, table location and a model-free structural analysis of tables. T-Recs works on the output of commercial OCR systems that provide the word bounding box geometry together with the text itself (e.g. Xerox ScanWorX). While T-Recs performs well on a number of document categories, business letters still remained a challenging domain because the T-Recs location heuristics are mislead by their header or footer resulting in a low recognition precision. Business letters such as invoices are a very interesting domain for industrial applications due to the large amount of documents to be analyzed and the importance of the data carried within their tables. Hence, we developed a more restrictive approach which is implemented in the T-Recs++ prototype. This paper describes the ideas of the T-Recs++ location and also proposes a quality evaluation measure that reflects the bottom-up strategy of either T-Recs or T-Recs++. Finally, some results comparing both systems on a collection of business letters are given.

Retrieval of electronic documents is a fundamental component for intelligent access to the contents of documents. For the retrieval of long documents, a method called passage-based document retrieval has proven to be effective. In this paper we experimentally show that the passage-based retrieval is also advantageous for dealing with short queries on condition that documents are long. We employ a passage-based method based on density distributions of query terms in documents, and compare it with three conventional methods: the vector space model, pseudo-feedback and latent semantic indexing.

This paper introduces three approaches for an industrial, comprehensive document analysis system to enable it to spot tables in documents. Searching for a set of known table headers (approach 1) works rather well in a significant number of documents. But this approach (though it is implemented tolerant to OCR errors) is not tolerant enough towards some kinds of even minor aberrations. This not only decreases the recognition results, but also, even worse, makes users feel uncomfortable. Pragmatically trying to mimic for what the human eyes might key, leads to our two further, complementary approaches: searching for layout structures which resemble parts of columns (approach 2), and searching for groupings of similar lines (approach 3). The suitability of the approaches for our system requires them to be very simple to implement and simple to explain to users, computationally cheap, and combinable. In the domain of health insurances who receive huge amounts of so called medical liquidations on a daily basis we obtain very good results. On document samples representative for the every day practice of five customers-health insurance companies-tables were spotted as good and as fast as the customers expected the system to be. We thus consider our current approaches as a step towards cognitive adequacy.

In document analysis, it is common to prove the usefulness of a component by an experimental evaluation. By applying the respective algorithms to a test sample, effectiveness measures such as recall, precision, and accuracy are computed. The goal of such an evaluation is two-fold: on the one hand it shows that the absolute effectiveness of the algorithm is acceptable for practical use. On the other hand the evaluation can prove that the algorithm has a better or worse effectiveness than another algorithm. We argue that the experimental evaluation on relative small test sets-as is very common in document analysis has to be taken with extreme care from a statistical point of view. In fact, it is surprising how weak statements derived from such evaluations are.

Summary form only given, as follows. Increasing the performance of document analysis systems requires a detailed quality evaluation of the achieved results. By focussing on segmentation algorithms, we point out that the results produced from the module under consideration should be evaluated directly; we show that the text based evaluation method which is often used in the document analysis domain is not sufficient for the purpose of a detailed quality evaluation of the segmentation module. Therefore, we propose a general evaluation approach for comparing segmentation results which is based on the segments directly. This approach is able to handle both algorithms which produce complete segmentations (partition) and algorithms which only extract objects of interest (extraction). Classes of errors are defined in a systematic way and frequencies for each class can be computed. The evaluation approach is applicable to segmentation or extraction algorithms in a wide range. We have chosen the character segmentation task as an example to demonstrate the applicability of our evaluation approach and we suggest applying our approach to other segmentation tasks.

The task to be solved within our core research was the design and development of a document analysis toolbox covering typical document analysis tasks such as document understanding, information extraction and text recognition. In order to prove the feasibility of our concepts, we have developed the prototypical analysis system OfficeMAID (Office Mail Analysis, Interpretation and Delivery). The system analyses documents, as used in the daily work of a purchasing department, by a priori knowledge about workflows and document features. In this way, the system provides goal-directed information extraction, shallow understanding and process identification for given documents (paper, fax, e-mail).

We describe a system which is capable of learning the presentation of document logical structures, exemplarily shown for business letters. Presenting a set of instances to the system, it clusters them into structural concepts and induces a concept hierarchy. This concept hierarchy is taken as a source for classifying future input. The paper introduces the different learning steps, describes how the resulting concept hierarchy is applied for logical labeling and reports on the results.

The indexing of inaccurately recognized OCR text yields unsatisfactory results, where the quality of the index terms decreases rapidly when the quality of the documents get worse. Index terms of OCR processed documents can be used for archiving or classification tasks. We present an indexing component whose input are character hypothesis lattices which are post-processed by a generate-and-test component feeding a morphology, a rule based substitution system, and a trigram correction component with word candidates. Stop words are filtered by a Levenshtein-based elimination routine. The recognized words are subsequently processed by our indexing component. Our system minimizes the number of generated index terms which are correct German words. The experiments have shown an increase in accuracy of next to 10%.

Proposes an approach for automatically generating a decision tree which is applied as a model for the logical labeling of business letters. Instead of top-down determination of the discriminating attributes, the system inspects a finite set of document instances that are presented to a learner in a bottom-up position. The learner itself figures out local similarities, rates them with respect to the overall structure, and determines the best structural match of two instances (neighborhood). The entire decision tree is grown step by step deducing subtrees by forming generalizations from a neighborhood. Consequently, heuristics are learned for structurally discriminating documents during subsequent classification.>

One obstacle for introducing a general and homogeneous information system is the lack of possibilities for combining the information processing capabilities of the electronic medium and traditional ways in which information is generated, interchanged, categorized, processed or stored. In this context, document analysis and understanding technology seems to be the key to overcome this problem. An overview of efforts focusing on the integration of document analysis and understanding techniques into active interfaces for multimedia information systems is given. Moreover, the role and the potential of this technology are outlined using sample application scenarios.>