Baseline detection is a simplified text-line extraction that typically serves as pre-processing for Automated Text Recognition. The cBAD competition benchmarks state-of-the-art baseline detection algorithms. It is the successor of cBAD 2017 with a larger dataset that contains more diverse document pages. The images together with the manually annotated groundtruth are made publicly available which allows other teams to benchmark and compare their methods. We could also evaluate the winning method of cBAD 2017 on the newly introduced dataset which now serves as baseline. This competition shows that the performance of automated baseline detection increased substantially since 2017.

This work is concerned with the binarization of ancient manuscripts that have been imaged with a MultiSpectral Imaging (MSI) system. We introduce a new dataset for this purpose that is composed of 130 multispectral images taken from two medieval manuscripts. We propose to apply an end-to-end Convolutional Neural Network (CNN) for the segmentation of the historical writings. The performance of the CNN based method is superior compared to two state-of-the-art methods that are especially designed for multispectral document images. The CNN based method is also evaluated on a previous and smaller database, where its performance is slightly worse than the two state-of-the-art techniques.

MultiSpectral (MS) imaging enriches document digitization by increasing the spectral resolution. We present a methodology which detects a target ink in document images by taking into account this additional information. The proposed method performs a rough foreground estimation to localize possible ink regions. Then, the Adaptive Coherence Estimator (ACE), a target detection algorithm, transforms the MS input space into a single gray-scale image where values close to one indicate ink. A spatial segmentation using GrabCut on the target detection's output is computed to create the final binary image. To find a baseline performance, the method is evaluated on the three most recent Document Image Binarization COntests (DIBCO) despite the fact that they only provide RGB images. In addition, an evaluation on three publicly available MS datasets is carried out. The presented methodology achieved the highest performance at the MultiSpectral Text Extraction (MS-TEx) contest 2015.

This work is concerned with the digitization and analysis of historical documents. The investigation of the documents has been conducted in three successive interdisciplinary projects. The team involved in the projects consists of philologists, chemists and computer scientists specialized in the field of digital image processing. The manuscripts investigated are partially degraded since they have been infected by mold, are corrupted by background clutter or contain faded-out or even erased writings. Since these degradations impede a transcription by scholars and worsen the performance of automated document image analysis techniques, the documents have been imaged with a portable multispectral imaging system. By using this non-invasive investigation technique, the contrast of the faded out characters can be increased, compared to ordinary white light illumination. Post-processing techniques, such as dimension reduction tools, can be used to gain a further legibility increase. The resulting images are used as a basis for further document analysis methods. These methods have been especially designed for the historical documents investigated and involve Optical Character Recognition and writer identification. This paper presents an overview on selected methods that have been developed in the projects.

Text recognition in natural scene images is an application for several computer vision applications like licence plate recognition, automated translation of street signs, help for visually impaired people or image retrieval. In this work an end-to-end text recognition system is presented. For detection an AdaBoost ensemble with a modified Local Ternary Pattern (LTP) feature-set with a post-processing stage build upon Maximally Stable Extremely Region (MSER) is used. The text recognition is done using a deep Convolution Neural Network (CNN) trained with backpropagation. The system presented outperforms state of the art methods on the ICDAR 2003 dataset in the text-detection (F-Score: 74.2%), dictionary-driven cropped-word recognition (F-Score: 87.1%) and dictionary-driven end-to-end recognition (F-Score: 72.6%) tasks.

A ruling classification is presented in this paper. In contrast to state-of-the-art methods which focus on ruling line removal, ruling lines are analyzed for document clustering in the context of document snippet reassembling. First, a background patch is extracted from a snippet at a position which minimizes the inscribed content. A novel Fourier feature is then computed on the image patch. The classification into void, lined and checked is carried out using Support Vector Machines. Finally, an accurate line localization is performed by means of projection profiles and robust line fitting. The ruling classification achieves an F-score of 0.987 evaluated on a dataset comprising real world document snippets. In addition the line removal was evaluated on a synthetically generated dataset where an F-score of 0.931 is achieved. This dataset is made publicly available so as to allow for benchmarking.

This paper presents the results of the HDRC 2013 competition for recognition of handwritten digits organized in conjunction with ICDAR 2013. The general objective of this competition is to identify, evaluate and compare recent developments in character recognition and to introduce a new challenging dataset for benchmarking. We describe competition details including dataset and evaluation measures used, and give a comparative performance analysis of the nine (9) submitted methods along with a short description of the respective methodologies.

Text line detection is a pre-processing step for automated document analysis such as word spotting or OCR. It is additionally used for document structure analysis or layout analysis. Considering mixed layouts, degraded documents and handwritten documents, text line detection is still challenging. We present a novel approach that targets torn documents having varying layouts and writing. The proposed method is a bottom up approach that fuses words, to globally minimize their fusing distance. In order to improve processing time and further layout analysis, text lines are represented by oriented rectangles. Even though, the method was designed for modern handwritten and printed documents, tests on medieval manuscripts give promising results. Additionally, the text line detection was evaluated on the ICDAR 2009 and ICFHR 2010 Handwriting Segmentation Contest datasets.

In this paper a method for writer identification and writer retrieval is presented. Writer identification is the task of identifying the writer of a document out of a database of known writers. In contrast to identification, writer retrieval is the task of finding documents in a database according to the similarity of handwritings. The approach presented in this paper uses local features for this task. First a vocabulary is calculated by clustering features using a Gaussian Mixture Model and applying the Fisher kernel. For each document image the features are calculated and the Fisher Vector is generated using the vocabulary. The distance of this vector is then used as similarity measurement for the handwriting and can be used for writer identification and writer retrieval. The proposed method is evaluated on two datasets, namely the ICDAR 2011 Writer Identification Contest dataset which consists of 208 documents from 26 writers, and the CVL Database which contains 1539 documents from 309 writers. Experiments show that the proposed methods performs slightly better than previously presented writer identification approaches.

This work aims at enhancing ancient and degraded writings, which are captured by MultiSpectral Imaging systems. The manuscripts captured, contain faded out characters and are partly corrupted by mold and hardly legible. Several works have shown that such writings can be enhanced by applying unsupervised dimension reduction tools - like Principal Component Analysis (PCA) or Independent Component Analysis (ICA). In this work the Fisher Linear Discriminate Analysis (LDA) is applied in order to reduce the dimension of the multispectral scan and to enhance the degraded writings. Since Fisher LDA is a supervised dimension reduction tool, it is necessary to label a subset of multispectral data. For this purpose, a semi-automated label generation step is conducted, which is based on an automated detection of text lines. Thus, the approach is not only based on spectral information - like PCA and ICA - but also on spatial information. The method has been tested on two Slavonic manuscripts. A qualitative analysis shows, that the LDA based dimension reduction gains better performance, compared to unsupervised techniques.

In this paper a public database for writer retrieval, writer identification and word spotting is presented. The CVL-Database consists of 7 different handwritten texts (1 German and 6 English Texts) and 311 different writers. For each text an RGB color image (300 dpi) comprising the handwritten text and the printed text sample are available as well as a cropped version (only handwritten). A unique ID identifies the writer, whereas the bounding boxes for each single word are stored in an XML file. An evaluation of the best algorithms of the ICDAR and ICHFR writer identification contest has been performed on the CVL-database.

Document analysis is done to analyze entire forms (e.g. intelligent form analysis, table detection) or to describe the layout/structure of a document for further processing. A pre-processing step of document analysis methods is a skew estimation of scanned or photographed documents. Current skew estimation methods require the existence of large text areas, are dependent on the text type and can be limited on a specific angle range. The proposed method is gradient based in combination with a Focused Nearest Neighbor Clustering of interest points and has no limitations regarding the detectable angle range. The upside/down decision is based on statistical analysis of ascenders and descenders. It can be applied to entire documents as well as to document fragments containing only a few words. Results show that the proposed skew estimation is comparable with state-of-the-art methods and outperforms them on a real dataset consisting of 658 snippets.

Writer identification determines the writer of one document among a number of known writers where at least one sample is known. Writer retrieval searches all documents of one particular writer by creating a ranking of the similarity of the handwriting in a dataset. This paper presents a method for writer retrieval and writer identification using local features and therefore the proposed method is not dependent on a binarization step. First the local features of the image are calculated and with the help of a predefined codebook an occurrence histogram can be created. This histogram is compared to determine the identity of the writer or the similarity of other handwritten documents. The proposed method has been evaluated on two datasets, namely the IAM dataset which contains 650 writers and the Trigraph Slant dataset which contains 47 writers. Experiments have shown that it can keep up with previous writer identification approaches. Regarding writer retrieval it outperforms previous methods.

Segmenting page images into text lines is a crucial pre-processing step for automated reading of historical documents. Challenging issues in this open research field are given \eg by paper or parchment background noise, ink bleed-through, artifacts due to aging, stains, and touching text lines. In this paper, we present a novel binarization-free line segmentation method that is robust to noise and copes with overlapping and touching text lines. First, interest points representing parts of characters are extracted from gray-scale images. Next, word clusters are identified in high-density regions and touching components such as ascenders and descenders are separated using seam carving. Finally, text lines are generated by concatenating neighboring word clusters, where neighborhood is defined by the prevailing orientation of the words in the document. An experimental evaluation on the Latin manuscript images of the Saint Gall database shows promising results for real-world applications in terms of both accuracy and efficiency.

In general document image analysis methods are pre-processing steps for Optical Character Recognition (OCR) systems. In contrast, the proposed method aims at clustering document snippets, so that an automated clustering of documents can be performed. Therefore, words are classified according to printed text, manuscripts, and noise. Where, the third class corrects falsely segmented background elements. Having classified text elements, a layout analysis is carried out which groups words into text lines and paragraphs. A back propagation of the class weights - assigned to each word in the first step - enables correcting wrong class labels. The proposed method shows promising results on a dataset consisting of document snippets with varying shapes, content writing and layout. In addition, the system is compared to page segmentation methods of the ICDAR 2009 Page Segmentation Competition.

We propose a layout analysis method for historical manuscripts that relies on the part-based identification of layout entities. A layout entity -- such as letters of the text, initials or headings -- is composed of a set of characteristic segments or structures, which is dissimilar for distinct classes in the manuscripts under consideration. This fact is exploited in order to segment a manuscript page into homogeneous regions. Historical documents traditionally involve challenges such as uneven writing support and varying shapes of characters, fluctuating text lines, changing scripts and writing styles, and variance in the layout itself. Hence, a part-based detection of layout entities is proposed using a multi-stage algorithm for the localization of the entities, based on interest points. Results show that the proposed method is able to locate initials, headings and text areas in ancient manuscripts containing stains, tears and partially faded-out ink sufficiently well.

The proposed binarization algorithm uses a scale space to avoid the estimation of script size dependent parameters. Due to the continous smoothing from finer to coarse scales, noise such as background clutter is suppressed since coarse scales characterize homogeneous regions of the image. Thus, coarser scales of the scale space can be used as a foreground estimation to apply a weigthing scheme robust against noise present in, for instance carbon copies or ancient and degraded documents. Additionally the information of filled regions is propagated through the scales. The use of integral images for the calculation of the mean, standard deviation and morphological operations allow for an efficient implementation of the method presented. The binarization of each scale is based on changes of the local intensity as proposed by Su et al.

Document analysis is done to analyze entire forms (e.g. intelligent form analysis, table detection) or to describe the layout/structure of a document. In this paper document analysis is applied to snippets of torn documents to calculate features that can be used for reconstruction. The main intention is to handle snippets of varying size and different contents (e.g. handwritten or printed text). Documents can either be destroyed by the intention to make the printed content unavailable (e.g. business crime) or due to time induced degeneration of ancient documents (e.g. bad storage conditions). Current reconstruction methods for manually torn documents deal with the shape, or e.g. inpainting and texture synthesis techniques. In this paper the potential of document analysis techniques of snippets to support a reconstruction algorithm by considering additional features is shown. This implies a rotational analysis, a color analysis, a line detection, a paper type analysis (checked, lined, blank) and a classification of the text (printed or hand written). Preliminary results show that these features can be determined reliably on a real dataset consisting of 690 snippets.

Multi-spectral imaging for the analysis and preservation of ancient documents has gained high attention in recent years. While readability enhancement is based on the multi-spectral image corpus, foreground-background separation still relies mainly on gray level or color images. In this paper we propose a foreground-background separation algorithm designed for multi-spectral images. The main contribution is the simultaneously utilization of spectral and spatial features. While spectral features incorporate the spectral components of the multi-spectral images, the spatial features are based on stroke properties. Higher order Markov Random Fields enables an efficient way to combine both features. To solve higher order energy functions, we introduce a new message update rule in the well known belief propagation algorithm based on a higher order potential function.

The main problems of Optical Character Recognition (OCR) systems are solved if printed latin text is considered. Since OCR systems are based upon binary images, their results are poor if the text is degraded. In this paper a codex consisting of ancient manuscripts is investigated. Due to environmental effects the characters of the analyzed codex are washed out which leads to poor results gained by state of the art binarization methods. Hence, a segmentation free approach based on local descriptors is being developed. Regarding local information allows for recognizing characters that are only partially visible. In order to recognize a character the local descriptors are initially classified with a Support Vector Machine (SVM) and then identified by a voting scheme of neighboring local descriptors. State of the art local descriptor systems are evaluated in this paper in order to compare their performance for the recognition of degraded characters.

An automated assembling of shredded/torn documents (2D) or broken pottery (3D) will support philologists, archaeologists and forensic experts. An automated solution for this task can be divided into shape based matching techniques (apictorial) or techniques that analyze additionally the visual content of the fragments (pictorial). In the case of visual content techniques like texture based analysis are used. Depending on the application, shape matching techniques are suitable for entities of the puzzle problem with small numbers of pieces (e.g. up to 20). Also artefacts like broken and lost pieces or overlapping parts of fragments increase the error rate of shape based techniques since the matching of adjacent boundaries can fail. As a result additional features, e.g. color, document structure, have to be used. This paper presents an overview about current puzzle applications in Cultural Heritage, and introduces also the main problems in puzzle solving.

In this paper we propose a character segmentation method for multispectral images of ancient documents. Due to the low quality of the images the main idea of this study is to combine the multispectral behavior and contextual spatial information. Therefore we utilize a Markov random field model using the spectral information of the images and stroke properties to include spatial dependencies of the characters. Since the stroke properties and the Gaussian parameters for the imaging model are evaluated automatically the proposed segmentation method requires no training phase. We compared the method to state of the art character segmentation methods and demonstrate the effectiveness of combining spectral and spatial features for the segmentation of characters in multispectral images.

This paper deals with the enhancement of the readability in historic texts written on parchment. Due to mold, air, humidity, water, etc. parchment and text are partially damaged and consequently hard to read. In order to enhance the readability of the text, the manuscript pages are imaged in different spectral bands ranging from 360 to 1000 nm. The readability enhancement is based on a spectral and spatial analysis of the multivariate image data by multivariate spatial correlation. The main advantage of the method is that especially the text regions are enhanced which is provided by generating a mask image. This mask is based on the automatic reconstruction of the ruling scheme of the text pages. The method is tested on two medieval Slavonic manuscripts written on parchment.