Processing automatically historical document images to allow the search of textual information requires the preparation of ground-truth data for training and evaluation. This process is an expensive and arduous task, especially when the historical document images contain specialized vocabulary and/or tabular information. In the latter case, relevant decisions have to be taken to annotate the tabular parts. This paper presents a complex collection of historical document images and the resulting database, which is called HisClima. In this database, half of the images are in tabular format and half as running text. Both types of images contain pre-printed and handwritten text. The textual information is plenty of abbreviations and specific vocabulary related to weather conditions and old ships. This database can be used to research technologies related to historical document image processing and analysis, both for tabular and running text recognition. Baseline results are presented for Document Layout Analysis, Text Recognition, and Probabilistic Indexing. Although these results are good, there is still room for improvement and some indications are provided in this direction.

Document Image Understanding is a demanding Pattern Recognition problem that requires complex recognition models. This problem is even more difficult for document images with complicated layouts like tables, where the reading order is often intrinsically ambiguous, and consequently, the context is generally ambiguous as well. In this paper, we compare two machine learning approaches for extracting information in pre-printed historical tables with handwritten information. We analyze the performance of each approach at each step of the extraction process over different corpora, up to a realistic scenario where documents with different table layouts written by different hands are used. The results are good in general and show that a model based on Multilayer Perceptrons yields better results on more homogeneous documents, while another model based on Graph Neural Networks generalizes better on heterogeneous corpora.

The measurement, monitoring, modeling and forecasting of atmospheric phenomena such as temperature, precipitation, humidity, speed, wind direction, among others, requires the installation and maintenance of a network of meteorological stations for the development of this activity. Depending on its purposes and scope, this network will be more or less sophisticated in the capture and transmission of information. The present work proposes to formulate a methodology to establish the minimum network of meteorological stations necessary for the recording of temperature, as well as the thermal zones of the Colombian territory as a function of altitude. For this purpose, first, the Dynamic Time Warping (DTW) algorithm was used as a technique to calculate the similarity between time series. From the results obtained with DTW, a hierarchical grouping was developed to determine the station clusters. Finally, the thermal zones, 10 in total as a result of the clustering, were the result of selecting those stations that were within the interquartile range with respect to the altitude coordinate. From an initial network of 452 stations, an optimal network of 230 stations was arrived at. Daily historical temperature records from the network of meteorological stations managed by the Institute of Environmental Studies (IDEAM) were the input with which this methodology was implemented.

This work is framed in the domain of software engineering. Specifically, it is situated in the subdomain of user interface evaluation. The context of the same comprises the phenomenon of visual attention and its evaluation through indicators that allow evaluating the quality of these interfaces. Specifically, it presents a model for the prediction of visual attention based on saliency maps, machine learning and biometric data. Its objective is to serve as a support to promote the usability of user interfaces. Experiments carried out with the eye tracker by the Institute for Cognitive Sciences at the University of Osnabrück and the University Medical Center in Hamburg-Eppendorf, among which free visualization tasks on user interfaces such as web pages, formed the input with which the model was developed. Its general structure consists of two elements: a convolutional neural network and Guided Grad-CAM (a convolutional layer visualization method). Biometric components were used to train the network: images whose size was set as a function of the foveal radius and the user's distance from the interface. The natural units of information (nats) were used as a measure to evaluate the accuracy of the model.

Transcription of handwritten text can be speed-up by using off-line Handwritten Text Recognition techniques, that allow the obtention of an initial draft transcription of an image with handwritten text. However, this draft transcription usually contains errors that must be amended by the transcriber by providing a feedback signal. The usual approach is post-edition, where each error is corrected without modifying the rest of the current transcription. A more sophisticated approach can employ the current modification to provide a new whole transcription, hopefully with less errors. Apart from that, feedback can be provided in different modalities: keyboard input, on-line handwritten text, or speech. Each of these modalities presents different features with respect to ambiguity, derived errors, and final transcription time. In this work we study how the different modalities behave in the assisted transcription of a historical handwritten text document in Spanish and we evaluate their transcription productivity.

Abstract Knowledge mining from documents usually use document engineering techniques that allow the user to access the information contained in documents of interest. In this framework, transcription may provide efficient access to the contents of handwritten documents. Manual transcription is a time‐consuming task that can be sped up by using different mechanisms. A first possibility is employing state‐of‐the‐art handwritten text recognition systems to obtain an initial draft transcription that can be manually amended. A second option is employing crowdsourcing to obtain a massive but not error‐free draft transcription. In this case, when collaborators employ mobile devices, speech dictation can be used as a transcription source, and speech and handwritten text recognition can be fused to provide a better draft transcription, which can be amended with even less effort. A final option is using interactive assistive frameworks, where the automatic system that provides the draft transcription and the transcriber cooperate to generate the final transcription. The novel contributions presented in this work include the study of the data fusion on a multimodal crowdsourcing framework and its integration with an interactive system. The use of the proposed solutions reduces the required transcription effort and optimizes the overall performance and usability, allowing for a better transcription process.

We present βTap, a Back-of-device (BoD) tap detection software for mobile devices that uses commodity sensors, without the need to instrument the device. Although just basic interactions are supported (namely, single and double taps), βTap is highly accurate and performance-friendly, since it uses a low-cost yet highly discriminative set of features. Our software is publicly available at the Google Play Store, so that others can build upon our work.

Transcription of handwritten documents is an important research topic for multiple applications, such as document classification or information extraction. In the case of historical documents, their transcription allows to preserve cultural heritage because of the amount of historical data contained in those documents. The transcription process can employ state-of-the-art handwritten text recognition systems in order to obtain an initial transcription. This transcription is usually not good enough for the quality standards, but that may speed up the final transcription of the expert. In this framework, the use of collaborative transcription applications (crowdsourcing) has risen in the recent years, but these platforms are mainly limited by the use of non-mobile devices. Thus, the recruiting initiatives get reduced to a smaller set of potential volunteers. In this paper, an alternative that allows the use of mobile devices is presented. The proposal consists of using speech dictation of handwritten text lines. Then, by using multimodal combination of speech and handwritten text images, a draft transcription can be obtained, presenting more quality than that obtained by only using handwritten text recognition. The speech dictation platform is implemented as a mobile device application, which allows for a wider range of population for recruiting volunteers. A real acquisition on the contents of a Spanish historical handwritten book was obtained with the platform. This data was used to perform experiments on the behaviour of the proposed framework. Some experiments were performed to study how to optimise the collaborators effort in terms of number of collaborations, including how many lines and which lines should be selected for the speech dictation.

Automatic transcription of historical documents is becoming an important research topic, specially because of the increasing number of digitised historical documents that libraries and archives are publishing. However, state-of-the-art handwritten text recognition systems are far from being perfect. Therefore, to have perfect transcriptions, human expert revision is required to really produce a transcription of standard quality. In this context, an interactive assistive scenario, where the automatic system and the human transcriber cooperate to generate the perfect transcription, would allow for a more effective approach. In this paper we present a multimodal interactive transcription system where user feedback is provided by means of touchscreen pen strokes, traditional keyboard and mouse operations. The combination of both the main and the feedback data stream is based on the use of Confusion Networks derived from the output of the on-line and off-line handwritten text recognition systems. The use of the proposed combination help to optimise overall performance and usability.

Transcription of handwritten historical documents is one of the main topics in document analysis systems, due to cultural reasons. State-of-the-art handwritten text recognition systems allow to speed up the transcription task. Currently, this automatic transcription is far from perfect, and human expert revision is required in order to obtain the actual transcription. In this context, crowdsourcing emerged as a powerful tool for massive transcription at a relatively low cost, since the supervision effort of professional transcribers may be dramatically reduced. However, current transcription crowdsourcing platforms are mainly limited to the use of non-mobile devices, since the use of keyboards in mobile devices is not friendly enough for most users. This work presents the alternative of using speech dictation of handwritten text lines as transcription source in a crowdsourcing platform. The experiments explore how an initial handwritten text recognition hypothesis can be improved by using the contribution of speech recognition from several speakers, providing as a final result a better hypothesis to be amended by a professional transcriber with less effort.

Back-of-device (BoD) interaction using current smartphone sensors (e.g. accelerometer, microphone, or gyroscope) has recently emerged as a promising novel input modality. Researchers have used a different number of features derived from these commodity sensors, however it is unclear what sensors and which features would allow for practical use, since not all sensor measurements have an equal value for detecting BoD interactions reliably and efficiently. In this paper, we primarily focus on constructing and selecting a subset of features that is a good predictor of BoD tap-based input while ensuring low energy consumption. As a result, we build several classifiers for a variety of use cases (e.g. single or double taps with the dominant or non-dominant hand). We show that a subset of just 5 features provides high discrimination power and results in high recognition accuracy. We also make our software publicly available, so that others can build upon our work.

Transcription of historical documents is an interesting task for libraries in order to make available their funds. In the lasts years, the use of Handwritten Text Recognition allowed paleographs to speed up the manual transcription process, since they are able to correct on a draft transcription. Another alternative is obtaining the draft transcription by dictating the contents to an Automatic Speech Recognition system. When both sources (image and speech) are available, a multimodal combination is possible, and an iterative process can be used in order to refine the final hypothesis. In this work, a multimodal combination based on confusion networks is presented. Results on two different sets of data, with different difficulty level, show that the proposed technique provides similar or better draft transcriptions than a previously proposed approach, allowing for a faster transcription process.

According to statistics data, the population of many countries is aging. Moreover, there is also a small sector of the population that is affected by disabilities. Researchers should develop systems to improve their quality of life. In recent years, the development of smartphones and reduced size devices, but with high processing capacity, has increased dramatically. We can take profit of the sensors and applications embedded in the smartphones in order to monitor disabled and elderly people. In this paper, we describe a smart collaborative system based on the sensors placed in the mobile devices, which allow us to monitor the status of a person based on what is happening in the group people. The network algorithm and the smart system protocol is described and simulated in order to show the performance of our proposal.