In this paper, we show preliminary results on the quality assessment of crowdworker output based on the movements of the mouse and the eyes while the task is performed. We assume that the mouse and the eyes stop longer if the quality is lower due to the lack of knowledge, or confidence, etc. Because the mouse- and eye-stopping duration follows lognormal distribution, we estimate its parameters (mean and standard deviation) to evaluate the quality. Results of preliminary experiments with 10 participants show that the parameters of correct outputs are different from those of incorrect ones. As compared to the task duration, which is often used as a feature for assessment, we have found that the mouse-and the eyestopping duration is advantageous and complementary for the assessment.

We present the Vocabulometer, a reading assistant system designed to record the reading activity of a user with an eye tracker and to extract mutual information about the users and the read documents. The Vocabulometer stands as a web platform and can be used for analyzing the comprehension of the user, the comprehensibility of the document, predicting the difficult words, recommending document according to the reader's in order to increase his skills, etc. Since the last years, with the development of low-cost eye trackers, the technology is now accessible for many people, which will allow using data mining and machine learning algorithms for the mutual analysis of documents and readers.

Comics is usually classified into broad categories called "genres" according to its contents such as comedy, horror, science fiction, etc. Because a genre expresses a comics story briefly, people read comics which has contents based on their interest, by relying on comics genres. However, giving only one genre to one comic cannot express the detailed difference of the story. In this paper, we propose a system for generating comics story representation as a sub-sequence of genres. Our comics story representation can be applied to a new search engine based on stories or to a recommendation system which analyzes the tastes of the user's favorite comics by finding comics with similar story representation. We use a deep neural network to classify each page into the corresponding genre. Experimental results confirm the advantage of the proposed system.

This paper presents an approach for identifying reader specific difficult words while someone is reading a textual document. The work is motivated by the need of developing human-document interaction systems, in general and creating person-specific online educational content, in particular. Eye gaze information gives person specific behavior whereas textual content is analyzed to get general linguistic aspect of the document content. These two pieces of information are fused together through machine learning algorithms to identify the set of difficult words for a particular reader reading a particular document. An annotated dataset has been created where each word in a document is marked with its bounding box information and each reader identifies a set of difficult words while reading the document. The dataset consists of sixteen documents and each document is read by five subjects. The method is evaluated through recall-precision analysis. The impressive precision at high recall attests the feasibility of building a practical application based on this research. The experiment further brings out several interesting facts about human reading behaviour.

We propose a system for document annotation using a camera mounted on a smartphone. It works both on paper documents and electronic documents thanks to the functionality of document image retrieval. An important characteristic of this system is in the way of annotating documents. The system employs simple character stickers which represent user's opinions ("hard to understand", "interesting", "boring", "surprising", "doubtful") for friendly annotation on documents. We evaluated our system by changing the way of annotation and found that users most liked the proposed way of annotation with stickers though it sometimes caused a confusion about the interpretation of stickers. We discuss the possible way of solving this issue as a result of the analysis of experimental results.

The popularity of storing, distributing and reading comic books electronically has made the task of comics analysis an interesting research problem. Different work have been carried out aiming at understanding their layout structure and the graphic content. However the results are still far from universally applicable, largely due to the huge variety in expression styles and page arrangement, especially in manga (Japanese comics). In this paper, we propose a comic image analysis approach using eye-tracking data recorded during manga reading sessions. As humans are extremely capable of interpreting the structured drawing content, and show different reading behaviors based on the nature of the content, their eye movements follow distinguishable patterns over text or graphic regions. Therefore, eye gaze data can add rich information to the understanding of the manga content. Experimental results show that the fixations and saccades indeed form consistent patterns among readers, and can be used for manga textual and graphical analysis.

Instead of analyzing directly the document images, analyzing the document reading can offer new perspectives for extracting information about both the reader and the document. Analyzing how people read texts can help to understand the cognitive process of the reading and might lead to new approaches and new solutions for pattern recognition and document image analysis. It can also lead to create smart documents that can measure reading information, provide feedback and adapt themselves depending on the behavior of the readers. As a step towards document reading analysis, the authors propose in this paper a solution for extracting the reading information and creating a “reading-life log”. This reading-life log contains basic features that can be used for many different kinds of applications. A tag cloud evolving according to the reading is presented as a first application of the reading-life log.

Comics and manga are one of the most important forms of publication and play a major role in spreading culture all over the world. In this paper we focus on balloons and their association to comic characters or more generally text and graphic links retrieval. This information is not directly encoded in the image, whether scanned or digital-born, it has to be understood according to other information present in the image. Such high level information allows new browsing experience and story understanding (e.g. dialog analysis, situation retrieval). We propose a speech balloon and comic character association method able to retrieve which character is emitting which speech balloon. The proposed method is based on geometric graph analysis and anchor point selection. This work has been evaluated over various comic book styles from the eBDtheque dataset and also a volume of the Kingdom manga series.

Reading-life log is a research field of analyzing our activities of reading documents to know more about readers and documents. In this paper we propose an implementation of reading-life log which is to estimate the English language skill by analyzing the activities of reading English documents. As input for the analysis, we employ eye movement information, because we consider the eye movement of skillful readers is far different from that of novices. From the experiments, we have found that the following two features are informative: (1) the sum of fixation duration, and (2) the sum of the velocity of saccades. By using these features the proposed method is to estimate the class of English skill from among low, middle and high, which are defined based on the scores of English standardized test called TOEIC. From the experimental results with 11 subjects and 10 documents, we have been successful to estimate the class with the accuracy of 90.9%.

Template matching is a technique used in classifying an object by comparing portions of images with another image. Finding a given template in an image is typically performed by scanning the image and evaluating the similarity with the template. When the scanning is concerned with the entire image template matching is optimal. This paper considers a special case of template matching where the templates are binary. Although binary template matching has been studied extensively since the early days of pattern recognition, this technique seems not longer in use in Document Image Analysis (DIA). The major reasons arête time complexity, the no-invariance to scale and rotation and the lack of adaptability of similarity measures. However, different contributions have been investigated during the last years to improve these aspects: robustness and discrimination capability of similarity measures, their characterization, time-processing optimization with hardware support, etc. In this paper, we will review first some of the recent issues about binary template matching. We will present then a system exploiting bitwise operators and parallel processing supporting fast and accurate binary template matching for Manga copyright protection. This system is compared to a FFT-based template matching, and it outperforms both in processing-time and detection accuracy.

Manga (Japanese style comics) is one of the most popular publications. Nowadays manga is often handled as digital images not only in consumers' use but also in digital media. However, they hardly handle manga as content-based materials. Some digital media use tags or text data for retrieval, where the tags and text data are produced by handmade input. Therefore our goal is achieving content-based retrieval method for manga images. As the first step to the goal, we investigate the performance of Sun's method applying to manga character retrieval. Manga character retrieval means a image retrieval of which the input and output are a character image and page images where the input character appears respectively. It is useful for convenient use of manga images, for example, character retrieval or auto-tagging. We modify Sun's method so as to be applicable to manga character retrieval and then investigate the performance.

Concerning camera-captured Japanese character recognition, we have proposed a method to recognize characters, both simple and complex, that may not be linearly aligned and may be printed with a complex background. Recognition is performed based on local features and their arrangement. The arrangement is validated with an algorithm called local RANSAC. However, at least four corresponding local features are required. To relax that condition, we propose a new recognition method making it possible to recognize a character region with at least three corresponding local features. This method enables recall and precision to be improved with the simpler characters using more corresponding local features and computation times to be reduced by 7%.

In this paper a novel method for automatic ground truth generation of camera captured document images is proposed. Currently, no dataset is available for camera captured documents. It is very difficult to build these datasets manually, as it is very laborious and costly. The proposed method is fully automatic, allowing building the very large scale (i.e., millions of images) labeled camera captured documents dataset, without any human intervention. Evaluation of samples generated by the proposed approach shows that 99.98% of the images are correctly labeled. Novelty of the proposed approach lies in the use of document image retrieval for automatic labeling, especially for camera captured documents, which contain different distortions specific to camera, e.g., blur, occlusion, perspective distortion, etc.

In this paper we argue that a key-region detector designed to take into account the special characteristics of document images can result in the detection of less and more meaningful key-regions. We propose a fast key-region detector able to capture aspects of the structural information of the document, and demonstrate its efficiency by comparing against standard detectors in an administrative document retrieval scenario. We show that using the proposed detector results to a smaller number of detected key-regions and higher performance without any drop in speed compared to standard state of the art detectors.

It is thought that a large quantity of data improve quality of recognition. A large database, however, is not easy to obtain. The hardest task is labeling (also known as ground truthing), which usually requires human intervention. Since labeling by human is laborious and costly, labeling without human (automatic labeling) or minimization of human intervention (semi-automatic labeling) are ideal scenarios. As a step toward realization of the scenarios, knowing how much an automatic labeling system can perform without human intervention is important. In the current paper we propose a comprehensive automatic labeling technique for a scene text database, which performs segmentation and labeling for unsegmented and unlabeled character images. To our best knowledge, this is the first method to realize the comprehensive process for automatic labeling for scene text databases In experiments, we confirmed that the proposed method could add new unlabeled data in parallel with improving recognition performance of the classifier.

Reading life log is a type of techniques to automatically and unconsciously record people's reading intentions, interests and habits. Besides, it can also serve as various assistants in our daily life. In this paper, a reading-life log system is implemented by a head-mounted and unobtrusive video camera with a high resolution and a high shutter speed. We utilize DP matching, and propose a text-based frame mosaicing method to integrate multiple frames in a clip. The developed system is tested in the various environments indoor and outdoor. The experimental results show that our system can provide reliable outputs with respect to the most correct responses. The infrequent misregistration between lines also indicates the feasibility and validity of the text-based frame mosaicing.

In a scene image, some characters are difficult to recognize and some others are recognized easily. Such difficult characters usually make the processing time long while easy characters are recognized in a short time. In this paper, we propose a system which recognizes each character with a proper cost for the difficulty. Through the process, easy characters are recognized early and difficult ones are recognized late. This is a desired property of an anytime algorithm that the recognition accuracy does not decrease as the time increases. In order to realize it, we propose a method which splits the recognition process into several times and accumulates the recognition results and extracted features. We also discuss what is required to realize the anytime algorithm for the scene character recognition task. Experiments reveal that the proposed method obtains recognition results of easy characters earlier than the conventional method.

We introduce the Wordometer, a novel method to estimate the number of words a user reads using a mobile eye tracker and document image retrieval. We present a reading detection algorithm which works with over 91 % accuracy over 10 test subjects using 10-fold cross validation. We implement two algorithms to estimate the read words using a line break detector. A simple version gives an average error rate of 13,5 % for 9 users over 10 documents. A more sophisticated word count algorithm based on support vector regression with an RBF kernel reaches an average error rate from only 8.2 % (6.5 % if one test subject with abnormal behavior is excluded). The achieved error rates are comparable to pedometers that count our steps in our daily life. Thus, we believe the Wordometer can be used as a step counter for the information we read to make our knowledge life healthier.

The document analysis community spends substantial resources towards computer recognition of any type of text (e.g. characters, handwriting, document structure etc.). In this paper, we introduce a new paradigm focusing on recognizing the activities and habits of users while they are reading. We describe the differences to the traditional approaches of document analysis. We present initial work towards recognizing reading activities. We report our initial findings using a commercial, dry electrode Electroencephalography (EEG) system. We show the feasibility to distinguish reading tasks for 3 different document genres with one user and near perfect accuracy. Distinguishing reading tasks for 3 different document types we achieve 97 % with user specific training. We present evidence that reading and non-reading related activities can be separated over 3 users using 6 classes, perfectly separating reading from non-reading. A simple EEG system seems promising for distinguishing the reading of different document genres.

Comic books are a kind of storytelling graphic publications mainly expressed by abstract line drawings. As a clue of story lines, comic characters play an important role in the story, and their detection is an essential part of comic book analysis. For this purpose, the task includes (1) locating characters in comics pages and (2) identifying them, which is called specific character detection. Corresponding to different scenes of comic books, one specific character can be represented by various expressions coupled with rotations, occlusions, and other perspective drawing effects, which challenge the detection. In this paper, we focus on stable features regarding the possible transformations and proposed a framework to detect them. Specifically, some discriminative features are selected as detectors for characterizing characters, on the basis of a training dataset. Based on the detectors, the drawings of the same characters in different scenes can be detected. The methodology has been experimented and validated on 6 titles of comics. Despite the terrific changes for different scenes, the proposed method achieved detection of 70% comic characters.

We present a new system that assists people's reading activity by combining a wearable eye tracker, a see-through head mounted display, and an image based document retrieval engine. An image based document retrieval engine is used for identification of the reading document, whereas an eye tracker is used to detect which part of the document the reader is currently reading. The reader can refer to the glossary of the latest viewed key word by looking at the see-through head mounted display. This novel document reading assist application, which is the integration of a document retrieval system into an everyday reading scenario for the first time, enriches people's reading life. In this paper, we i) investigate the performance of the state-of-the-art image based document retrieval method using a wearable camera, ii) propose a method for identification of the word the reader is attendant, and iii) conduct pilot studies for evaluation of the system in this reading context. The results show the potential of a document retrieval system in combination with a gaze based user-oriented system.

Recognition of scene text using a hand-held camera is emerging as a hot topic of research. In this paper, we investigate the use of a head-mounted eye-tracker for scene text recognition. An eye-tracker detects the position of the user's gaze. Using gaze information of the user, we can provide the user with more information about his region/object of interest in a ubiquitous manner. Therefore, we can realize a service such as the user gazes at a certain word and soon obtain the related information of the word by combining a word recognition system with eye-tracking technology. Such a service is useful since the user has to do nothing but gazes at interested words. With a view to realize the service, we experimentally evaluate the effectiveness of using the eye-tracker for word recognition. The initial results show the recognition accuracy was around 70% in our word recognition experiment and the average computational time was less than one second per a query image.

Serial animated cartoons, also called animations, is a kind of popular video documents that describe narratives by videos of cartoons. By using digital techniques, illegal users can divide animations into fragments and distribute them without any copyright permissions. In this paper we focus on the copyright problem of animations and try to retrieve copyright infringement fragments based on key frames. Because of the huge volumes and rapid release of animations, it is impossible to store all the episodes into the database. We propose applying bag-of-features model to retrieve similar fragments based on the visual words extracted from a limited data set. In the experiments, 12 titles of animations are employed and the similar fragments outside database are applied as queries. Our method has achieved above 98% precision at 80% recall for fragments whose durations are over 140 seconds. From the results, we show that a latest release can be retrieved based on the features of the former ones from the same title.

This paper presents a novel interface running on smart phones which is capable of seamlessly linking physical and digital worlds through paper documents. This interface is based on a real-time document image retrieval method called Locally Likely Arrangement Hashing. By just only pointing a smart phone to a paper document, the user can obtain its corresponding electronic document. This can easily provide the user with the information associated with the retrieved document. This relevant information can be superimposed on the display of smart phones. Therefore, we consider that with the help of this interface, the user can utilize paper documents as a new medium to display various information.

Large datasets are always demanded for better recognition performance. However, it is not easy to produce them because costly and slow human operators have been necessary for labeling. In the current paper, in order to resolve the problem on yielding large datasets, we propose a scenario for automatic labeling based on the self-corrective recognition algorithm. The strong point of the proposed method is the capability of expanding recognizable distorted characters unlike existing methods. In the experiments, we show a possibility to realize automatic labeling by the method.

Recognizing characters in a scene helps us obtain useful information. For the purpose, character recognition methods are required to recognize characters of various sizes, various rotation angles and complex layout on complex background. In this paper, we propose a character recognition method using local features having several desirable properties. The novelty of the proposed method is to take into account arrangement of local features so as to recognize multiple characters in an image unlike past methods. The effectiveness and possible improvement of the method are discussed.

In this paper we propose a complete system for online stroke recovery from offline data. The key idea of our approach is to use a novel pen device which is able to embed meta information into the ink during writing the strokes. This pen-device overcomes the need to get access to any memory on the pen when trying to recover the information, which is especially useful in multi-writer or multi-pen scenarios. The actual data-embedding is achieved by an additional ink dot sequence along a handwritten pattern during writing. We design the ink-dot sequence in such a way that it is possible to retrieve the writing direction from a scanned image. Furthermore, we propose novel processing steps in order to retrieve the original writing direction and finally the embedded data. In our experiments we show that we can reliably recover the writing direction of various patterns. Our system is able to determine the writing direction of straight lines, simple patterns with crossings (e.g., "x" and "II"), and even more complex patterns like handwritten words and symbols.

Manga, a kind of graphic novels expressed by sequential line drawings, is an important document image publication which is invoking more and more attentions for their copyright protection. Because of simple constructions and abstract expression styles, similar copies are always applied in plagiarisms of mangas. In addition, the enormous volume of copyrighted manga publications issues a challenge to the task of detecting suspicious images. Considering only some regions of interests (ROIs) require copyright protection, we propose a bag-of-features method using visual words based on ROIs for similar manga retrieval. In the experiments, we applied real manga publications as data and proved the effectiveness of the proposed method.

This paper presents a real-time document image retrieval method for a large-scale database with Locally Likely Arrangement Hashing (LLAH). In general, when a database is scaled up, a large amount of memory is required and retrieval accuracy drops due to insufficient discrimination power of features. To solve these problems, we propose three improvements: memory reduction by sampling feature points, improvement of discrimination power by increasing the number of feature dimensions and stabilizing features by reducing redundancy. From the experimental results, we have confirmed that the proposed method realizes 50% memory reduction, and achieves 99.4% accuracy and 38ms processing time for a database of 10 million pages.

This paper addresses how to quickly recognize a character pattern using a lot of case examples without learning. Here without learning means just finding the most similar example from the case examples, and pretend as if the OCR understands the definition of the character. This strategy is expected to work well in most cases with a large dataset, however, also expected to take a lot of time for finding the most similar example. In this paper, we show that a lot of case examples can be processed in a short time. As a testbed, we handle recognition problem of camera-captured printed characters. Using a database storing 100 fonts, the proposed method achieved 97.0% of recognition rate for images captured from the right angle and 95.8% for those from 45 deg. with 4.56ms of processing time, that is about 220 characters per second including every process.

This paper presents a method of improving the accuracy of document image retrieval focusing on the application to a camera-pen system. In a camera-pen system, document image retrieval is employed for locating the pen-tip position on a page. A serious problem is that since the camera is mounted close to the pen-tip, the camera captures only a tiny portion of the page and the resultant image is under severe perspective distortion, resulting in lowering the retrieval accuracy. To solve this problem, we propose new geometrically invariant features as well as expansion techniques which increase the number of index features of either the database or the query images. From the experimental results, it has been found that the query expansion technique with features by combining affine and perspective invariants allows us the best performance that improves the accuracy of a baseline method more than 27%.

In this paper we present the first operational version of the data-embedding pen. During writing a pattern, this pen produces an additional ink-dot sequence along the ink stroke of the pattern. The ink-dot sequence represents, for example, meta-information (such as the writer's name and the date of writing) and thus drastically increases the value of the handwriting on a physical paper. Since the information is placed on the paper, it can be extracted just by scanning or photographing the paper. There is no need to get access to any memory on the pen to recover the information. This is useful especially in multi-writer or multi-pen scenarios. The experiments using an encoding scheme and a decoding algorithm showed very promising results. For example, it was proved that we can embed 28 or more bits of information on simple handwritten patterns and decode them with a high reliability.

Camera-based character recognition systems should have the capability of quick operation and recognizing perspectively distorted texts in a complex layout. In this paper, in order to realize such a system, we propose a simple but efficient implementation of camera-based recognition of characters and pictograms. With help of new hashing and voting techniques, the proposed method runs well in real-time even on a laptop PC with a web camera.

This paper presents a new method of capturing digital ink for pen-based computing. Current technologies such as tablets, ultrasonic and the Anoto pens rely on special mechanisms for locating the pen tip,which result in limiting the applicability.Our proposal is to ease this problem --- a camera pen that allows us to write on ordinary paper for capturing digital ink. A document image retrieval method called LLAH is tuned to locate the pen tip efficiently and accurately on the coordinates of a document only by capturing its tiny fragment.In this paper, we report some results on captured digital ink as well as to evaluate their quality.

We propose a real-time retrieval method for document images in various languages. In this method, queries are images of documents captured by a web-camera. The document images corresponding to the queries are retrieved from the document image database in real time. Since we have already proposed a document image retrieval method for English documents, the proposed method is an extension for retrieval of documents in various languages. In the previous English document image retrieval method, only centroids of word regions are used as feature points. Therefore it cannot be applied to some languages including Japanese and Chinese due to no separation between words and periodic arrangements of characters. In the proposed method, additional features are introduced to realize real-time retrieval for document images in various languages.

The partial copy is a kind of copy produced by cropping parts of original materials. Illegal users often use this technique for plagiarizing copyrighted materials. In addition, original parts are not necessarily copied intact but may be modified by various techniques, and embedded into other materials, which make the detection quite difficult. In this paper, we propose a method of copyright protection applicable to partial copies aiming at the protection of line drawings such as comics. In order to cope with handwritten partial copies, we apply local feature matching with a database of copyrighted line drawings. Experimental results show that the proposed method not only performs good for detecting printed copies of line drawings, but also has effectiveness on the detection of handwritten ones, even if partial copies are embedded in complex backgrounds.

Detection of characters in scenery images is often a very difficult problem. Although many researchers have tackled this difficult problem and achieved a good performance, it is still difficult to suppress many false alarms and although missings. This paper investigates a conspicuous character pattern, which is a special pattern designed for easier detection. In order to have an example of the conspicuous character pattern, we select a character font with a larger distance from a non-character pattern distribution and, simultaneously, with a smaller distance from a character pattern distribution. Experimental results showed that the character font selected by this method is actually more conspicuous (i.e., detected more easily) than other fonts.

There are many problems to realize camera-based character recognition. One of the problems is that characters in scenes are often distorted by geometric transformations such as affine distortions. Although some methods that remove the affine distortions have been proposed, they cannot remove a rotation transformation of a character. Thus a skew angle of a character has to be determined by examining all the possible angles. However, this consumes quite a bit of time. In this paper, in order to reduce the processing time for an affine invariant recognition, we propose a set of affine invariant features and a new recognition scheme called "progressive pruning."' The progressive pruning gradually prunes less feasible categories and skew angles using multiple classifiers. We confirmed the progressive pruning with the affine invariant features reduced the processing time at least less than half without decreasing the recognition rate.

There is an approach of annotation extraction from printed documents in which annotations are extracted by comparing the image of an annotated document and its original document image. In one of the previous methods, the image of an original document is actually printed and scanned in order to reproduce image degradations of the image of the annotated document. However such a method lacks convenience since users have to use the same printer and scanner to obtain images of an annotated document and its original document. In this paper, we propose an improved annotation extraction method in which the image degradations are compensated by image processing. In the proposed method, the difference between original and annotated document images due to image degradations is reduced by not only removal of the degradations in the annotated document images but also reproduction of the degradations in the original document images. The proposed method consists of three steps of processing which are for dithering, for color change, and for local displacement. We also propose an objective evaluation of extracted annotations to compare the experimental results accurately. Experimental results of the proposed method have shown that the recall of extracted annotations was 80.94% and the precision was 85.59%.

This paper proposes a novel skew estimation method by instances. The instances to be learned (i.e., stored) are rotation invariants and a rotation variant for each character category. Using the instances, it is possible to estimate a skew angle of each individual character on a document. This fact implies that the proposed method can estimate the skew angle of a document where characters do not form long straight text lines. Thus, the proposed method will be applicable to various documents such as signboard images captured by a camera. Experimental evaluation using synthetic and real images revealed the expected robustness against various character layouts.

This paper is concerned with a universal pattern, which is defined as a character pattern designed to have high machine-readability. This universal pattern is a charac- ter pattern printed with stripes. The cross ratio calculated from the widths of the stripes represents the character class. Thus, if the boundaries of the stripes can be detected for measuring the widths, the class can be determined without ordinary recognition process. Furthermore, since the cross ratio is invariant to projective distortions, the correct class will be still determined under those distortions. This pa- per describes a practical scheme to recognize this universal pattern. The proposed scheme includes a novel algorithm to detect the stripe boundaries stably even from the universal pattern image contaminated by non-uniform lighting and noise. The algorithm is realized by a combination of a dy- namic programming-based optimal boundary detection and a finite state automaton which represents the property of the universal pattern. Experimental results showed the pro- posed scheme could recognize 99.6% of the universal pat- tern images which underwent heavy projective distortions and non-uniform lighting.

We propose a method of document image retrieval using digital cameras. The proposed method takes as input a part or the whole of a document acquired as a query by a digital camera, and retrieves a document image that includes the query. For this purpose, it is required to solve the problem of "perspective distortion" of images, as well as to establish a way of matching parts of document images flexibly. These are achieved based on the following characteristics of the proposed method: (1) indexing of document images using the projective invariants called the "cross-ratios", (2) retrieval as voting for partial signatures of document images defined by the cross-ratios. From experimental results using digital cameras with high and low resolutions, we demonstrate the effectiveness of the proposed method.

In order to realize seamless integration of paper and electronic documents, it is at least necessary to assure error free conversion from one to the other. In general, the conversion from paper to electronic documents is the task of document image understanding. Although its research has made remarkable progress, it is still a hard task without limiting the type of documents. This paper presents a completely different approach to this task on condition that printed documents have their originals in electronic form. The proposed method employs fine dots to represent data of electronic documents and places the dots on white space (backgrounds) of pages. Since the data is encoded with an error correcting code, it is guaranteed to be correctly recovered from the scanned images of documents. Experimental results show that a page with normal foreground objects (characters and other things) can contain more than 4KB of data, even when errors up to 20% of the data are permitted. 1.

Document image retrieval is a task to retrieve document images relevant to a user's query. Most existing methods based on word-level indexing rely on the representation called "bag of words" which originated in the field of information retrieval. This paper presents a new representation of documents that utilizes additional information about the location of words in pages so as to improve the retrieval performance. We consider that pages are relevant to a query if they contain its terms densely. This notion is embodied as density distributions of terms calculated in the proposed method. Its performance is improved with the help of "pseudo relevance feedback", i.e., a method of expanding a query by analyzing pages. Experimental results on English document images show that the proposed method is superior to conventional methods of electronic document retrieval at recall levels 0.0-0.6.

Retrieval of electronic documents is a fundamental component for intelligent access to the contents of documents. For the retrieval of long documents, a method called passage-based document retrieval has proven to be effective. In this paper we experimentally show that the passage-based retrieval is also advantageous for dealing with short queries on condition that documents are long. We employ a passage-based method based on density distributions of query terms in documents, and compare it with three conventional methods: the vector space model, pseudo-feedback and latent semantic indexing.

This paper presents a method of automated generation of hypertext links for electronic documents. The goal is to generate links from an arbitrary part of a document (a source of a link) to its relevant parts of target documents (destinations). To achieve this goal, we assume that words are often shared by parts of documents if these parts are relevant with each other. In order to extract parts densely including words of a source (keywords), we employ density distributions of keywords. This enables us to determine destinations simply by extracting parts whose density exceeds a threshold. Experiments on generating links from figures/tables to parts of documents, as well as from texts to parts of different documents show that our method with the optimal parameters yields recall of 60% and precision of 50%.

A conceptual diagram is a kind of diagram which represents a logical structure among concepts using simple physical objects (loops, lines and character strings). This paper presents a method of interpretation of conceptual diagrams. In conceptual diagrams, a single physical object plays various logical roles depending on surrounding physical objects, because there are no specific ruled of writing conceptual diagrams. To cope with this problem, we introduce the strategy of hypothesis generation and verification; hypothesized interpretations are verified by relaxation which takes account of logical relations to other physical objects. From the experimental results using 50 conceptual diagrams, we discuss the effectiveness of our method.

Document image analysis systems often utilize the knowledge about layout structures to extract layout objects labeled logically. However, the lack of the facility for knowledge acquisition limits the applicability of the systems. The authors propose a method of acquiring knowledge for document image analysis. Given examples of document images and their layout objects labeled logically, the method generates and modifies the knowledge. The method is incremental so that the knowledge can be efficiently modified using additional examples. Counterexamples generated as errors obtained from the analysis of an example image can also be reflected into the knowledge so that the system may no longer generate the errors. Experimental results on both knowledge acquisition and the analysis using the acquired knowledge are also presented.>