This paper presents an autonomous conversational mobile robot Butsukusa that can describe its own observations and internal states during patrolling tasks. The proposed robot can observe the surrounding environment using the recognition module for objects, humans, environment, localization, and speech and then move autonomously around an indoor living space. Interaction skills via language are required for the robot to perform in such human-centered spaces. To investigate a better communication protocol with users, we evaluate various language generation patterns based on different observations and interaction patterns. The evaluation results indicate that the importance of describing the robot's observation results and internal states, as well as the necessity of an appropriate description, depends on the situation.

Images captured in participating media such as murky water, fog, or smoke are degraded by scattered light. Thus, the use of traditional three-dimensional (3D) reconstruction techniques in such environments is difficult. In this paper, we propose a photometric stereo method for participating media. The proposed method differs from prvious studies with respect to modeling shape-dependent forward scatter. In the proposed model, forward scatter is described as an analytical form using lookup tables and is represented by spatially-variant kernels. We also propose an approximation of a large-scale dense matrix as a sparse matrix, which enables the removal of forward scatter. We discuss the approximation in the proposed method using synthesized data. Then, experiments with real data demonstrate that the proposed method improves 3D reconstruction in participating media.

Images captured in participating media such as murky water, fog, or smoke are degraded by scattered light. Thus, the use of traditional three-dimensional (3D) reconstruction techniques in such environments is difficult. In this paper, we propose a photometric stereo method for participating media. The proposed method differs from previous studies with respect to modeling shape-dependent forward scatter. In the proposed model, forward scatter is described as an analytical form using lookup tables and is represented by spatially-variant kernels. We also propose an approximation of a large-scale dense matrix as a sparse matrix, which enables the removal of forward scatter. Experiments with real and synthesized data demonstrate that the proposed method improves 3D reconstruction in participating media.

Sea surface temperature(SST) satellite images are often partially occluded by clouds. Image inpainting is one approach to restore the occluded region. Considering the sparseness of SST images, they can be restored via learning-based inpainting. However, state-of-the-art learning-based inpainting methods using deep neural networks require large amount of non-occluded images as a training set. Since most SST images contain occluded regions, it is hard to collect sufficient non-occluded images. In this paper, we propose a novel method that uses occluded images as training images hence we can enlarge the amount of available training images from a certain SST image set. This is realized by comprising a novel reconstruction loss and adversarial loss. Experimental results confirm the effectiveness of our method.

Sea surface temperature (SST) images taken from satellites are partially occluded by clouds. In this paper, we propose an inpainting approach for restoration of the partially occluded images. Assuming the sparseness of the SST images, we employ a learning based inpainting for filling the occluded parts. Images taken in the past several days is another clue for filling the occluded parts. These images are regarded as time series data and a video inpainting method is also available. We employ PCA-based inpainting as a learning-based approach and optical-flow-based inpainting as video inpainting, and combine the two restored images according to the expected their restoration error. Experimental results with real satellite images show the effectiveness of our method.

In this paper, we present a regression for predicting 3D rigid transformations from real-valued vectors. We use a unit dual quaternion to represent the transformation. The regression is formulated as blending unit dual quaternions. To formulate it in a closed form, we introduce an approximation based on error metrics according to geometric algebra. Finally, we take an articulated motion and an elastic deformation as examples to present the descriptive power of our method in modeling the motion and the deformation.

Image restoration is a fundamental problem in the field of image processing. The key objective of image restoration is to recover clean images from images degraded by noise and blur. Recently, a family of new statistical techniques called variational Bayes (VB) has been introduced to image restoration, which enables us to automatically tune parameters that control restoration. While information from one image is often insufficient for high-quality restoration, however, current state-of-the-art methods of image restoration via VB approaches use only a single-degraded image to recover a clean image. In this paper, we propose a novel method of multiframe image restoration via a VB approach, which can achieve higher image quality while tuning parameters automatically. Given multiple degraded images, this method jointly estimates a clean image and other parameters, including an image warping parameter introduced for the use of multiple images, through Bayesian inference that we enable by making full use of VB techniques. Through various experiments, we demonstrate the effectiveness of our multiframe method by comparing it with single-frame one, and also show the advantages of our VB approach over non-VB approaches.

Sensing the intention of a user’s forthcoming action is a necessary function for systems that assist human physical activity. In this article, a strategy for recipe guidance systems that can predict the forthcoming intended subtask in a cooking task is investigated. The focus is on user accessing objects, that is, touching and releasing objects. Touching can indicate the start of the forthcoming subtask and releasing can indicate the end of the task. The main difficulty lies in the fact that humans may move objects because they are in the way and use cooking tools that are unanticipated by an assistive system. In such cases, the accessed object should not indicate the forthcoming subtask. A method is proposed to track the progress of a task based on the object access history. This enables to eliminate object accesses that are out of context. Simultaneously, the method predicts the forthcoming subtask based on a combination of progress and materials rather than tools and materials. Then, a guidance system that runs as a web service is developed. In experiments, real cooking activities navigated by this system are observed. The Wizard of OZ method is utilized to simulate a system that detects object accesses. The experimental results show that 73.6% accuracy is achieved in the selection of the displayed information. This result supports the use of “access to objects” realize effective intention-sensing systems.

In this paper, we adopt the coded aperture technique to the alignment process for industrial machinery. As a special setting for assembly and inspection, such machinery uses an illumination of narrow wavelength range for imaging with less aberration. This leads to significant influence of light diffraction on the image restoration. Although the diffraction was treated as negligible in most previous studies of coded aperture since they were carried out for natural images, the aperture patterns of them do not achieve enough accuracy for the alignment. We optimize the aperture pattern by performing a simulation of light diffraction, and we experimentally show that it achieves better performance in the alignment.

Most vision-based 3D acquisition methods including both passive and active methods have a limitation in that cameras must be able to observe the surface to be measured. If this is not possible, that is to say, if the surface is occluded, most of the methods cannot acquire the surface shape. In this paper, we present a method that can acquire the 3D points on the occluded surfaces. The main idea is to observe the scattering of reflected light in a participating medium instead of observing only the reflected light. We place the target in a participating medium, specifically a liquid tank filled with a participating medium, and irradiate a laser beam on it. Even if the reflecting point is occluded, our method can acquire the 3D position of occluded reflecting points from the scattering. Experimental results showed that our method can measure 3D reflecting points on several objects even if the objects are completely occluded.

Most shape and reflectance acquisition methods use the light reflected from objects' surfaces. When the reflection cannot be observed, e.g., when the object's surface is black matte or highly specular, its shape and reflectance are difficult to acquire. In this paper, we propose a method that can measure shape and reflectance with another approach. Our method involves the use of the scattering of reflected light in a participating media instead of using only the light reflection. We place the target in a participating medium and focus a laser beam on it. Cameras can observe the scattering of reflected light toward all reflection angles even if they can only observe light reflected toward the cameras. Experimental results showed that our method can acquire the shape and reflectance of various surfaces.

In this paper, we propose a novel method of skeleton estimation for the purpose of constructing and manipulating individualized hand models via a data glove. To reconstruct actual hand accurately, we derive motion constraints in fully 6-DOF at the joints without assuming either a center of rotation or a joint axis. The constraints are derived by regression analysis on the configuration of the finger segments with respect to the sensor data. In order to acquire pairs of the configuration and the sensor data, we introduce graspable reference objects for reproducing postures with and without the data glove. To achieve accurate regression on fewer samples, we introduce a regression model according to a detailed investigation using a fused motion capture system, which enabled us to perform optical motion capture and sensor data collection simultaneously. The effectiveness of the method is demonstrated through practical applications involving grasped reference objects.

Multiple-shot person re-identification tackles the problem to build the correspondences between sets of human images obtained from distributed cameras. It is challenging due to large within-class variations and small between-class differences, caused by the changing of human appearance and environment. Existing methods for addressing this issue include designing the representation to capture the within-set correlation, or crafting the measure to explore the between-set separation. This paper proposes a novel set based matching model called “Locality Based Discriminative Measure (LBDM)”, in which the discriminative potentiality of a new set-to-set distance is exploited by using the learned local metric field. As experimentally demonstrated, the proposal remarkably outperforms state-of-the-art schemes on public benchmark datasets.

Set based recognition has been attracting more and more attention in recent years, benefitting from two facts: the difficulty of collecting sets of images for recognition fades quickly, and set based recognition models generally outperform the ones for single instance based recognition. In this paper, we propose a novel model called collaboratively regularized nearest points (CRNP) for solving this problem. The proposal inherits the merits of simplicity, robustness, and high-efficiency from the very recently introduced regularized nearest points (RNP) method on finding the set-to-set distance using the l2-norm regularized affine hulls. Meanwhile, CRNP makes use of the powerful discriminative ability induced by collaborative representation, following the same idea as that in sparse recognition for classification (SRC) for image-based recognition and collaborative sparse approximation (CSA) for set-based recognition. However, CRNP uses l2-norm instead of the expensive l1-norm for coefficients regularization, which makes it much more efficient. Extensive experiments on five benchmark datasets for face recognition and person re-identification demonstrate that CRNP is not only more effective but also significantly faster than other state-of-the-art methods, including RNP and CSA.

Person re-identification concerns about the problem of recognizing people across space (captured by different cameras) and/or over time gaps. Though recently the literature on it grows rapidly, all the proposed solutions have treated it as a normal classification or ranking problem. In this paper, however, we argue that it is in fact a natural transfer learning problem, thus it's valuable and also necessary to investigate how the progress on transfer learning could benefit the research on it. We present so far the first study on justifying the effectiveness of a representative transfer learning methodology: feature-based inductive transfer learning, for person re-identification. Extensive experiments on standard datasets with typical methods result in several important findings.

In this paper we propose a simple and effective solution to the important and challenging problem of across-camera person re-identification. We focus on the common case in video surveillance where multiple images or video frames are available for each person. Instead of exploring new features, the proposed approach aims at making a better use of such images/frames. It builds a collaborative representation over all the gallery images (of known person individuals) to best approximate the query images (containing an unknown person) via affine combinations. The approximation is measured by the nearest point distance between the two affine hulls constructed by the query images and gallery images, respectively. By enforcing the sparsity of the samples used for approximating the two nearest points, the relative importance of the gallery images belonging to different persons has the ability to reveal the identity of the querying person. Extensive experiments on public benchmark datasets demonstrate that the proposed approach greatly outperforms the state-of-the-art methods.

Person re-identification tackles the problem whether an observed person of interest reappears in a network of cameras. The difficulty primarily originates from few samples per class but large amounts of intra-class variations in real scenarios: illumination, pose and viewpoint changes across cameras. So far, proposals in the literature have treated this either as a matching problem focusing on feature representation or as a classification/ranking problem relying on metric optimization. This paper presents a new way called Common-Near-Neighbor Analysis, which to some extent combines the strengths of these two methodologies. It analyzes the commonness of the near neighbors of each pair of samples in a learned metric space, measured by a novel rank-order based dissimilarity. Our method, using only color cue, has been tested on widely-used benchmark datasets, showing significant performance improvement over the state-of-the-art.

Person re-identification is one of the most challenging issues in network-based surveillance. The difficulties mainly come from the great appearance variations induced by illumination, camera view and body pose changes. Maybe influenced by the research on face recognition and general object recognition, this problem is habitually treated as a verification or classification problem, and much effort has been put on optimizing standard recognition criteria. However, we found that in practical applications the users usually have different expectations. For example, in a real surveillance system, we may expect that a visual user interface can show us the relevant images in the first few (e.g. 20) candidates, but not necessarily before all the irrelevant ones. In other words, there is no problem to leave the final judgement to the users. Based on such an observation, this paper treats the re-identification problem as a ranking problem and directly optimizes a listwise ranking function named Mean Reciprocal Rank (MRR), which is considered by us to be able to generate results closest to human expectations. Using a maximum-margin based structured learning model, we are able to show improved re-identification results on widely-used benchmark datasets.

In this paper, we propose a real-time system for measuring the consumption of various types of seasonings. In our system, all seasonings are placed on a scale, and we continuously take images of these items using a camera. Our system estimates the consumption of each condiment by calculating the difference between the weight when the seasoning was picked up and the weight when it was placed back on the scale. Our system identifies the type of seasoning that was used by determining whether or not the seasoning was present on the scale. By using our system, users can automatically log their usage of seasoning. Then, they can adjust the seasoning according to their desired taste.

This paper presents a method for recognizing recipe ingredients based on the load on a chopping board when ingredients are cut. The load is measured by four sensors attached to the board. Each chop is detected by indentifying a sharp falling edge in the load data. The load features, including the maximum value, duration, impulse, peak position, and kurtosis, are extracted and used for ingredient recognition. Experimental results showed a precision of 98.1% in chop detection and 67.4% in ingredient recognition with a support vector machine (SVM) classifier for 16 common ingredients.

This paper presents an artifact-free super resolution texture mapping from multiple-view images. The multiple-view images are upscaled with a learning-based super resolution technique and are mapped onto a 3D mesh model. However, mapping multiple-view images onto a 3D model is not an easy task, because artifacts may appear when different upscaled images are mapped onto neighboring meshes. We define a cost function that becomes large when artifacts appear on neighboring meshes, and our method seeks the image-and mesh assignment that minimizes the cost function. Experimental results with real images demonstrate the effectiveness of our method.

This paper describes our work in computer vision to track food materials in the food preparation process. Tracking such food materials is difficult, because they are often hidden when moved by hand. Furthermore, their appearance may change in hand when they are cut or peeled. For tracking these objects in such situations, we propose a novel method that matches an object on a cooking table to one grasped in the past. We use the following three criteria to match the objects even when they are cut or peeled: the similarity in their appearance, the validity of their change in appearance, and the grasped order. We experimentally evaluated our method by applying it to the scenes of cutting and peeling food materials. As a result, we achieved an accuracy of 83.6% in matching the objects.

In this paper, we address a novel task "cooking recognition task". Cooking recognition task is to keep recognizing a cooking action of a cook and the target food product of the action at any time of the cooking. The target object of the recognition could change its visual feature completely by peeling, cutting, and mixed with others during the task. Nonetheless, such food product is called by the name that is given at the beginning of the cooking. Because the existing image recognition methods recognizes an object based on such idea that objects of the same name have similar visual feature, such methods cannot solve this cooking recognition task. Therefore, we propose new methods that recognize an object based on such idea that object keeps its identity even after manufactured and/or mixed with others.

This paper develops a graph-theoretic framework for estimating comprehension in classroom. To deal with imprecise data gathered in classroom, we propose multi-step comprehension propagation over a semantic graph. Random walks on the graph measure students' comprehension with probabilities absorbed at student nodes.

We propose a method for synthesizing an undistorted highresolution image of a lecture room. A certain degree of high-resolution videos can be captured using commercially available high-resolution cameras. However, the resolution of objects captured by these cameras is not enough for viewers of a lecture video. To get sufficient high-resolution lecture videos, we synthesize multiple images captured by low cost cameras using a planar perspective projection. However, lecturers are distorted by the projection because they are not planar objects and do not exist on planar objects in a lecture room. To get an undistorted figure of a lecturer, we introduce a fixed-viewpoint projection using a lecturer tracking camera. As a result, we could get more high-resolution images including an undistorted figure of a lecturer than ones captured by a standard high-resolution camera.

We consider concept representation in the framework of conceptual spaces. We propose the introduction of fuzzy sets to express the entries of the concept (co-occurrence) matrix as fuzzy probabilities. Initial experimental results indicate that the proposed approach can support a much more sophisticated and useful concept representation.

Photometric stereo is a method of recovering surface normals (needle map) from images. The surface integral of surface normals is used to reconstruct a depth map; however, the depth edges, which are discontinuous boundaries of the depth map, pose a problem for photometric stereo. When the surface of objects includes depth edges, the reconstructed depth map may contain errors. To solve this problem, we detect depth edges using shadows and compute a relative depth between two distant points using the widths of the corresponding shadows. We define an error function and reconstruct the depth map by minimizing the error function. Experimental results with synthetic and with real image data demonstrate the effectiveness of our approach.

3D shapes are reconstructed from silhouettes obtained by multiple cameras with the volume intersection method. In recent work, methods of integrating silhouettes in time sequences have been proposed. The number of silhouettes can be increased by integrating silhouettes in multiple frames. The silhouettes of a rigid object in multiple frames are integrated with its rigid motion. This motion is often estimated with 3D feature points extracted from silhouettes. When the estimated motion has large error, shapes are reconstructed with missing parts. This error is given by the incomplete extraction of 3D feature points, which is caused by additional and missing regions of extracted silhouettes. We cannot prevent silhouettes from being extracted with the additional and missing regions in real environments. Here, we propose an intelligent method of integrating incomplete silhouettes where outcrop points, which are 3D feature points for estimating motion, play an important role. The reconstructed shape can be evaluated referring to how many outcrop points have been included in the reconstructed shape of another frame. Although the evaluation does not represent the accuracy of estimated motion directly, it does guarantee that outstanding parts will be preserved in the reconstructed shape. Silhouettes in multiple frames can be integrated with fewer missing and additional parts based on this evaluation.

Abstract In the last few years, live pay‐TV systemviathe Internet has come to be established with the development of broadband network. To develop such a system, a mechanism for content protection is required. In this paper, a new architecture integrating fingerprint with cryptosystem is proposed to protect digital contents (image and video) from unauthorized viewing and illegal redistribution. Then, a scheme integrating anti‐collusion code and Home Page public‐key cryptosystem (PKC) is described. Such a scheme is shown feasible and realistic for implementing with respect to computation/transmission cost. Copyright © 2008 John Wiley & Sons, Ltd.

Measuring proximity (or similarity) underlies many algorithms in information processing (by humans or indeed animals, or by machines). When the data involved are heterogeneous this issue is further complicated. This paper discusses some of the difficulties of measuring similarity between heterogeneous data and possible directions for addressing these difficulties using computational intelligence/fuzzy set based techniques. An important concept underlying the discussion is that of consistent meaning of proximity.

This study designed a distance education support system (DESS) by integrating face-recognition techniques and blog system. The DESS system was then applied in an international distance course conducted between two universities of Taiwan and Japan with an evaluation study for investigating the usability and effectiveness of the system in supporting distance education. Findings suggested that the system is helpful and clear for representing students' learning profiles as individual, within group and in class meeting. Besides, students reported that they are comfortable about using it in class, especially for improving interaction between instructors and students. However, students do not prefer instructors to use it for assessment purpose. We concluded that this DESS system is has potential in improving interaction and reducing students' feeling of isolation in distance course. Also we provided some suggestions in future studies and system development.

This paper describes the implementation of blog system in an international distance course between Japan and Taiwan. In the study, blog was used as a tool to encourage students' reflective learning and communication. Findings suggested that blog was effective for students to document their learning, share experience and knowledge, have direct interaction with peers especially internationally. Students preferred blog over the LMS where this course builds an course Web site and they proved to use blog in developing their e-learning experience. We proposed the use of blog could lead to a technologically enhanced support for instructional strategy that serves as an informative model for other Web-assisted international courses

In this article, we discuss 3D shape reconstruction of an object in a rigid motion with the volume intersection method. When the object moves rigidly, the cameras change their relative positions to the object at every moment. To estimate the motion correctly, we propose new feature points called outcrop points on the reconstructed 3D shape. These points are guaranteed to be located on the real surface of the object. If the rigid motion of the object can be correctly estimated, cameras at different moments serve as the cameras in different positions virtually. With these cameras in time sequences, we can increase accuracy of the reconstructed 3D shape without increasing the number of cameras. Based on this idea, we reconstruct an accurate shape of the object in motion from images obtained by limited number of cameras. As the result, we can acquire an accurate shape from images in time sequences

We present an active lighting method that can control brightness of multiple objects in a scene to any values on video images. Our approach does not modify pixel values such as in retouching but changes intensities of multiple lights in the scene. Our algorithm needs little computation to calculate appropriate intensities of the lights because we formulate the brightness of the objects so that the essential parameters in our formulation can be measured in advance. We have implemented a prototype system that can control brightness of faces of two people in a room to intended values with eight controllable lights as they are walking.

An advanced multimedia lecture recording system is presented in this paper. The purpose of our system is to capture multimodal information that can be received only when lectures are being held in a classroom where a teacher and students share the same time and space. The system captures not only hand-writings and slide switching intervals but also audio and video of the people with their spatial location information. These recorded media will be served to users in distance learning process. We designed the system so that it does not interfere with classes. It can archive lectures while they hold classes on regular basis and generate a multimedia archive automatically. Teachers are neither asked to remain at a certain place nor wired with devices they would have to put on. We have implemented our approach and our lecture archive system currently works for six classes per week, since October, 2002.

In this paper, we propose a method for detecting the location of the speaker that is a target of automatic video filming in distance learning and lecture archive. It is required that a face of a speaking student is filmed in a lecture video. For this purpose, it is necessary to detect the location of a speaker. An acoustic sensor such as a microphone array is used widely to detect the location of a sound source. However, it is difficult to detect the location of a sound source precisely using only microphone array because of sound noise in a large space such as a lecture room. In this paper, we propose a method for detecting more precise location of a speaker in the lecture room using not only the microphone array but also visual sensors. The result shows that the precision ratio of detecting the location of a speaker was improved about 20% by our sensor-fusion method.

This article discusses face translation (FT), which replaces the user's face with an expression by a different synthetic face, such as the face of an avatar or a cartoon character, with the same expression. In order to realize FT, we have to establish a criterion to define which expression on the synthetic face is the “same ” expression as the user wears on his/her face. Different from previous work, it is proposed in this article to adopt the user’s subjective viewpoint for this criterion. The user’s viewpoint is acquired online through the interaction with the user. When the user is not satisfied with the synthetic facial expression given as the result of FT by the system, the user corrects it by a graphical user interface (GUI). The system obtains examples of the FT from the user's subjective viewpoint every time the user corrects the results of translation. With increase of examples, the result of FT by the system gradually comes to reflect the user’s subjective viewpoint, so that the user does not need to correct the result. 1.

Recognizing structure of the human body is important for modeling human motion. The human body is usually represented as an articulate model, which consists of the rigid parts and the joint points between them. The structure of the human body is specified by the joint points. We propose a method for estimating the location of joint points from successive volume data. Our joint point estimation method consists of three steps. In the first step, rigid parts are extracted from two successive volume data under the constraint of the rigid transformation. In the second step, the joint points are estimated based on the rigid parts. As the last step, false joint points are eliminated by using more successive data. Applying the method to the simulated data, the locations of joint points of the human are correctly estimated.

The characteristics of a multimedia content structure based on its temporal parallelism are typically found in counterpoint musical pieces. These characteristics act as a rhythmic synchronization mechanism that forms a temporal image of the multimedia content. In our Multimedia Montage study, we have developed a prototype system that produces a movie according to the script description of the component structure. We applied this system to the counterpoint structure and produced movies that we call counterpoint movies. In order to automate the rhythmic synchronization of the multimedia components, we started a new study called Image Wave. In this study, we are engaged in the rhythmic information extraction of multimedia components and their composition based on the rhythmic information.

This paper describes a task-oriented approach to cope with the difference between the three dimensional (3D) space in a virtual reality (VR) system and that perceived by the user from its stereoscopic images with binocular disparity. This difference occurs for the reason that human visual perception is affected not only by binocular disparity but also by various kinds of visual cues. Although it is possible to set some of those cues consistent with each other when they are presented by a stereoscopic display, it is difficult to do the same for other cues especially vergence and focus. As the result, the users of VR systems often fail to manipulate virtual objects due to this difference. In this article, we propose to adjust the virtual space in a VR system through object manipulation by the user so that the user does not fail in each manipulation. We verified by experiments that this adjustment is effective to improve the rate of successful manipulation.

When the dimensionality of the feature space increases and takes beyond a certain point, the classification performance of a parametric classifier begins to deteriorate. This is because the number of parameters of the classifier depends on the dimensionality and gets too large in a high dimensional space. To obtain the density value at the point of interest without deriving the parameters, we propose a nonparametric Gaussian density model, where the sum of the log-density values at two points is described, without any parameter, by a function of the distance between the two points. The density value at the point of interest is estimated from the distance between the point and each of the training sample points using this model. We will empirically show that the density values estimated by the nonparametric model are more accurate than those by the parametric model in a high dimensional space. In a character recognition experiment, our nonparametric classifier achieved higher classification accuracy in comparison with the parametric classifier.

Document clustering is a powerful tool for browsing throughout a document database. Similar documents are gathered into several clusters and a representative document of each cluster is shown to users. To make users infer the content of the database from several representatives, the documents must be separated into tight clusters, in which documents are connected with high similarities. At the same time, clustering must be fast for user interaction. We propose an O(n/sup 2/) time, O(n) space cluster extraction method. It is faster than the ordinal clustering methods, and its clusters compare favorably with those produced by Complete Link for tightness. When we deal with OCR-ed documents, term loss caused by recognition faults can change similarities between documents. We also examined the effect of recognition faults to the performance of document clustering.

The authors have implemented a document image retrieval system; it automatically stores document images in the form of character candidates obtained by character recognition process. When we give some keywords to the system, it can retrieve the images containing at least one of the keywords. The strategy of using character candidates remarkably lowers the rate of missing the keywords on retrieval, because it includes several hypotheses in character segmentation and in character recognition. For finding the keywords from the storage of the character candidates, the authors have developed a fast and efficient searching algorithm. It is able to locate all occurrences of any finite number of keywords in the character candidates. The improvement of using character candidates and the efficiency of the searching algorithm are also described.>

A method to extract character strings from unformed document images, which have inclined strings of character patterns and have no structure at all, is described. To process such kinds of unformed documents, previous schemes, which are intended only to deal with such documents as have just horizontal or vertical strings of characters, do not work well. The method is based on the idea that the processes of recognition and extraction of character patterns should cooperate together, and on the characteristic that the character patterns are located close to each other when they belong to the same string and apart when they do not. The method has been implemented and the experimental results show the robustness of the method.>

A general-purpose graphical symbol extraction method is proposed in which the visual characteristics of graphical symbols are used. The graphical symbols are modeled as gatherings of short line segments and loops. Based on these models, two different methods are applied in parallel to a diagram, and two results are generated and integrated into the candidates. Further, a method to classify the extracted graphical symbol candidates by comparing their shape and structure is described. An important point of the method is that no a priori knowledge of the diagram under consideration is introduced, hence the method is applicable to a wide range of diagrams.>

Recently there has been an increasing demand for high quality color pictures that can be printed using color dot printers such as ink jet or thermal transfer printers. In this paper, we consider the criteria for the good representation of color pictures and the essential points of the representation processing to satisfy these criteria. We describe a new method for the representation of color pictures based on arranging fixed size primary color dots. This method makes it possible to reproduce the original image colors without losing their smoothness or fineness.

In this paper, we describe a new model of bilevel digital images of line drawings. The key idea is that the constraints of generating the line drawings are available to select patterns which are considered to be a certain local appearance of line drawings. Concretely, when a bilevel digital image of line drawings is observed by 3 × 3 pixels' unit mesh, the patterns occurring in the unit mesh are restricted by the constraints that the digital image is line-structured. At first, lines in a digital image are strictly defined and then two assumptions related to the minimum line width and observing unit mesh are introduced. It is theoretically derived what kinds of patterns occur under these conditions and we call them ``legal patterns,'' while the other patterns not satisfying above conditions are called ``illegal patterns.'' Next, new representations such as SYM pictures and LSC pictures, which consist only of legal patterns, are proposed as a model of bilevel digital images of line drawings and the algorithms to transcribe the original bilevel digital image into the new representations in the model are introduced. Finally, the experimental results are evaluated from the viewpoint of the amount of data involved in the new representations. The results show that our theory is well-fitted to actual images and that many useful and practical applications will be derived not only in the field of documents processing, but also of digital communication.