We propose an approach to efficiently adapt the non-verbal information of personal robots to a user's preferences. As non-verbal information focuses on gestures except meaningful gestures, in our method the adaptation problem is formalized as choreography; the action (as one unit of the non-verbal information) is assigned to each state as preferred by the user. Since the state is defined based on the classification of language focusing on the shallow discourse structure proposed by [1], our method is constructed with independence of tasks and situations unlike previous methods. Furthermore, using a preferences database obtained from multiple users, we produce a User Preference Model (UPM) that can represent various user preferences by a small number of parameters. A new user is asked to assign actions on a few states to adapt the UPM for the user preference. After the adaptation, the UPM can be used to predict actions on other states as preferred by the user, that accomplishes adaptive choreography. We implemented the proposed method with a personal robot, and validated it through experiments on multiple tasks with human subjects. As a result, we confirmed that the user's impressions of the robot were greatly improved by our method for multiple tasks.

We describe a method for generating an informative wide-view image using images captured by a moving camera. The generated image allows for events in the scene observed by the camera to be understood easily. Our method does not use 3D shape information explicitly. Instead, it employs the trajectory of feature points across multiple images and generates a composite image by taking into account the distribution of the trajectories of the feature points.

We present a real-time method for estimating the pose of a human body using its 3D volume obtained from synchronized videos. The method achieves pose estimation by pose regression from its 3D volume. While the 3D volume allows us to estimate the pose robustly against self occlusions, 3D volume analysis requires a large amount of computational cost. We propose fast and stable volume tracking with efficient volume representation in a low dimensional dynamical model. Experimental results demonstrated that pose estimation of a body with a significantly deformable clothing could run at around 60 fps.

We propose a method for estimating the pose of a human body using its approximate 3D volume (visual hull) obtained in real time from synchronized videos. Our method can cope with loose-fitting clothing, which hides the human body and produces non-rigid motions and critical reconstruction errors, as well as tight-fitting clothing. To follow the shape variations robustly against erratic motions and the ambiguity between a reconstructed body shape and its pose, the probabilistic dynamical model of human volumes is learned from training temporal volumes refined by error correction. The dynamical model of a body pose (joint angles) is also learned with its corresponding volume. By comparing the volume model with an input visual hull and regressing its pose from the pose model, pose estimation can be realized. In our method, this is improved by double volume comparison: 1) comparison in a low-dimensional latent space with probabilistic volume models and 2) comparison in an observation volume space using geometric constrains between a real volume and a visual hull. Comparative experiments demonstrate the effectiveness of our method faster than existing methods.

Large-print textbooks are used by low vision students in school. Because these books are mainly prepared by volunteers and almost all steps in the preparation process are performed manually, they cannot be mass-produced. The chronic shortage of these books has been a social problem in Japan. The procedure for preparing a large-print textbook involves (1) converting the size of figures and characters in the original textbook to one suitable for low vision students and (2) positioning them appropriately on the pages. In this paper, we propose a novel method for automatically optimizing a layout by employing a context structure. We represent the context structure by using a graph called a context structure graph. The proposed method first allocates each material to an appropriate page, and then optimizes the layout of each page by using the sequence-pair method. Throughout these operations, we employ an objective function derived from the context structure graph to ensure that the context in the original textbook is preserved in the large-print textbook prepared.

We propose a matching method for images captured at different times and under different capturing conditions. Our method is designed for change detection in streetscapes using normal automobiles that have an off-the-shelf car mounted camera and a GPS. Therefore, we are able to analyze low-resolution and low frame-rate images captured asynchronously. To cope with this difficulty, previous and current panoramic images are created from sequential images which are rectified based on the view direction of a camera, and are then compared. In addition, in order to allow the matching method to be applicable to images captured under varying conditions, (1) for different lanes, enlarged/reduced panoramic images are compared with each other, and (2) robustness to noises and changes in illumination is improved by the edge features. To confirm the effectiveness of the proposed method, we conducted experiments matching real images captured under various capturing conditions.

This paper proposes a free-viewpoint imaging method that can be used in a complicated scene such as an office room by using sparsely located cameras. In our method, a free-viewpoint image is generated from multiple image patches obtained by dividing observed images. The quality of the generated image strongly depends on how to divide the observed images. In an incorrect patch in the generated image, the images projected from different cameras differ significantly. With this property, the incorrect patches can be detected. These patches are then re-divided. We demonstrated the effectiveness of our method by generating free-viewpoint images from the real images observed by the cameras in an office room.

This paper proposes a method for precise overlapping of projected images from multiple steerable projectors. When they are controlled simultaneously, two problems are revealed: (1) even a slight positional error of the projected image, which does not matter in the case of a single projector, causes misalignments of multiple projected images that can be perceived clearly when using multiple projectors; and (2) as the projectors usually do not have architectures for their synchronization it is impossible to display a moving image that is by tiling or overlaying precisely the multiple projected images. To overcome (1), a method is proposed that measures preliminarily the misalignments through every plane in the environment, and hence displays the image without the misalignment. For (2), a consideration and a new proposal for the synchronization of multiple projectors are also discussed.

We are designing a self controlling active camera system for a 3D video of a moving object (mainly human body). We made up our system of cameras with long focal length lenses for high resolution input images. However, such cameras can get only partial views of the object. We present, in this paper, a multiple active (pan-tilt) camera assignment scheme. The goal is to assign each camera to a specific part of the moving object so as to allow the best visibility of the whole object. For each camera, we evaluate the visibility to the different regions of the object, corresponding to different camera orientations and with respect to the field of view of the camera in question. Thereafter, we assign each camera to one orientation in such a way to maximize the visibility to the whole object.

This paper presents an analysis study of handwritten Kanji characters on a digitizing tablet. To solve a writer identification and verification problem, we have tried to extract significant individual characteristics of a certain set of Kanji characters, which might be defined as feature parameters derived from the knowledge of document examiners. A set of feature parameters in terms of character shape and writing behavior is evaluated by the statistical test. Evaluation results have shown that some feature parameters are good enough to identify and verify a writer himself.

Analysis of the human gaze is a basic way to investigate human attention. Similarly, the view image of a human being includes the visual information of what he/she pays attention to.This paper proposes an interface system for extracting the region of an object viewed by a human from a view image sequence by analyzing the history of gaze points. All the gaze points, each of which is recorded as a 2D point in a view image, are transfered to an image in which the object region is extracted. These points are then divided into several groups based on their colors and positions. The gaze points in each group compose an initial region. After all the regions are extended, outlier regions are removed by comparing the colors and optical flows in the extended regions. All the remaining regions are merged into one in order to compose a gaze region.

People tend to forget where they placed an object, which is needed to achieve a certain task, in their everyday circumstances. To support a user's object-finding tasks, we have proposed a wearable interface system named "I'm Here!". The system manages Augmented Memories, a video database of the user's viewpoint labeled with previously registered objects, to display the last video of the target object held by the user. We evaluate the function of the system with laboratory experiments, and discuss the positive and negative effect of the system with the experimental results.

We propose the Wearable Virtual Tablet (WVT), where a user can draw a locus on a common object with a plane surface (e.g., a notebook and a magazine) with a fingertip. Our previous WVT[1], however, could not work on a plane surface with complicated texture patterns: Since our WVT employs an active-infrared camera and the reflected infrared rays vary depending on patterns on a plane surface, it is difficult to estimate the motions of a fingertip and a plane surface from an observed infrared-image. In this paper, we propose a method to detect and track their motions without interference from colored patterns on a plane surface. (1) To find the region of a plane object in the observed image, four edge lines that compose a rectangular object can be easily extracted by employing the properties of an active-infrared camera. (2) To precisely determine the position of a fingertip, we utilize a simple finger model that corresponds to a finger edge independent of its posture. (3) The system can distinguish whether or not a fingertip touches a plane object by analyzing image intensities in the edge region of the fingertip.

In this paper, we are concerned with the problem of how a physical robot can get an appropriate internal representation to its task and environment. Learning from experience is effective for the problem, but it is very time-consuming to learn a representation from the beginning in a real environment. On the other hand, the representation learned only in a simulated environment has the risk of not serving the purpose in a real environment because of the uncertainty in sensors, actuators, and the environment. In, order to have the best of both worlds, it is effective to transplant the learned state representation of a virtual agent to a physical robot. For this purpose., we improved our developed incremental learning architecture for use in the real environment and developed a new architecture, called STNS-R. In this architecture, inappropriate negative instances caused by uncertainties are found on the basis of the distribution of instances and removed in order to correct the distorted shapes of the states. The effectiveness of STNS-R is shown in the experimental results.

This paper describes 3D vision techniques for an advanced robot system in terms of efficient hardware and practical software. Arm-controlled stereo cameras have been developed for efficient 3D sensing, whose interocular distance, yaw and tilt are under program control. Stereo vision programs are implemented with the aid of the stereo cameras and a high-speed vision processor. Knowledge-guided vision algorithms are realized in a high-level processor, where a priori knowledge regarding the environments, objects and procedure controls are utilized.

We consider the problem of texture discrimination. Random walks are performed in a plain domain D bounded by an absorbing boundary ¿ and the absorption distribution is calculated. Measurements derived from such distributions are the features used for discrimination. Both problems of texture discrimination and edge segment detection can be solved using the same random walk approach. The border distributions and their differences with respect to a homogeneous image can classify two different images as having similar or dissimilar textures. The existence of an edge segment is concluded if the boundary distribution for a given window (subimage) differs significantly from the boundary distribution for a homogeneous (uniform grey level) window. The random walk procedure has been implemented and results of texture discrimination are shown. A comparison is made between results obtained using the random walk approach and the first-or second-order statistics, respectively. The random walk procedure is intended mainly for the texture discrimination problem, and its possible application to the edge detection problem (as shown in this paper) is just a by-product.

A new edge detection technique for digital image processing is presented. The technique is viewed as a low-level operator in the context of the more complex problem, that of scene analysis, and a conceptual comparison with some previous edge detectors is done. The new edge detection technique has been implemented and its results are compared with two other edge detection techniques using the same kind of pictures as input data. The software implementation of this new edge detector written in machine language takes 32 s for an image of size 128 by 128 picture elements. The edge detector can be hardware-implemented and such an implementation, for which the estimated processing time will be about half a second, is given in the Appendix.