This paper explores the integration of mobile augmented reality technology in museums, specifically through the development of the "Talkative Museum" prototype, aiming to enrich the museum visiting experience through interactive and collaborative engagements. As vital conduits for cultural and knowledge transmission, museums face the challenge of captivating the digital-native younger generation. Our research addresses this by leveraging advancements in mobile computing and human-computer interaction to foster engaging, intuitive, and collaborative learning environments within the museum context. The prototype facilitates interactions among parents and children and exhibits and encourages exploration and dialogue through augmented reality-enhanced activities. Preliminary user tests involving participants from the National Museum of Ethnology indicated positive responses to the app's usability and collaborative aspects of the museum experience. The analysis of questionnaire responses and observational data suggests that the prototype successfully promotes user engagement, collaborative learning, and the discovery of new knowledge.

Extensive city navigation remains an unresolved problem for autonomous mobile robots that share space with pedestrians. This paper proposes a configuration for a navigation map that expresses urban structures and an autonomous navigation scheme that uses the configuration. The proposed map configuration is a hybrid structure of multiple 2D grid maps and a topological graph. The occupancy grids for path planning are automatically converted from a given 3D point cloud and publicly available maps. The topological graph enables the connections between the subdivisions of occupancy grids to be managed and are used for route planning. This hybrid configuration can embed various urban structures automatically and is applicable to a wide range of autonomous navigation tasks. We evaluated the map by generating the pro-posed navigation map in real city and performing path planning using on the hybrid map. Experimental results demonstrated that the hybrid map can reduce the planning time and memory usage compared to the conventional single 2D grid map based path planning.

We have developed a preliminary learning support system for zoos where children can learn about the ecologies of animals while moving their bodies. For children, the zoo is a place of learning, where they can observe live animals carefully and learn about ecology. However, some animals are exceptionally difficult to observe carefully, and the observers may find it challenging to learn in such scenarios. Therefore, in this research, we developed a learning support system that efficiently acquires knowledge about the ecologies of animals that are difficult to observe in zoos. This system uses a sensor to measure the body movements of the learner; certain animals also tend to respond based on these movements. By doing so, live animals can be carefully observed virtually, and ecological learning is achieved via touch. In this work, we describe the results of evaluating the usefulness of the current system by developing a prototype and evaluating experiments as the first step towards realizing a learning support system to achieve ecological learning of animal observations in zoos.

The paper presents a method to generate a large-scale 3D fundamental map from a running vehicle. To create an easy-to-use approach for frequent updates, we propose a system to utilize simultaneous localization and mapping (SLAM), which is robot mapping technology. In traditional methods, special machines or many manual operations cause higher mapping costs. The existing mobile mapping method (MMS) requires manual anchoring point measurement for ensuring accuracy. To solve this problem, we propose a 3D map optimization method by using road information from the standard map issued by the Geospatial Information Authority of Japan. From the SLAM result, the road center line of 3D shape map is estimated by assuming the car is running on road. Pose graph optimization between the estimated road center line and that of the standard map corrects cumulative distortion of the SLAM result. The experimental results of on-vehicle 3D LIDAR observation show that the proposed system could correct the cumulative distortion of the SLAM results and automatically generate a large-scale 3D map assuring reference accuracy.

In this paper, we develop a collaborative/interactive learning game for deaf children. We developed two separate activities for the proposed game: 'jumping' and 'filling in the blanks'. These two activities are described in detail. Because deaf children tend to find it difficult to acquire voice information, we examined a system that supports the construction of an environment for deaf children in which collaborative learning is encouraged from the perspective of science education. The results of an evaluation confirm that the proposed game encourages collaborative learning among deaf children and improves their motivation to learn grammar.

We have developed a full-body interaction game that allows children to cooperate and interact with other children in small groups. The aim of the game developed in this study is to encourage cooperation between children. The game requires two children to jump together at the same time. In a series of experiments, we observed children using several strategies to coordinate the timing of their jumps, such as shouting to one another and watching each other. The results of a questionnaire to evaluate the proposed interactive game indicate that children strategize to cooperate with each other while enjoying the game. Therefore, the game enables children to cooperate with others using bodily movements.

We are developing an art-appreciation system that will provide learning information about pictures while generating interest in the pictures themselves. This system can provide learning information on pictures by talking to an artist or people portrayed in the pictures using voice recognition. Furthermore, the system operates through movement and location of the user, and it generates sensation such that the user feels he/she is in the pictures, thus actively appreciating the pictures. As the first step toward this system, we have developed a system that provides learning information to the user by actively talking using ukiyo-e. Here, we quantitatively evaluate this system for primary schoolchildren and determine whether the user is interested in the pictures while talking to the pictures using voice recognition measured by electrodermal activity. In this paper, we summarize the current system and describe two evaluation results.

We have developed a full-body interaction game that allows children to cooperate and interact with other children in small groups. The intent of the full-body interaction game developed in this study is to encourage cooperation between children. The game requires two children to jump together with the same timing. We let children experience the game, and observed the children using several strategies to coordinate the timing of their jumps. These included shouting time, watching each other, and jumping in a constant rhythm as if they were skipping rope. In this manner, we observed the children playing the game while cooperating with each other. Therefore, the game enabled children to experience cooperation with others by using body movements.

We are developing an immersive learning support system for paleontological environments in museums. The system measures the physical movement of the learner using a Kinect sensor, and provides a sense of immersion in the paleontological environment by adapting the surroundings according to these movements. As the first step toward realizing an immersive learning support system for museums, Yoshida et al. developed and evaluated a prototype system. However, their system cannot learn about certain paleontological features such as the names of extinct animals and their characteristics. Therefore, we developed an improved version of this system that allows for an enriched knowledge of paleontological environments, focusing in particular on extinct animals and plants and the ecological environment. Here, we evaluate the system’s learning assistant and immersive features insofar as they are directed toward children in primary school. This paper summarizes the current system and describes the evaluation results.

Applying Internet of Things (IoT) technology to art can open up new possibilities for the interaction between systems and humans. In this study, interaction between appreciators and fine art was realized using Ion technology. Raspberry Pi, a low-cost credit card-sized single-board computer, was used to enable appreciators to change animations displayed onscreen in fine art using physical movement. This paper reports on our realization of interaction between appreciators and objet d'art using IoT technology and describes a demonstration conducted via an exhibition that was open to the public.

This paper proposes a method for separating the voice of a signal person from the voices of multiple speakers' in a mixed recording. When the sounds of multiple speakers are present when a voice recognition system is operating, the quality of speech recognition decreases. The proposed method separates the sound of multiple speakers using signal processing techniques and a microphone array. The position of the speaker is also required; however, the microphone array cannot be used for this purpose. Consequently, the position of the speaker was determined using the Kinect V2 depth image sensor. An experiment was performed to separate a signal voice from many voices. The results of the experiment show that, it is possible to separate two voices using the proposed technique.

A method is presented for collecting images of static objects from various directions using a mobile robot equipped with a LIDAR unit and an omni-directional camera. A system using this method collects an image sequence for each object along with its geometric information (3D shape and position). The robot estimates its position while moving and detects static objects by using LIDAR. At each time frame, it extracts a part of the object (the surface element, or “surfel”) in the view range.' The object images are extracted from the time-series of camera images as surfel existing frames. The collected information is automatically collected from the robot. Experimental evaluation demonstrated that this method can be used to continuously collect images and geometric information of static objects in the robot periphery as it is moving.

This paper presents a support system to assist learning within museums. Museums represent important places of scientific learning for children. However, because the main learning method in museums is the study of displayed specimens and their corresponding explanations, the motivation of a child to learn is low. Hence, we are developing an experience-based learning support system such as entertainment that will enable learners to enhance their motivation to learn in any museum. We develop two systems to realize experience-based entertainment. One is "Live Biblia," which is a digital installed book system that uses real fossil. Another is the immersive learning system we called "Body Experience and Sense of Immersion in Digital paleontological Environment" or "BESIDE," which enables learners to explore a virtual paleontological environment. In this paper, we present a summary of the current system of Live Biblia and BESIDE.

In this paper, we propose and develop a full-body interaction system and simulation game called â??Human SUGOROKU,â?? which helps elementary school students learn about vegetation succession while having fun. We found that the students became more involved in the game because they were required to play it using their body movements. An experiment conducted with students verified that the participants became immersed in the virtual world of vegetation succession while playing Human SUGOROKU. This paper describes the structure of our game and the results of its evaluation.

A method is presented for planning the path of an autonomous mobile robot operating in a human environment. Traditional planning methods use a map showing only static obstacle information, so the path generated does not reflect movement rules in the environment. As a result, the path planning does not take into consideration such things as inaccessible area that there is not the obstacle and passage that is one side traffic. The proposed method overcomes this problem by using a pedestrian information map that shows the traffic frequency and velocity in each direction. The map shows the traffic areas and human movement manner. A pedestrian is defined as person moving toward a destination, and a pedestrian information map is generated using only the pedestrian trajectories for all logged human trajectories. Experimental results showed that the proposed method can effectively plan a path through traffic areas.

The authors developed a computer-animated "interactive museum" to improve observers' levels of interest and immersion during the exploration of virtual paleontological environments. In the system, animals on the screen move in synchronization with the observer's actions, and hence, the observer feels as though he/she has entered a real-life paleontological environment. In this paper, in order to enhance the immersive experience, we upgraded the system based on the results of past preliminary evaluation. In the upgraded system, animals on the screen animate whenever an observer walks across the screen, and the animal under observation changes in response to the observer walking backward or forward towards the screen. In this way, the observer becomes part of the virtual paleontological environment. Then, in order to verify the upgrade's effectiveness, we conducted an evaluation. The results indicated that the upgrade of the system was effective in immersing observers because of the fidelity of the simulation and the enjoyment afforded by moving one's own body.

This study proposes a system for supporting the Shotoku Taishi game, which is a language play game that uses the voice of children. The Shotoku Taishi game is a group game in which multiple people presenting a problem vocalize different words at the same time and the respondents are required to guess what the combination of the words is. The authors developed and implemented a system using a microphone array to extract the voice of a specific person presenting a problem in this game. The participants were 36 elementary school students whose native language was Japanese. The results showed that the participants were enjoying the Shotoku Taishi game and that this group activity was a creative activity that deepened their awareness of the Japanese language.

This paper proposes a simulation-based learning support system that animates extinct animals in synchronization with a human’s location and actions. The system provides learners with a near-real body experience, and the sense that they have entered the paleontological environment. The aim of the proposed system is for this experience to improve the learning effect. Obviously, it is impossible for learners to experience the actual paleontological environment, and it is therefore difficult for them to understand the link between this environment and its actuality. This difficulty can be decreased by the proposed system, which provides learners with a real body experience and sense of immersion. At this point in time, the authors are implementing a system based on the proposed concept. As a preliminary evaluation, we conduct an experiment using the system, and interview the participants about the improvement in the learning effect. The experimental results confirm that most learners feel a tangible improvement in the learning effect due to the enjoyment of moving their body during the learning process.

For an autonomous mobile robot that works in real-world environment, recognition of its surrounding environment is necessary. This paper presents an object identification system, which can identify known and unknown objects and estimate their locations using 3D point cloud data acquired from a 3D LIDAR sensor mounted on the mobile robot. The proposed system is divided into two main steps; the first step is segmentation based on the 3D point cloud and the second step is identification of the extracted objects. The 3D LIDAR sensor gives sparse 3D shape information accurately, and covers a wide range. One of the main problems of such data is that the object shape information varies according to object's orientation and distance from the sensor. To solve this problem, we use nested infinite Gaussian mixture models for object identification. The experimental results show that the proposed system can extract various types of objects and identify both known and unknown objects.

In this study, we developed a simulation game called "Human SUGOROKU" that consists of a full-body interaction system to enable elementary school students to enjoy and learn vegetation succession. The students' sense of immersion is improved by enabling them to play this game using their body movements. We conducted an experiment with the students and investigated the affects of the full-body interaction through interviews. The results showed that the full-body interaction promotes a sense of immersion in the game. This paper describes the structure of this system and the interview results.

In this study, evaluation experiment was conducted on an interactive puppet theater, a puppetry system designed for deaf children to enhance their enjoyment of performances. An interactive puppet theater has two features: a feature that ensures the transmission of information by textualizing dialogue information and a feature that encourages deaf children to participate in the story by using physical movements. For this study, experiment was conducted to evaluate whether the feature of participation in the story by using physical movements prompts these children to engage with the story setting in a puppet theater viewing experience. The evaluations were also aimed at ascertaining whether this feature could ensure an enjoyable viewing experience. This study targeted deaf children. The results obtained from the experiments revealed that this feature could be an effective way to encourage them to participate in the story setting, and to assist them in having an enjoyable viewing experience.

We developed a full-body interaction digital game “Human SUGOROKU”. Th is game enables elementary school students to enjoy and learn vegetation succession by playing simulation game with their body movement. We conducted this game to elementary school students and effects of the system were investigated with questionnaires. The result showed that the full-body interaction promotes a sense of immersion in the game. This paper describes the structure of this game and the questionnaire results.

The purpose of this study was to investigate the ‘Pocket Plant Guide’ qualitatively through interviews. We allowed Japanese sixth-grade elementary school students to use the ‘Pocket Plant Guide’ in order to identify and observe plants. We interviewed the students to investigate their subjective impressions of using this guide. The results indicated that this guide was effective in supporting the identification and observation of indicator plants.

The authors perform promotion activity on science and technology for primary school children with human following robot. This robot can follow a particular person as target. In our activity, the children enjoy interaction with robot by being followed as the target. Enjoying with the robot, they are interested in the science and technology. In this paper, we present framework about the human following robot. Also we demonstrate the interaction with the robot in our promotion activity.

This paper investigates that effect of neutral voltage on shaft voltages. The shaft voltages are expected to provide cause of bearing currents. The bearing currents damage bearing in motor. We expect neutral voltages to cause shaft voltages. Thus, we construct circuit of interior motor. And we predict generation of shaft voltages by simulation using the circuit. Then we measure the neutral voltage and draw a comparison between predictive value and measured value. As a result, we can confirm connections between shaft voltages and neutral voltages.

This paper presents following continuation function for human following robot losing target at corner. To realize such function, the authors propose a human trajectory model of turning the corner. Using the proposed model, the robot can predict trajectory of the target human when he/she turns the corner and disappears from the robot's view. To move on the target trajectory, the robot can rediscovery the target and continue to follow. In order to confirm effectiveness of the proposed function and the following robot using it, the authors conduct experiment. In the experiment, the robot can continue human following. Experimental result proves the effectiveness and supports our idea successfully.

Robotic mapping such as two-or three dimensional occupancy grid map has been a field of study. However such occupancy maps are not enough for advanced tasks including object search, object grasping and change detection. We focus on tables and tabletop objects because a table is one of the most important tool in human activity. This paper describes an observation system of tabletop objects based on 3D-sensors and environmental map. We developed a mobile robot equipped with a Velodyne HDL-32E Lidar sensor and a Microsoft Kinect RGB-D sensor.

This paper proposes robust discriminator between bicycle rider and pusher from video sequence. There are a large number of accidents happened while riding a bicycle. The discriminator between bicycle rider and pusher is required to prevent accidents. However, previous works make no mention of the discrimination between similar shape objects such as bicycle rider and pusher. To realize the discrimination, the authors pay attention to difference of their leg movements. Bicycle rider has a pedaling movement, while pusher has a walking movement. To evaluate the difference quantitatively, we utilize CHLAC (Cubic Higher-order Local Auto-Correlation). We conduct an experiment using video sequence in order to confirm effectiveness of the proposed discriminator. In the experiment, success rate of the discrimination can be achieved more than 80%. Thus, we confirm effectiveness of the proposed discriminator.

In this paper, an automatic cancer detection method that combines multiple features to support pathologists was proposed. Cancer is the most cause of death in Japan, and patients suffering with cancer are increasing every year, while the number of pathologists is almost constant. Such issues increase the burden on the pathologists and causes service degradation for the patients. The proposed method combined three image features, Higher-order Local Auto-Correlation (HLAC) feature, Wavelet feature, Delaunay feature. At first, the features were calculated from gastric lymph node images. Then we connected each feature into each vector of varying combinations of the features, and discriminated cancer and no cancer by Support Vector Machine (SVM). HLAC, Wavelet and Delaunay features are shape, frequency, and cell-position geometrical one respectively. Cancer detection rates with more than two features combination were better than only one. In the best performance, sensitivity and specificity were 94.6% and 84.9% respectively.

In order to develop useful assistive devices, analyses and understanding of human functions are necessary. We recorded the activities of daily living, and classified the usage of both hands using the vocabulary of ICF (International Classification of Functioning, Disability and Health) approved by WHO. We confirmed that the usage of the non-dominant hand is auxiliary and greatly different from that of the dominant hand. This indicates a possibility of a reasonable design of assistive devices, e.g. a prosthetic hand for non-dominant hand which has fewer functions but is cheaper than that for dominant hand.

Lacerations occur frequently in numerous common environments, but the mechanisms by which they are generated remain unclear. This makes it difficult for designers to create products that can prevent such injuries. Although a significant number of studies have dealt with injury prevention from the standpoint of avoiding bone fractures, lacerations have yet to be thoroughly investigated and a scientific approach to laceration prevention is urgently required. In this paper, we describe a system for analyzing the dynamic characteristics of human-skin like materials, such as pigskin, in an effort to understand laceration generation mechanisms. The system consists of a high speed camera and a drop impact device. Additionally, we developed a finite element model (FEM) for use in determining the inner behavior of materials via computer simulation. According to statistical analysis using injury surveillance data, a typical situation involves a collision between a victim's head and the edge of an object, such as a table, chair, bathtub, or other such item. Accordingly, we conducted experiments aimed at determining the dynamic characteristic of pigskin, which is a substitute for human skin, when it collides with various materials. This paper reports our experimental results and discusses laceration generation mechanisms in order to obtain scientific knowledge that can be used when designing product safety features.

In this study, we examined the feature of the audience participation in the progress of the story in Puppet Theater. Puppet Theater is a puppet show system developed that is designed for appreciation by hearing-impaired children. Wada et al. (2012) conducted preliminary evaluation experiments performed with 52 healthy elementary school children who were surveyed using both questionnaires and interviews. The results indicated that audience participation in the progress of the story had the effect of encouraging the audience to think about the story. It was also confirmed that, through this participation, the audience is imbued with a sense of emotional investment in the characters, which stimulates audience into immersive participation in the story.

We are developing a digital game, "Human SUGOROKU," in which enables game players as pieces to learn vegetation succession. In order to realize this game, it is necessary to robustly measure 3D position of people and identify them. In this paper, we proposed a method for 3D position measurement and identification of people by integrating ultrasonic sensors and Kinect sensors. The evaluation results show that proposed method is more robust than methods using ultrasonic sensor and Kinect sensor separately.

The authors developed a smartphone-based Pocket Plant Guide to support the outdoor learning of vegetation succession, and evaluated the utility of this device by administering a questionnaire to 35 elementary school children. The results indicated that the Pocket Plant Guide not only functions as a supportive learning tool, but also offers a user-friendly interface.

This paper describes a method to estimate a set of body dimensions of a subject using the Microsoft KINECT sensor. Principal Component Analysis(PCA) of the AIST anthropometric database shows first and second principal components are interpreted as scale and degree of obesity respectively. A few body dimensions were chosen by linear multiple regression analysis. Applying the resulting estimator to a given subject using the KINECT sensor to obtain joint positions and 3 body dimensions by analyzing depth image, we can estimate a set of body dimensions for the subject. The system is described and experimental results are shown.

This paper presents a grasp planning method that takes into account center of gravity of objects in a pre-computation phase, thereby improving an existing method for grasp planning in complex scenes[1]. In the pre-computation phase, the system additionally calculates the distance from fingertips to the center of gravity of an object to use in ranking different proposed grasps, in order to achieve stable grasping with less force. Then we implement the proposed system in the OpenRAVE[2] environment that provides the original method[1]. Evaluation experiments were conducted with the Kawada Nextage[3] robot fitted with our original hand. Results show that the proposed method always grasps target objects closer to their center of gravity.

This paper describes visualizing sound pressure distribution by combining microphone array processing and camera image processing. In microphone array signal processing, capturing a sound at desired position is achieved by Delay and Sum Beam Forming (DSBF). In camera image processing, taking information of 3D position is achieved by triangulation method with stereo camera. We calculate sound level with capturing sound at 3D position taken by camera image processing. In order to visualize sound pressure distribution, calculated sound levels are superimposed on camera image. Experiments were conducted to examine visualization of sound pressure distribution by using different noise and different sound pressure level.

A function to find objects and to estimate their 6D poses(x, y, z, pitch, yaw, roll) is crucial for a home service robot that works in human living environment. If a robot obtains 6D poses of target object, it can bring to that and use as tools. We have proposed SIFT-Cloud-Model (SCM) [1], that is designed to represent 3D objects with textures by 1) SIFT feature descriptors, 2) their 3D positions, and 3) their observed directions (hereafter "view vectors"). This model is used to find target objects in a scene and to estimate their relative 6D poses. In this paper, we propose the SIFT-Cloud-Model generation method based on structure from motion technique combined with optimization technique. Before this research, we had only one model. So we built this system of efficiently generating models to save our time and evaluated its to confirm the effectiveness of SCM to more objects. Finally experimental results of its accuracy evaluation and computational cost will be shown.

Many injures during childhood are related to the use of playground equipment. Until recently, scientific data of how children actually use playground equipment were scarce. Childhood injury cases were not examined thoroughly from the perspective of how equipment can be modified for improving safety without ruining its attraction to children. To design age-appropriate and safer playground equipment, it is essential that scientific data on the interaction between children and this equipment be accumulated. Herein we report on studies to develop new playground equipment by applying sensor technology to examine the science behind children's interaction with playground equipment. We developed a rock-climbing wall equipped with force sensors to record the physical behavior of children while on the wall, thus allowing measurement of these behaviors in a more natural environment. Fifty force sensors installed in the developed rock-climbing wall are able to collect a large amount of data while children are playing with the equipment. The behavior data of 623 children were recorded in the present study. Herein, we also report on a child behavior prediction model created from the collected data.

A function to find objects and to estimate their poses is crucial for a home service robot that works in human living environment.

The detecting and tracking a specific person by using the image processing with the color camera is often influenced by illumination changes. The proposed robust detecting and tracking method to illumination changes is using the HSV color space and updating the histogram. However, the quantitative evaluating is not enough in the proposed method. In this paper, the robustness to illumination changes of the proposed method is evaluated quantitatively. Moreover, the accuracy of the tracking in the proposed method is also evaluated quantitatively. The extraction rate, the object detecting rate, and the tracking error are defined to evaluate quantitatively. The tracking error is also calculated by the MeanShift method based on the color information. The quantitatively experimental results show that the proposed method is the stable and robust tracking method.

The spherical area where sound pressure is high on a partial target made by Speaker Array is called Sound Spot. In this paper, the amount of characteristic was calculated by using HLAC for an image in video sequence. This study aims to make a peculiar person hear the sound of an individual content.

This paper describes a particle filter based sound source mapping system that builds 2D sound source maps from directional sound readings taken from a mobile robot. The method uses a sound source localization model that is represented by Gaussian distribution for both direction and distance. To do this, accurate directional localization of sound sources is required, and two key components have been developed to achieve this: 1) a 32ch low side-lobe microphone array that is designed by beam forming simulation to have a) an omni-directional response, b) a narrower main-lobe, and c) lower side-lobes, in 700-2500[Hz] acoustic signals; 2) directional localization of different pressure sound sources by combining the Delay and Sum Beam Forming (DSBF) and the Frequency Band Selection (FBS) methods. Finally, experimental results show the proposed method can map sound sources in two dimesionals with high accuracy (less than 50[cm] error).

This paper proposes a sound identification method for a mobile robot in home and office environment. We propose a simple sound database called Pitch-Cluster-Maps(PCMs) based on Vector Quantization approach. Binarized frequency spectrum is used for PCMs codebook generation. It can describe a variety of sound sources, not only voice, from short term sound input. The proposed PCMs sound identification requires several tens(msec) of sound input, and is suitable for a mobile robot application which condition is dynamically changing. We implemented the proposed method on our mobile robot audition system equipped with a 32ch microphone array. Robot noise reduction using proposed PCMs recognition is applied to each input signal of a microphone array. The performance of daily sound recognition for separated sound sources from robot in motion is evaluated.

In order to understand utterance based human-robot interation, and to develop such a system, this paper initially analyzes how loud humans speak in a noisy environment. Experiments were conducted to measure

Eels wiggle their long and thin body in swimming. This form is called ldquoAnguilhform swimmingrdquo. Their body is not only a thrust mechanism but also a helm. The method of Anguilhform swimming has features that can accelerate easily and stop instantaneously, compared to method using a screw propeller. And this method has also advantage to be able to whirl around immediately. We think that this advantage will be of much help to swim in a narrow space. We developed an Eel-like swimming robot that swims by Anguilhform swimming. And we assume that Eel-like swimming robot swim in a narrow space, investigated that swimming performance in a narrow water way.

In this paper, we propose the technique classifying background area automatically by calculating the color variance in the RGB space of the picture. By this technique, the background classification robust to change of the background brightness and the background color was enabled. Moreover, we show the effectiveness of this technique by performing particle detection and verifying the result of counting. We aim at development of the particle detector which performs particle counting automatically about the “dispersion staining method” which is one of the processes of asbestos analysis. We used the pictures taken by phase contrast microscopy used for a “dispersion staining method.”

This paper describes a speech recognition system that detects basic voice commands for a mobile robot operating in a home space. The system recognizes arbitrary timed speech with position information in a noisy housing environment. The microphone array is attached to the ceiling, and localizes sound source direction in azimuth and elevation, then separates multiple sound sources using delay and sum beam forming (DSBF) and frequency band separation (FBS) algorithm. We implement the sound localization and separation method on our 32 channel microphone array. The separated sound source is recognized using an open source speech recognizer. These sound localization, separation and recognition functions are implemented as online processing in real world. We define four indices to evaluate the performance of the recognition system, and the efficiency in a noisy environment or with distant sound sources is confirmed from experiments in varied conditions. Finally, an application for a mobile robot interface is reported.

This paper describes the new design of the exhibition support system in a museum. ldquoIt is a support system which can be suitable individually in a learner's diversity and situation.rdquo The loudspeaker array system can form the high-sound pressure area. This area is called sound spot. The loudspeaker array system is a noncontact sound interface. In this paper, the effectiveness of this system is verified for science education. The two experiments are conducted. First experiment, the effectiveness of sound information and paper information are compared. The comparison between explaining exhibition by texts and sounds is performed at the museum. We search the effective of explaining by letters and sound by questionnaire and quiz of examinees. Second experiment, the ease of listening is compared by the loudspeaker array system and a loudspeaker. We search the effective of ease of listening by questionnaire. In all the experiments, elementary school students are targets. A participant is a sixth grader from a third grader.

This paper presents a design concept of a novel 6-axis force sensor with simultaneous plantar deformation measurement by high-speed cameras under the sensor is presented. The sensor is based on two custom beams and some strain gages in order not to be limited a view of the high-speed cameras. In walking analysis research field, the 6-axis force and plantar skin deformation are very important factors to identify the risk of fall and to reduce risk of fall-related injuries in elderly populations. However, there were no sensor can measure 6-axis force and plantar skin deformation simultaneously. In this paper, the design of the sensor is described and perpendicular axis calibration results are shown. Furthermore, the walking experimental results show that the development sensor can be used for walking analysis.

This paper describes the Sound spot generation and evaluation by a large scale panel loudspeaker array. The authors research the sound spot by loudspeaker array. The sound spot is local high sound pressure area. We develop 128 ch panel loudspeaker array and 512 ch panel loudspeaker array to generate the sound spot. The 512 ch panel loudspeaker array is using four the 128ch panel loudspeaker array systems, and is surrounded by the square form. The authors simulate the sound spot generated by the 128 ch panel loudspeaker array and the 512 ch panel loudspeaker array. From the result of the simulation, the 128 ch panel loudspeaker array can generate straight line sound spot and the 512 ch panel loudspeaker array can generate spherical sound spot. Actual measurement of sound spot is performed by using 64 ch microphones measurement system. We confirm that each panel loudspeaker array generate three dimensional sound spot from result of measurement. This means that communication of voice information to many people is possible.

This paper reports on the development of a mobile robot system for operation within a house equipped with a ubiquitous sensor network.Human robot interaction is achieved through the combination of on-robot audio and laser range sensing and additional audio sensors mounted in the ceiling of the ubiquitous environment. The ceiling mounted microphone arrays can be used to summon a mobile robot from a location outside the robot's range of hearing. After the robot autonomously navigates to the desired location, the on-board microphone array can be used to locate the sound source and to recognise a series of greetings and commands.

This paper describes a mobile robot for home service purpose together with ubiquitous ceiling ultrasonic locator and microphone array. User call can be detected by those ubiquitous devices and then mobile robot navigates toward given location. After reaching to the given location, the robot tries to find out user location by using stereo camera, laser and onbody microphone array. We implemented those system at our experimental house "Rokko Holone". System components and experimental results are shown.

This paper describes a path smoothing method that results in paths with clothoidal curvature and which includes a safety margin from obstacles. The method modifies grid based A* results, and tries to push the given path away from obstacles. A technique to take into account the initial direction of robot is also shown.

This paper describes the three-dimensional simulation and measurement of the sound pressure distribution generated by 120 ch plane loudspeaker array. The authors research the local high sound pressure distribution generated by loudspeaker array. We develop the 120 ch plane loudspeaker array. The 120 plane loudspeakers are arranged at the 120 ch plane loudspeaker array. The authors simulate the three dimensional sound pressure distribution generated by the 120 ch plane loudspeaker array. From the result of the simulation, the 120 ch plane loudspeaker array can generate three dimensional sound pressure distribution. The 120 ch plane loudspeaker array is developed based on the simulation result. Actual measurement of sound pressure distribution is performed with 64 ch microphones measurement system. We confirm that the 120 ch plane loudspeaker array can generate three dimensional local high sound pressure area from result of measurement. In addition, result of simulation is compared with result of measurement. The simulation is available for the simulation of the 120 ch plane loudspeaker array from result of comparison.

This paper describes a method of measuring a plantar skin contact area and a plantar incipient partial slip in order to clarify the relation between a plantar tactile sensibility and a stabilization mechanism for slip. The information of the plantar sense stimulus during walk had been not acquired in the conventional walk analysis. The meaning of plantar slip measurement is explained and the plantar skin deformation is measured by the image processing using the developed measurement device. An increase of the plantar skin contact area by using ice immersion approach suggests the relation between plantar tactile stimulus and gait pattern.

This paper introduces a person following shopping cart robot. The developed prototype robot is using a shopping cart as the robot's main body and attaching a driving module. The aim of this study is to develop a porting system for large-sized facilities (i.e. shopping mall, airport, etc.), where the object item is autonomously carried by a person following ("after you") porting robot. We believe such study on porting robots is highly practical and necessary in today's aging society. By using the computer's signal output for autonomous run and the manually control function for risk aversion, a compatible control with automatically and manually control is realized. The structure and the principle of control system are described

The paper describes a 2D sound source mapping system for a mobile robot. We developed a multiple sound sources localization method for a mobile robot with a 32 channel concentric microphone array. The system can separate multiple moving sound sources using direction localization. Directional localization and separation of different pressure sound sources is achieved using the delay and sum beam forming (DSBF) and the frequency band selection (FBS) algorithm. Sound sources were mapped by using a wheeled robot equipped with the microphone array. The robot localizes sounds direction on the move and estimates sound sources position using triangulation. Assuming the movement of sound sources, the system set a time limit and uses only the last few seconds data. By using the random sample consensus (RANSAC) algorithm for position estimation, we achieved 2D multiple sound source mapping from time limited data with high accuracy. Also, moving sound source separation is experimentally demonstrated with segments of the DSBF enhanced signal derived from the localization process

We have developed the person following robot "ApriAttenda(TM)". This robot can accompany a person using vision based target detection and avoid obstacles with ultrasonic sensors. The robot first identifies an individual with its image processing system by detecting a person's region and recognizing the registered color and texture of his/her clothes. Our newly developed algorithm allows the robot to extract a particular individual from a cluttered background, and to find and reconnect with the person if it loses visual contact. Tracking people with vision was realized by systematizing visual and motion control with a robust algorithm that utilizes various characteristics of the image data. ApriAttenda(TM) has been exhibited at Aichi EXPO 2005, and its robust functions and smooth person following capability were successfully demonstrated.

We have developed the person following robot "ApriAttendatrade". This robot can accompany a person using vision based target detection and avoid obstacles with ultrasonic sensors while following the person. The robot first identifies an individual with its image processing system by detecting a person's region and recognizing the registered color and texture of his/her clothes. Usually, the person following robot has to detect and recognize the specified person and calculate his/her position in a complicated real-life environment of fixed objects and moving people. Our newly developed algorithm allows the robot to extract a particular individual from a cluttered background, and to find and reconnect with the person if it loses visual contact. Tracking people with stereo vision was realized by systematizing visual and motion control with a robust algorithm that utilizes various characteristics of the image data. The developed algorithm uses several analyses to extract information on the distance to each feature point, speed of target, color and texture of clothes for a stable tracking in many situations including changes of view due to self motion, shifts in lighting and objects similar to the target. The person following robot ApriAttendatrade has been exhibited at Aichi EXPO 2005, and its robust functions and smooth person following capability were successfully demonstrated

This paper describes a method of measuring the incipient slip of the sole in order to clarify the relation between a plantar tactile sensibility and a stabilization mechanism for a slip. The incipient slip measuring device for walking analysis by using high-speed cameras and acceleration sensor is developed. The measurement device can be measured the plantar skin deformation during a walk. The incipient slip is confirmed by using the developed device.

This paper proposes a learning-by-doing (LBD) system for assisting learners of English as a foreign language, by providing educational content generated based on real-world behaviors in a sensorized environment. The LBD system consists of an ultrasonic location sensing system developed by the authors and an educational content server. The location of the student is monitored by ultrasonic sensors embedded around the learning room. The system voices English verbs related to objects around the student so that the student can obtain the language skills for expressing various activities of everyday life in English by performing real-world behaviors. The system was implemented experimentally in an English language school. Six subjects participated in an experiment evaluating the educational efficacy of the proposed system. Recall rates of learned verbs were compared among three methodologies, "learning by text," "learning by pictures and text," and "learning by doing". Recall rates were measured at the following intervals: soon after the learning task, 2 days after, 4 days after, 7 days after, and 14 days after the learning task. The results indicate that the proposed LBD system increased the recall rate of learned words by 113% 14 days after the learning task. Overall, the system significantly increased the average recall rate by 102% in comparison to the other learning methodologies.

This paper describes a 32 /spl times/ 32 matrix scan type high-speed pressure sensor for the feet of humanoid robots that has 1 kHz sampling rate. This sensor has matrix scan circuit. The matrix scan method has a problem of interference by bypass current. To resolve this problem, we suggest a novel method using a very thin conductive rubber. We adopted a very thin (0.6 mm) force sensing conductive rubber sheet for high speed sensing. Each sensing area is 4.2 /spl times/ 7.0 mm and can measure vertical force of approximately 0.25-20 N. Walking cycle of humanoid robot as well as human being is about 0.4-0.8 s and dual leg phase is about 0.1-0.15 s. The target of the sensor is biped walk stabilization so that high-speed input is important. Matrix scan type circuit is connected to sensor, and the system runs 1 kHz with 14 bit resolution at 4.2 /spl times/ 7.0 mm grid for 32 /spl times/ 32 points, and the sensor size is the same as humanoid robot foot 135 /spl times/ 228 mm, The system is running high-speed because of the very thin conductive rubber and simultaneous measurement. The sensor system, a novel scan method, and evaluation results are described.

This paper describes a three ring microphone array estimates the horizontal/vertical direction and distance of sound sources and separates multiple sound sources for mobile robot audition. Arrangement of microphones is simulated and an optimized pattern which has three rings is implemented with 32 microphones. Sound localization and separation are achieved by delay and sum beam forming (DSBF) and frequency band selection (FBS). From on-line experiments results of sound horizontal and vertical localization, we confirmed that one or two sounds sources could be localized with an error of about 5 degrees and 200 to 300 mm in the case of the distance of about lm. The off-line experiments of sound separation were evaluated by power spectrums in each frequency of separated sounds, and we confirmed that an appropriate frequency band could be selected by DSBF and FBS. The system can separate 3 different pressure speech sources without drowning out.

The purpose of this paper is to report the devised arrangement of a microphone array suitable for a mobile robot and to develop a robotic audition system to recognize the environment. The paper first describes the sum and delay beam forming (SDBF) algorithm and its common problem: side lobes. The array we developed shows smaller side lobes when beam forming. It provides high quality localization and separation for multiple sound sources. Then we achieved a sound sources mapping system by using a wheeled robot equipped with the microphone array. The robot localizes sound direction on the run and estimates sound positions using triangulation. Accumulation of data results in high accuracy. The system can estimate 3 different pressure sounds with a 200 mm position error. Moreover, the high quality sound source separation has proved useful in improving speech recognition.

This work describes two circular microphone arrays and a square microphone array which can be used for sound localization and sound capture. Sound capture by microphone array is achieved by sum and delay beam former (SDBF). A dedicated PCI 128-channel simultaneous input analog-to-digital (AD) board is developed for a 128 ch microphone array with a maximum sampling rate of 22. 7 /spl mu/s/sample. Simulation of sound pressure distribution of 24 and 128 ch circular microphone array and 128 ch square microphone array are shown. Then a 24 ch circular microphone array and a 128 ch square microphone array have been developed. The 24 ch circular microphone array can capture sound from an arbitrary direction. The 128 ch square microphone array can capture sound from a specific point. Both systems are evaluated by using frequency components of the sound. The circular type system can be used on a mobile robot including humanoid robot and square type can be extended towards room coverage type application.

This paper describes a method for robustly detecting and efficiently recognizing daily human behavior in the real world. The proposed method involves real-world sensorization using ultrasonic tags to robustly observe behavior, real-world virtualization to create a virtual environment by modeling real objects using a stereovision system, and virtual sensorization of virtualized objects in order to quickly register the handling of objects in the real world and efficiently recognizing specific human behavior. A behavior-to-speech system created based on this recognition method is also presented as a new application of this technology.

This paper proposes a system for quickly realizing a function for robustly detecting daily human activity events in handling objects in the real world. The system has three functions: 1) robustly measuring 3D positions of the objects; 2) quickly calibrating a system for measuring 3D positions of the objects; 3) quickly registering target activity events; and 4) robustly detecting the registered events in real time. As for 1), the system realizes robust measurement of 3D positions of the objects using an ultrasonic 3D tag system, which is a kind of a location sensor, and robust estimation algorithm known as random sample consensus (RANSAC). The paper evaluates the robustness by comparing RANSAC with a least-squares optimization method. As for 2), the system realizes quick calibration by a calibrating device having three or more ultrasonic transmitters. Quick calibration enables the system to be portable. As for 3), quick registration of target activity events is realized by a stereoscopic camera with ultrasonic 3D tags and interactive software for creating 3D shape model, creating virtual sensors based on the 3D shape model, and associating the virtual sensors with the target events. The system makes it possible to quickly create object-shaped sensors to which a new function for detecting activity events are added while maintaining the original functions of the objects.

This paper presents a method of visually steerable sound beam forming. The method is a combination of face detection and tracking by motion image processing and sound beam forming by speaker array. Direction towards a target person can be obtained by the face tracking in real-time. By continuously updating the sound beam direction with the result of the face detection and tracking, the system is possible to keep transmitting sounds towards the target person selectively, even if he or she moves around. Experimental results prove the feasibility and effectiveness of the method.

This paper describes sound beam generation and capture by speaker array and microphone array system. 16ch online speaker array system generates sound beam towards appropriate directions. 128ch square speaker array system generates sound spot towards appropriate positions. Since it transmits necessary sound information to only a specific person, it can be used in quiet places, such as a museum. 8ch online microphone array system captures sound from appropriate direction. It can be used as ears of a humanoid robot. The authors call a position to want to generate/capture sound "focus". Focusing is achieved by sum and delay beam former. (SDBF) More than two focuses can be simultaneously generated/captured by speaker array/microphone array. Speaker array system is evaluated by using gantry and sound level meter that are controlled by PC system. Microphone array system is evaluated by using frequency ingredient of the sound.

This paper presents a working prototype system of a novel computer-human interface, named invisible messenger. It integrates visual face detection and tracking, and speaker array signal processing. By speaker array it is possible to form acoustic focus at the arbitrary location that is measured by the face tracking. Thus the implemented system can whisper in a person's ear as if an invisible virtual messenger were standing by the person. In this paper, the authors describe both ideas behind the system and key technology for the implementation. In order to confirm the effectiveness of the system, the authors conduct experiments using it. This paper also reports the experiment. Experimental remits demonstrate the effectiveness of the proposed idea.

Nakadai et al. (2001) have developed a real-time auditory and visual multiple-talker tracking technique. In this paper, this technique is applied to human-robot interaction including a receptionist robot and a companion robot at a party. The system includes face identification, speech recognition, focus-of-attention control, and sensorimotor task in tracking multiple talkers. The system is implemented on a upper-torso humanoid and the talker tracking is attained by distributed processing on three nodes connected by 100Base-TX network. The delay of tracking is 200 msec. Focus-of-attention is controlled by associating auditory and visual streams by using the sound source direction and talker position as a clue. Once an association is established, the humanoid keeps its face to the direction of the associated talker.

Linear discriminant analysis (LDA) is applied to broad areas, e.g. image recognition. However, successive learning algorithms for LDA are not sufficiently studied while they have been well established for principal component analysis (PCA). A successive learning algorithm which does not need N/spl times/N matrices has been proposed for LDA (Hiraoka and Hamahira, 1999, and Hiraoka et al., 2000), where N is the dimension of data. In the present paper, an improvement of this algorithm is examined based on Sanger's (1989) idea. By the original algorithm, we can obtain only the subspace which is spanned by major eigenvectors. On the other hand, we can obtain major eigenvectors themselves by the improved algorithm.

Convergence of a matrix dynamics for online LDA is analyzed. Especially, stable spurious solutions are pointed out and two schemes to prevent the spurious solutions are proposed. The performance of the algorithm is confirmed by simulations of face identification.

Proposes a face detection method intended to be used for a practical intelligent environment and human-interactive robot. Face detection and recognition are very crucial for such applications. However in the real situation, it is not easy to realize the robust detecting function because position, size, and brightness of the face image are very changeable. The proposed method solves these problems by combining correlation-based pattern matching, histogram equalization, skin color extraction, and multiple scale images generation. The authors have implemented a prototype system based upon the proposed method and conducted some experiments using the system. Results support the effectiveness of the proposed idea.

This paper proposes a novel human symbiosis robot named Expressive Mobile Robot. Through its behavior and movement it successfully realize nonverbal communication to humans. Towards the future aging society human-robot symbiosis is keenly needed. To realize such human-robot symbiosis, familiarity of robot is crucial. The proposed Expressive Mobile Robot produces familiar impression to humans through its behavior, action and motion. It does no real task other than the expression. Based upon consideration of human gestures, a prototype of the Expressive Mobile Robot has been designed and implemented. It is puppy-sized and has 7 DOF with a head, two arms and a body. Utilizing the prototype robot, several psychological experiments are conducted. Experimental results make it clear that there are proper speed pattern, distance to human and poses for familiar impression. The results support the feasibility of the proposed idea and robot.

Human activity support is being established as a novel active area for robotics. It is expected to make clear the mechanism of understanding situation and intention of humans through regarding behavior as media and the mechanism of human support based on it. This paper shows a method to support handiwork by recognizing human action conforming to situation and shows a soldering support tool constructed based on the method as an example.

This paper proposes an idea that it is possible for a mobile robot to display behavioral expressions by its motion. Behavioral expressions are expressions of vividness, sense of distance and attention. To confirm the idea concretely, an expressive mobile robot has been designed and implemented to display the behavioral expressions. Utilizing the robot, psychological experiments have been conducted to evaluate impressions on three items: 1) velocity changing pattern, 2) distance between human and the robot, and 3) various poses. The experimental results indicate: firstly, there is a proper speed pattern for expression of vividness, the pattern being triangular along the time axis; secondly, there is a proper distance range between human and robot for expression of mental distance between them, its average value being about 2.5 m; thirdly, when the robot faces the human, the impression of attention is increased where the robot puts its head on one side or raises its hands. The implemented expressive mobile robot is puppy-sized and has 2 DOFs for motion, 2 DOFs for two swingable arms and 3 DOFs for pan, tilt and yaw of its head. The experimental results prove feasibility of the proposed idea of the behavioral expression by the robot.

To support human activities is being established as a novel activity area for robotics. The mechanism to understand situation and intention of human from her/his behavior is expected to be made clear. Action recognition method will be indispensable to realize the mechanism. In this paper, we describe a system which finds human in a scene and tracks the human using vision to recognize actions. The proposed system is network oriented which distributes a lot of fixed stereo pair cameras, and tracks human as a whole. The constructed vision system utilizes skin-color extraction and template matching based tracking. Human action recognition experiment using the system is shown.

This paper proposes a function for monitoring a patient's respiration and posture in sleep using a human symbiosis system in a sickroom. The novel features of the system function lie in noninvasive and unrestrained monitoring to ensure the symbiosis with the human. Noninvasive monitoring eliminates the need for monitoring needles or catheters to invade the patient's body; sensors thus do not impose a physiological burden such as pain on the patient. In unrestrained monitoring, sensors and their electrical cords do not limit degrees of freedom of the patient's movement. Unrestrained sensing therefore does not impose a psychological burden caused by the limitations on the patient. A "robotic bed", which is a system to realize the functions stated above, is a bed-shaped system with 221 pressure sensors for monitoring the patient's respiration and posture without preventing doctors and nurses from performing their tasks. The sensors are set 5 cm from each other on the bed and surround the patient. Experiments to monitor respiration and posture in sleep demonstrate that the proposed function is feasible for monitoring for over 6 hours.

This paper proposes a novel human-robot symbiosis system, named a robotic office room (ROR). The understanding, recording and retrieving of telephone call behavior are successfully demonstrated as a typical example of an office work-supporting function of ROR. A unique object model is proposed, which accumulates information concerning motion of the object. This model enables the system to understand human behavior by monitoring the motion of the object-typically, a telephone or pen. Since the model requires the system to pay spatial and temporal attention only to the target object and related motions, it saves computing power. Repertoires of understandable behavior can easily be expanded by simply adding new object models. The ROR is equipped with networked machines. The concept of "ubiquitous link" consisting of unique network image links and remote control links is proposed. When a human starts to talk on the phone, the ROR not only starts to record the image of the human's behavior, utilizing a VCR, but also turns down the volume on an audio set. The ROR is experimentally proved to be a unique distributed robotic system to support human office work.

This paper proposes a robot behavior notation named "Behavioral Score" and a methodology of generating "familiar" robot behavior. Behavioral Score, designed with analogies to a musical score, is an effective method of describing and controlling the expressive behavior of a robot. The effectiveness of Behavioral Score is demonstrated by a choreographed robot dance. The methodology described is based upon the hypothesis that the passive reactions of a robot produce a familiar, comfortable, imprecision in human beings. Utilizing the robot behavior for generating familiarity will facilitate human-robot symbiosis. The results of psychological experiments verifies the validity of the methodology.

Understanding human intention is an essential function for a robot which can offer adequate support to human beings. It requires smooth communication between the human and the robot. Human behavior is an expressive media of communication. This paper proposes a new function of "active under standing of human intentions" by a robot through monitoring of human behavior. The unique feature of the proposed function lies in the fact that it utilizes multi-communication channels in parallel. i.e., human intentions are understood not only through conscious behavior but also through unconscious behavior. The paper also proposes a robot architecture to realize the function. The following points are the key features of the architecture: a robot possesses multi-sensors which surround the human; and information processing is carried out by dual loops-a loop for information exchange between the human and the robot and a loop for human intention understanding. As an example of a robot with human intention understanding functions, the authors constructed a micro-teleoperation robot. It can automatically understand an operator's intention through such unconscious behavior of touching a desk with an operator's hand. The understood result is utilized to change the control mode of a master-slave manipulation system from fine motion to rough motion and vice versa. The experimental results prove that the proposed function is effective in making the operation of the system easier. Consequently the system is friendly to the operator.>