As people age, walking can become difficult. This difficulty in walking can lead to further physical decline and eventually result in the need for caregiving. To prevent this, wearable robots supporting walking rehabilitation have been developed. In this study, we aim to develop a modular robot wear that can be customized according to the user’s needs. The modular robot wear consists of motor modules, sensor modules, and a processor. Motor modules can be attached to any part of the body that requires assistance, enabling support not only in the sagittal plane but also in the frontal plane. By calculating the relative movement of the center of gravity based on the information from the acceleration sensors in the sensor modules, commands to the motor modules are generated. This allows for assistance tailored to the user’s walking pattern. Through verification experiments of the robot wear’s operation, we confirmed its ability to provide support according to changes in walking patterns. However, while the robot wear can induce changes in walking patterns, there are challenges regarding the output it provides to the wearer.

With recent technological improvements, the performance of conversational robots such as context-aware dialogue is expected to be dramatically enhanced. While context-aware utterance is considered one of the crucial functions of conversational robots, only a few studies investigate the relationship between context awareness and user preference. As is the case with human social relationships, human-robot interactions are under the effect of user’s attributes such as personality, and thus considering these factors of users is essential to continue interacting with robots. In this paper, we evaluate the preference for context awareness based on personality trait effects in an open-domain dialogue using a conversational android robot system. Two types of utterances, context-aware and context-free, are implemented in the robot system, and the participants talk with and evaluate our conversational system. The preference for two systems and the personality traits of each participant are collected. We analyze the relationship between preferences and personalities. The results suggest that the participants’ preference for whether an utterance is context-aware or context-free is distinguished by some personality traits.

One of the major tasks of assistive robots for future domestic applications is to bring objects requested by the user, which is called a “Bring-me Task.” Many of those previous studies including the Bring-me Task focus on the technologies to perform the task autonomously. However, object recognition, grasping, and navigation on autonomous robots are still difficult and not yet practical in a real-world unstructured environment. As an alternative approach, Bring-me Tasks by teleoperated robots have also been studied, but such systems require remote operators to perform the task for the user. In this paper, we propose a novel approach to the Teleoperated Bring-me Task in which a user teleoperates a robot by him/herself and receives an object from the robot. The user wears an HMD showing a mirror-reversal image from the viewpoint of the robot. The results from the experiments on the Teleoperated Bring-me Task suggest that the proposed interface with the mirror-reversal image improved the performance of receiving the object by hand compared with the original image. Furthermore, it was also confirmed that our interface improved the operability of the robot. (A video introducing the proposed method can be viewed11https://youtu.be/Qwrj8tIK_lI?si=CgjGGOC4IZzgO-JC.)

Technologies for estimating self-position and orientation are important for both humans and robots. These technologies allow robots to perform tasks such as carrying objects and allow people to reach their destinations. Although self-position estimation technologies using GPS and laser rangefinders have been developed, few methods can be used by both humans and robots. Therefore, we developed a method that can estimate three-dimensional position and orientation using visual markers and an inertia measurement unit (IMU). Self-position can be measured with high accuracy by using a visual marker and monocular camera, but such measurement data is discrete and sparse. In contrast, an IMU can continuously measure acceleration data, but data obtained from an acceleration sensor are double-integrated, which increases position error. By combining visual marker and IMU information, position error calculations based on the acceleration sensor can be corrected, and the movement path of the object can be estimated. In demonstration experiments, the proposed method accurately estimates the three-dimensional movement distance when a person walks about 13 m, with an average error of about 40.3mm.

Automatic control of mobile robots and robot arms requires techniques for estimating the position and orientation of objects with high accuracy and robustness. Although methods using markers or machine learning have been developed, general-purpose and highly accurate estimations have not been realized. Our team developed a high-accuracy visual marker “LentiMark” for high-accuracy estimations of position and posture, but data are lost when the camera cannot obtain marker images. We therefore developed the Marker-IMU system for integrating visual markers with an inertia measurement unit (IMU). When cameras cannot acquire the image of a visual marker, any missing data are restored from IMU data. However, when calculating positions from acceleration sensor values, sensor error increases estimation error. Therefore, we developed a method for error correction using measurements from before and after the missing data. Evaluation experiments confirm that missing data can be estimated using the proposed method.

The recent demographic trend across developed nations shows a dramatic increase in the aging population, fallen fertility rates and a shortage of caregivers. Robotic solutions to clothing assistance can significantly improve the Activity of Daily Living (ADL) for the elderly and disabled. We have developed a clothing assistance robot using dual arms and conducted many successful demonstrations with healthy people. It was, however, impossible to systematically evaluate its performance because human arms are not visible due to occlusion from a shirt and robot during dressing. To address this problem, we propose to use another robot, Whole-Body Robotic Simulator of the Elderly that can mimic the posture and movement of the elderly persons during the dressing task. The dressing task is accomplished by utilizing Dynamic Movement Primitives (DMP) wherein the control points of DMP are determined by applying forward kinematics on the robotic simulator. The experimental results show the plausibility of our approach.

This paper describes the development of a whole-body robotic simulator of an elderly person for evaluating robotics devices for nursing care. To improve the quality of life of the elderly persons, physical assistance such as transfer, movement, and bathroom assistance is important. It is also important to reduce the workload of caregivers in an aging society. In recent years, assistive robotic devices for nursing care have been developed and commercialized for such purposes. However, such devices have not become popular in the care facilities yet. One of the reasons is that it is still difficult to evaluate the effects of the devices on the care receivers and caregivers. In particular, it is necessary to quantitatively evaluate the effect of the devices on the human body from the viewpoint of safety and comfort. We have developed a whole-body robotic system to simulate the pose and motion of the elderly persons. The purpose of this system is to realize quantitative physical evaluation of robotics devices for nursing care of the human body. The experimental results of the preliminary evaluation of assistive robotic devices are also presented.

Visual markers are useful tools that aid autonomous robots in object recognition and pose measurement; however, conventional markers have two fundamental problems in orientation estimation: (1) inaccurate estimation in the frontal direction and (2) pose ambiguity. We previously developed a visual marker “LentiMark” that solves problem (1). In this paper, we propose a solution to problem (2) by improving LentiMark. For this purpose, we use two wavelike slices that change two-tone patterns according to the angle of visual line. The new marker enables stable and robust pose estimation without pose ambiguity. We solve the two biggest problems of visual markers in the framework of small planar markers. We verify the efficiency of our method through experiments.

We conducted a pilot experiment to examine the potential of a communication robot for supporting verbal interaction among adolescents with autism spectrum disorders (ASD) and their caregivers. Three teenagers with ASD indirectly conversed with their teacher via a desktop humanoid robot for around 40 min on average per day. In this situation, they showed some positive tendencies such as non-echolalic responses and talk about problems related to human relationships; they had not shown such tendencies before when they directly talked with their teacher in their daily life. During the trials, they also showed changes in their attitudes toward their teachers and friends; this implies that individuals with ASD could use their experience of communication with a robot for communication with a human. A quantitative analysis of the utterances in the trials suggested that reduced utterances from a caregiver might positively influence individuals with ASD toward sharing more information.

This paper reports the results of a survey on introducing robots and other new technologies to care services, such as care for older adults. Demographic data show that the ratio of older adults in Japan has already exceeded 23.3%. Japan also is facing a severe shortage of the working-age population, which could lead to a reduction in the quality of care services for older adults; therefore, robots and the provision of new forms of information technologies are expected to be utilized in such care services. The purpose of this study is to help facilitate the introduction of novel technologies in care services, such as in long-term care facilities. We focus on information-related costs that are incurred prior to the purchase of technologies such as attending exhibitions and conducting Internet searches. This report shows the results of an investigation into such information-related costs for three different products, and discusses the importance of the appropriate management of information costs from the development phase through to the introduction phase of such products.

For mobile robots, 3D acquisition is required to model the environment. Particularly for humanoid robots, a modeled environment is necessary to plan the walking control. This environment can include both static objects, such as a ground surface with obstacles, and dynamic objects, such as a person moving around the robot. This paper proposes a system for a robot to obtain a sufficiently accurate shape of the environment for walking on a ground surface with obstacles and a method to detect dynamic objects in the modeled environment, which is necessary for the robot to react to sudden changes in the scene. The 3D acquisition is achieved by a projector-camera system mounted on the robot head that uses a structured-light method to reconstruct the shapes of moving objects from a single frame. The acquired shapes are aligned and merged into a common coordinate system using the simultaneous localization and mapping method. Dynamic objects are detected as shapes that are inconsistent with the previous frames. Experiments were performed to evaluate the accuracy of the 3D acquisition and the robustness with regard to detecting dynamic objects when serving as the vision system of a humanoid robot.

The ultimate role of assistive robots is not only supporting user's physical functions but also improving the quality of user's life. Designing such robots is not so easy, because most of the developers do not know about the real “effect” of assistive robots on the users' lives. We propose a concept design method of assistive robots which focuses on the effects of robots on the entire life of human. We introduce the concept of ICF (International Classification of Functioning, Disability and Health) in order to model the interactions among many factors in human activities. The developer uses the models as design knowledge, and defines the requirements along the use scenarios of the assistive robots. We utilize SysML (Systems Modeling Language) in writing the knowledge and scenarios. We developed a prototype of design support software which implemented the design method. In this paper, we describe the concept and the details of our method using some case examples of a mobility assistance robot, and present the future direction.

In order to let assistive robots be utilized in daily lives, it is important to analyze the daily life of users before development, and to enable users to understand the benefit of the robot after development. In this paper, we focus on the robotic arms for daily support, and firstly show the analysis of daily life in the viewpoint of “hand use.” The specification of robotic arms such as the DoF and payload can be determined based on the analysis. Then we show how to describe the benefit of robotic arms. A prototype of database for assistive device is created to show the benefit of assistive robots together with other information such as who can use, how to use, where to use. We utilize ICF approved by the WHO for such analysis and description.

Motion analysis of human body is indispensable for design and evaluation of assistive technologies. Especially, on-site analyses, which are conducted not in experimental environments but in life circumstances, are important to develop truly beneficial assistive technologies. We developed a portable motion tracking system using high-accuracy AR markers. It enables easy measurement of 6-DOF (six degrees of freedom) motion of multiple tracking points by a single camera. The pose estimation errors are about 5 [mm] in location (depth) and about 2 [deg] in orientation. We applied the prototype of the motion tracker to the on-site motion analysis tasks, and demonstrated its availability in the evaluation of assistive technologies.

Visual markers are useful tools assisting visual recognition of object pose in robotic applications. But they have two fundamental problems in orientation estimation. One is degradation of orientation accuracy in frontal observation. The other is “pose ambiguity” that the orientation cannot be determined uniquely. We previously developed a novel visual marker “LentiMark” which solves the former problem. This time we propose a solution to the latter problem by improving it. We utilize two new moiré patterns to get unique angle information and eliminate the pose ambiguity. The new marker enables stable and robust pose estimation without suffering from the pose ambiguities. The contribution of this study is that we solved the biggest two problems of visual markers, in the framework of small planar markers. We demonstrated the availability of our method through some comparative experiments.

This paper presents a basic study on feasibility of usage of humanoid robots as an evaluator of assistive devices, by taking advantage of its anthropomorphic shape. In this new application humanoid are expected to help evaluation through quantitative measures, which is difficult with human subjects, and also to reduce the burden coming from ethical concerns with costly tests by human subjects. Taking a passive supportive wear “Smart Suit Lite” designed to relieve the load at lower back as an example, we have conducted pilot experiments by using the humanoid robot HRP-4C. The motion to be performed by the humanoid is obtained through retargeting technique from measured human lifting motion. The supportive effect is first estimated by simulation taking into account the mechanism of the supportive device. The experimentation of humanoid hardware brought us encouraging results on the basic feasibility of this application, as we observed a clear decrease of the torque for lifting when wearing the device as expected by the simulation.

There are body-powered hooks and myoelectric prosthetic hands that trans-radial amputees can use for work. Though the body-powered hooks have good workability for complex operations, the design of the hook is unappealing and the harness is cumbersome. The myoelectric prosthetic hand has a natural appearance similar to the human hand and intuitive operability using a myoelectric control system. However, it is expensive and heavy. Because of these problems associated with prostheses for work, many amputees use cosmetic prostheses. In this paper, we report a lightweight, low-cost electric trans-radial prosthesis with three opposed fingers. A simple mechanism to control the fingers by a linear actuator contributes to its good workability, lightweight, and low-cost. An operation system using an inexpensive distance sensor allows intuitive operability equivalent to the myoelectric control system. A socket makes the prosthesis easily removable. The total weight of the hand and socket is 300 g, and both can be produced with a 3D printer. An evaluation using the Southampton Hand Assessment Procedure (SHAP) demonstrated that an amputee was able to operate abstract objects which require six types of grasps with the developed prosthesis.

Fiducial markers (AR/visual markers) are still useful tools in many AR/MR applications. But conventional markers have two fundamental problems in orientation estimation. One is degradation of orientation accuracy in frontal observation. The other is “pose ambiguity” where the estimated orientation repeats switching between two values. We previously developed a novel marker “ArrayMark” which uses a microlens array and solved the former problem. This time we propose a practical solution to the latter problem by improving the ArrayMark. We attach an additional reference point to detect the occurrence of invalid estimation, and modify the orientation of the marker by inverting the zenith-angle of the visual-line. This marker enables stable pose estimation from a one-shot image without using any filtering techniques. The method is applicable to conventional markers, too. We demonstrated the availability of this improvement to the pose-ambiguity problem.

In order to develop useful assistive devices, analyses and understanding of human functions are necessary. We recorded the activities of daily living, and classified the usage of both hands using the vocabulary of ICF (International Classification of Functioning, Disability and Health) approved by WHO. We confirmed that the usage of the non-dominant hand is auxiliary and greatly different from that of the dominant hand. This indicates a possibility of a reasonable design of assistive devices, e.g. a prosthetic hand for non-dominant hand which has fewer functions but is cheaper than that for dominant hand.

Visual marker is a useful assistive tool for service robots. Existing planar visual markers have poor accuracy and stability in pose estimation, especially in frontal direction. In this study, we developed a novel visual marker based on a new principle enabling accurate and stable pose estimation even by observation from frontal direction. The marker has moiré patterns which consist of lenticular lens and stripe pattern, which vary their appearance according to visual-line angle of observation. We can extract pose information from the pattern by a single camera. We developed a prototype of the marker and an algorithm for pose estimation, and then demonstrated its superiority to existing markers by some validation tests.

Visual markers are useful assistive tool for service robots. Existing planar visual markers have poor accuracy in pose estimation, especially in frontal direction. We solved this problem by a novel principle using a microlens array. The marker displays a changing moiré pattern according to the visual-line direction. We can extract the pose information from the pattern by image processing using a single camera. The developed marker and the processing algorithm enable high-accuracy pose estimation even by observation from frontal direction. We verified its superiority to the conventional method by some validation tests. The marker is an extension of a marker we previously developed, and is more easily fabricated.

With the aging population trends in developed countries, there is an increased expectation of the application of assistive robots. A variety of robots have been developed to address this challenge. Although there are many technical problems to be solved, the lack of objective evaluation indexes for evaluating the benefit to the users is one of the important problems to be addressed before the commercialization of the assistive robots can be realized. The ICF (International Classification of Functioning, Disability and Health) is a classification of the health components of functioning and disability, and is being utilized in medical, healthcare, and welfare fields. The ICF has a basic framework to evaluate healthcare and welfare equipment, including robots, and it will be the basis of the evaluation indexes for assistive robots. In this paper, we present the development of evaluation indexes, and provide the results of preliminary experiments which were conducted using iARM and RAPUDA robots performing a certain lifting task. The experiments illustrate the issues involved in the development of ICF-based indexes for assitive robots.

Many hand motion recognition methods using Electromyogram (EMG) signals have been developed. Because most studies used only surface EMG signals from a forearm, available motion-related information was limited. In this paper, we report a SVM (Support Vector Machine) based hand motion recognition method using hybrid sensors. The hybrid sensor consists of an EMG sensor and an optical distance sensor, and can measure myoelectric activities and distance between the sensor and the skin surface at the same time. It is expected that distance changes caused by muscle elevation compensate the limited information derived from myoelectric activities. To examine the effectiveness of our method, we performed hand motion recognition experiments with four subjects. Experimental results showed that our method using hybrid sensors can improve motion recognition accuracy compared to when using only EMG signals.

This paper deals with a multiagent path-planning problem where several robots track humans to obtain detailed information on human behaviors and characteristics. For this, agents' paths are planned on the basis of the similarity between the predicted positions of humans and the agents' field of view. The long-horizon path planned on the basis of an accurate long-horizon prediction improves the tracking performance. However, it requires heavy computation and is less useful if the prediction is inaccurate. Since the accuracy of the prediction depends on the situation, the prediction term is determined by the similarity between the current and previous predictions. The results of computer simulation showed that our path-planning method works well for trajectories of humans in a dynamic environment by changing the horizon length of the path planning.

In this research, a Universal map, which can be converted to individual maps for heterogeneous mobile robots, is proposed. A Universal map can be generated using our developed measurement robot, and it is composed of a textured 3D environment model. Therefore, every robot can use a Universal map as a common map, and it is utilized for various localization technologies such as view-based and LRF-based methods. In LRF-based localization, accurate localization is achieved using a specific map, which is generated from Universal map. In a view-based approach, localization and navigation are achieved using rendered images. The use of a Universal map enables generation of these maps automatically. The effectiveness of this approach is confirmed through experiments.

Biological creatures, some of them are very simple, seem to perform efficient search strategy. Recent researches show that noise in their internal mechanism may have an important role to manage this behavior. This paper focuses on realizing a simple, noise utilizing, mathematical framework that enables a mobile robot to perform random search adaptively and efficiently under changing target density. Our approach is to model and implement bacterial movement based on a recent perspective of noise utilizing mechanism in living beings: biological fluctuation. As a result, the robot will adaptively switch its random search pattern between Levy walk and Brownian walk to increase the search efficiency in a patchy environment where the target density naturally alternates.

Improvements in hand pose estimation, made possible by refining the model matching step, is necessary in creating a more natural human-robot interface. Individualizing the 3D hand model of the user can result to a better hand pose estimation. This paper presents a way to accomplish the individualization by estimating the length of the finger links (bones), which is unique for every user. The 3D model of the hand is made up of voxel data derived from silhouette images obtained by multiple cameras and the finger link is estimated by searching a set of models generated from the calibration motion of the fingers. Initial pose estimation result using the model shows the feasibility of the system.

In human-robot interaction, it is important for the robot to know the head movement, gaze direction and expression of the conversation partner, since such information are deeply related with the attention, intention and emotion. Recently, many types of real-time measurement systems for head pose and gaze direction have been proposed and utilized for human interfaces and ergonomics applications. We have proposed a face measurement system which is non-contact and passive by utilizing a stereo camera pair. However the necessity of the use of stereo camera has several drawbacks. To cope with this problem, this paper proposes a method to measure the 6 DOF motions of the head by utilizing a single camera. The use of 3D face model enables us to predict six 2D motion vector fields of facial features which correspond to six unit motions in translation and rotation. An actual 2D motion vector field obtained by tracking facial features is then resolved into a linear combination of the predicted six motion vector fields. The estimated 6 DOF motions can be obtained as the six coefficients of the linear combination. Through the evaluation of the implemented motion estimation method, it was shown that the accuracy of the proposed estimation method using a single camera was comparable with that of the conventional method using a stereo camera pair. Finally developed face tracking system is demonstrated by a humanoid robot interacting with two persons, which takes advantage of the wide field of view of the camera system due to the unnecessity of the stereoscopy.

Many biologically inspired approaches have been investigated in relation with researches on mobile robot(s) that can effectively locate targets that induce gradient information. Here, we concentrate on realizing an adaptive searching behavior in mobile robot that is simple, yet effective with and without sensing the gradient information. We are interested in two searching behaviors found in biological creatures: bacterial chemotaxis, probably the simplest yet effective gradient sources searching behavior found in living creatures; and Levy walk, specialized random walks with fractal movement trajectories that optimize random search for sparsely and randomly distributed target(s). Our approach is to implement and combine the two searching behaviors based on ¿yuragi¿, a Japanese word for biological fluctuation.

Living organisms have various kinds of flexibility and robustness which are realized by ¿yuragi,¿ i.e. biological fluctuations or noises. Bacterial motion is a form of noise-based motion, since bacteria can move towards a higher concentration of some chemical which they prefer even though they have only a limited 1-DOF of flagella for mobility. Bacteria also have only a limited sensory device which cannot detect the spatial gradient of the chemical at a time. The simple strategies that bacteria take to realize chemotaxis are (1) to tumble (or turn) to change orientation randomly with unbundled flagella in various directions being hit by surrounding water molecules, and (2) to change the frequency of tumbling according as the time change of chemical concentration. In this paper, we describe a small and simple, 1-DOF swimming robot developed by mimicking the bacterial motion generation mechanism. The robot only has a single motor and a single sensor (a photo detector), however, by changing its orientation in response to various noises which exist in the environment, and by changing the frequency of turning, the robot can approach its goal. Experimental results indicate that the robot statistically approaches the goal (a light source) in two dimensional space with its 1-DOF actuator, which would be impossible for the robot to achieve without utilizing noises in the environment.

Recently, many humanoid robots have been developed and actively investigated all over the world in order to realize partner robots which can coexist in an environment shared with humans. having good communication skills is essential in order to interact naturally with humans. However, even with state-of-the-art interaction technology, it is still difficult for humans to interact with humanoid robots without conscious effort. In this paper, we use the android robot, which has an appearance which is quite similar to that of a human, as a bystander in human-human communication. The android is not explicitly involved in the conversation, however it makes small reactions to the behavior of the humans, and the psychological effects of its behavior on the human subjects are investigated. Through the experiments, it is shown that if the android mimics the behavior of the subject this can be quite effective in harmonizing the human-human communication.

We propose a view-based indoor/outdoor navigation method as an extension of the view-sequence navigation. The original view-sequence navigation method uses the template matching method with normalized correlation for localization. Because the matching method is sensitive to local illumination changes, it is only used for indoor environment. In this paper, we propose to adopt the accumulated block matching method to improve robustness against locally changing illumination, in which a template is split into small patches and matched by maximizing the average of the normalized correlations of all the patches.We also propose a localization criterion which helps the robot decide its motion. Our experimental results demonstrate that the proposed methods can be applied to both indoor and outdoor environments.

In this paper, we propose a method to estimate the group attention that is the focus of attention of multiple people. It utilizes the face direction as 3D vectors and estimates the position of group attention defined as the intersection of the multiple vectors. As a result, the position of the group attention can be represented as an arbitrary 3D position unlike other research in which only registered objects can be the focus of attention. As experiments, the group attention at word-chain game is estimated and the feasibility of the method is confirmed. We applied the proposed method to video conferencing, and developed the attention tracking system which acquires the group attention in real-time.

This paper addresses pose estimation of a hand in motion through a model-based vision-based system. Previous research on human hand tracking and pose estimation usually suffers from limited number of degrees-of-freedom estimated, camera orientation (i.e., the hand is restricted to a particular pose while moving) and occlusion. We describe and evaluate a system that allows several DOF of the hand to be estimated without hindering its motion and while minimizing occlusion. To allow for a complete 3D motion, a voxel model and a skeletal model of the hand are used. The system uses multiple viewpoint cameras to obtain information of the hand motion. Due to the non-linear characteristics of the system, Unscented Kalman Filter (UKF) is used to track the hand motion. UKF estimates the hand pose by minimizing the difference between the skeletal model and the voxel model. Estimation results from different hand motions of up to 15DOF show the feasibility of the system.

This paper proposes a multi-modal telecommunication system using a facial expression robot. We developed a telecommunication system which projects the facial expression of an operator to a remote place using the facial expression robot "Infanoid2." The facial information of the operator is measured using a stereo camera system and projected through a robot in order to communicate with another person in a remote location. Impression evaluation experiment is performed using this system. This paper discusses the effectiveness of robots as a telecommunication medium based on the experimental results.

Intelligent robots will make a chance for us to use a computer in our daily life. We implemented a humanoid robot for the computerized university guidance at first and then some capabilities for the natural interaction are added. This paper describes the hardware and software system of this humanoid with interaction ability. HRP-2 sitting opposite to a user across a table can detect gaze direction, head pose and gestures using a stereo camera system attached to the head. In addition, our system can recognize the user's question utterance and non-stationary noise such as coughing and sneezing using microphones. Using efficiently such information, HRP-2 can answer the question by its synthesized voice with gestures and do tasks such as passing objects on the table

Controlling devices using physiological signals is an increasingly active research field in human-robot interaction. Using brain signals, or Electroencephalograms (EEG), to control devices is widely known as Brain-Computer Interfacing (BCI). In a BCI system, brain signals are used to directly control devices and not as a communication channel, so target applications are restricted to handicapped and aged people. On the other hand, it is possible for healthy people to simply control devices using voice and body movements. In this paper, quantifying human's state of mind and emotions is investigated by analyzing changes in brain activity. The existence of a relation between robots' bodily expressions and their impressions on the observer is demonstrated. We also show a correlation between the nature of the expression and the power of the low-alpha channel in brain signals. Robots can make use of this quantification as an extra channel of information to achieve smoother and more human-like adaptive interaction

In this research, we propose a method for generating templates used for manipulation recognition that considers familiarization and fatigue. Our recognition system allows quantitative comparison of similarities among manipulations performed by observing contact state transitions on a palm surface. The system detects the contact states on the palm surface using an attached tactile sensor sheet on a manipulated object. In experiments, familiarization and fatigue are detected by measuring the variance of the manipulation. Experimental results indicate that manipulation can be divided into three periods: learning, familiarization, and fatigue. The results also indicate that variation of the manipulation becomes small when the template is generated by manipulation data during the familiarization period

During an interaction process, people usually adapt their behavior according to the interpretation of their partner's bodily expressions. It is not known how much similar expressions performed by robots affect a human observer. This paper explores this issue. The study shows a correlation between the nature of the bodily expressions, through the result of questionnaires, and the effect on brain activity. It has been demonstrated that unpleasant bodily expressions of the robot elicit unpleasant impressions and vice versa. This was observed through brain activity in a specific area when the expression is pleasant, and in another area when it is unpleasant.

In this paper, the CPG-based control method for the rotating manipulation that can adaptively change the issuing cycle of motion-triggers is proposed. The CPG model consists of the mutual inhibition network and the rotating manipulation is performed by the motion-triggers that are issued from the CPG model. The smooth change of the desired velocity can be given by the output of neurons that keep smooth increase and decrease. Experimental results using 4-fingered hand system suggest that the issuing cycle of the motion-triggers can be adaptively changed by the feedback of joint angle according to an object size.

A number of researches for the dexterous manipulation have been studied in robotics and physiologic fields independently. Multi-fingered manipulation for dexterous robot hands should be based on analysis of human skills. The purpose of this study is to apply CPG-based control to the dexterous manipulation. In this paper, CPG-based manipulation is proposed for the rotating manipulation of the dexterous hands. Finger manipulation patterns are investigated when a subject rotates a cylindrical object. Typical contact patterns during rotating manipulation are measured by FSRs. Based on the observation, the neural circuit model which generates a similar contact pattern is constructed. The experimental results by the computer simulation suggest that rotating manipulation can be performed using the constructed CPG model.

In this paper, we describe a robotic wheelchair system as guide robot To build a smart interface for a guide robot, it is important for the system to know the user's attention. This system can detect gaze direction of the user using a real-time stereo vision system, and can recognize its position within the surrounding environment using a range sensor and a map. Based on the detection of both the gaze direction and the position within the map, the robotic wheelchair can estimate the user's fixation, from which the position where the user is paying attention is estimated. We adopt an attention representation based on histogram. The frequency of fixation is represented by an "attention histogram". Experimental results indicate the validity of the control assistance to change the route of the autonomous run using the estimation of user's attention.

In this paper, a grip-force control of an elastic object is proposed based on a visual slip margin feedback. When an elastic object is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip" occurs on the contact surface. The slip margin between an elastic object and a rigid plate is estimated based on the analytic solution of Hertzian contact model. A 1 DOF gripper consists of a camera and a force sensor is developed. The slip margin can be estimated from the tangential force measured by a force sensor, the deformation of the elastic object and the radius on the contact area both measured by a camera. In the proposed method, the friction coefficient is not explicitly needed. The grip force is controlled by a direct feedback of the estimated slip margin, whose stability is analytically guaranteed. As a result, the slip margin is maintained to a desired value without occurring the gross slip against a disturbance load force to the object.

A direct teaching method is effective for multi-fingered robot hand designed similar to human's hand structure. To achieve an instinctive teaching, it is necessary to perceive manipulation patterns by observing a human hand. In this paper, a perception system of human manipulation is developed as a part of direct teaching system for multi-fingered robot hand. Conventional perception systems have difficulties in perceiving complex manipulations obtained by dextrous finger actions due to the low accuracy of the CyberGlove. This paper proposes an operation measurement system using an object that is specially designed for teaching. This object consists of a six-degree-of-freedom position/orientation sensor and a tactile sensor sheet. This paper also proposes a method of spotting perception by observing a change of the contact state of the palm surface. The effectiveness of this method is confirmed by experiments.

In this paper, we introduce an annotation-based assistance system for an unmanned helicopter with a wearable augmented reality environment. In this system, an operator controls the helicopter remotely while watching an annotated view from the helicopter through a head mounted display (HMD) with a laptop PC in a backpack. Annotations assist the operation indicating some conditions of the helicopter and a name of buildings nearby. The position and the attitude of the helicopter is measured by GPS and a gyroscope, and sent to the operator's PC via a wireless LAN.

In this paper, we study the relation between the user's degree of understanding and his/her eye movements; in an effort to realize a proactive interface that monitors the user and provides a contextual support. The application is a gaze sensitive dictionary that helps the user when reading a text in a browser's window. Not only is the user's gaze analyzed but also the context and thus the difficulty degree of the text being read. The experiment results suggest using regressions as an indicator to trigger the help process along with a context grounding approach.

In this paper, a novel pointing device is proposed which utilizes the deformation of the fingerprint. When the fingertip is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip", occurs on the contact surface. Since the deformation around the center of the contact surface is small, the center of the fingerprint can be detected during the incipient slip. The group delay spectrum (GDS) tracking is utilized to detect the center of the fingerprint. The velocity of the pointer is controlled by the changes of the contact area on the basis of the tracked center. A prototype pointing device is developed to examine the operationality of the proposed method. The experimental results show that an operator can control the pointer with high accuracy.

In this paper, the role of the human being toe is investigated. The change of the sole pressure distribution using a sole pressure distribution measurement system and that of the toe pressure using a toe pressure measurement system at walk by the existence of a toe are investigated. In order to investigate the change of the whole walking, the affection of a walk posture using a 3D motion capture system by the existence of a toe is also measured. Furthermore, the change of the amount of energy consumption at walk by the existence of a toe is measured by a spirometry system. At last, we investigate the roles of the toe, such as, the efficiency of the big and 2nd toe, and the stabilization of the posture, by the synthesis of these results.

We implemented a humanoid robot, ASKA, in our university reception desk for the computerized university guidance. ASKA can recognize a user's question utterance, and answer the user's question by its text-to-speech voice, hand gesture and head movement. This paper describes the speech related parts of ASKA. ASKA can deal with a wide task domain of 20k large vocabulary using a word trigram model and an elaborated speaker-independent acoustic model. ASKA can also make a response with keyword and key-phrase detection in the N-best speech recognition results. The word recognition rate for the reception task is 90.9%, and the rate for the out-of-domain task is 78.9%. The correct response rate for the reception task is 61.7%. Users can enjoy their question-answering with ASKA.

Proposes a method for hand pose estimation that can be used for 3D free-form input interfaces. The aim of the method is to estimate all joint angles to manipulate objects in the virtual space. In this method, the hand regions are extracted from multiple images obtained by the multi-viewpoint camera system. By integrating multi-viewpoint silhouette images, a hand pose is reconstructed as a "voxel model". Then all joint angles are estimated using three dimensional model fitting between hand model and voxel model. Two experiments were performed: (1) the estimation of joint angles by the silhouette images from the hand-pose simulator, (2) hand pose estimation using real hand images. The experimental results indicate the feasibility of the proposed algorithm for vision-based interfaces, though it requires faster implementation for real-time processing.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are deeply related with his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. We describe our real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Since the 3D coordinates of the features on a face can be directly measured in our system, we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. Consequently we achieved a non-contact, passive, real-time, robust, accurate and compact measurement system for head pose and gaze direction.

Computer systems which analyse human face/head motion have attracted significant attention recently as there are a number of interesting and useful applications. Not least among these is the goal of tracking the head in real time. A useful extension of this problem is to estimate the subject's gaze point in addition to his/her head pose. This paper describes a real-time stereo vision system which determines the head pose and gaze direction of a human subject. Its accuracy makes it useful for a number of applications including human/computer interaction, consumer research and ergonomic assessment.

A map is a description which represents the structure of an environment. In order for a robot to move from a point to another efficiently in an environment, a map of the environment is the essential information. "Omni-View Sequence" is a kind of route representation which we have developed for robot navigation in a corridor environment. It contains a sequence of omnidirectional views as the information of a route. However it cannot be regarded as a map, because the start and the goal points of the route are both fixed. In this paper, we first propose a map named "view-sequenced map" which can represents a whole corridor environment on a floor in a building. Then we describe the method for acquiring a view-sequenced map automatically based on the exploration of a robot in a corridor using both stereo and omnidirectional vision. Finally experimental results of the map acquisition and the autonomous navigation are presented to show the feasibility of the proposed map representation.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are closely related to his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. In this paper, we describe a behavior recognition system based on the real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Our system we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. The recognition of attentions and gestures of a person is demonstrated in the experiments.

Recently, view-based or appearance-based approaches have been attracting the interests of computer vision research. Based on a similar idea, we have proposed a view-based navigation method using a model of the route called the "view sequence." It contains a sequence of frontal views along a route memorized in the teaching run, and the recognition of the environment is realized based on the matching of the current view and memorized view sequence. In this paper, we discuss the required characteristics of the view for the view sequence, and evaluate our former method of generating views. Then we confirm that the stereo disparity satisfies the requirement of the view sequence through an experiment, and the disparity view sequence is applied for outdoor navigation. The experimental results indicate such views other than normal camera images can be utilized for our view-based navigation method.

View-based or appearance-based approaches have been attracting the interests of computer vision research. We have already proposed a visual view-based navigation method using a model of the route called the "view sequence", which contains a sequence of front views along a route memorized in the teaching run. In this paper, we apply an omnidirectional vision sensor to our view-based navigation and propose an extended model of a route called "omni-view sequence". The omnidirectional vision sensor is a desirable sensor for real-time view-based recognition of a mobile robot because all the information around the robot can be acquired simultaneously. The use of omnidirectional images contribute to a better navigation method in three ways: (1) it enables the robot to come back along the route using the same view sequence, (2) it improves the accuracy of the navigation and (3) it improves the robustness of the matching. These improved points are shown through experiments, and indoor navigation, including getting on and off an elevator, is demonstrated.

Simulation is an important issue in robotics research because it is essential for evaluating and predicting the behavior of a robot. However conventional simulators don't have the function of simulating the visual perception process. This indicates that it is impossible to simulate the vision-based behavior of a robot. In this paper, we propose a new simulation system "View Simulation System (VSS)", which is designed to simulate the behavior of a mobile robot equipped with vision. VSS has vision processing hardware, and can run any programs including vision processing for our real mobile robots. To show the feasibility of the simulator, we applied it to the evaluation of our visual navigation method.

A legged robot "JROB-1" is developed for a robotics research platform as a result of inter-university research program on intelligent robotics supported by the Ministry of Education Grant-in-Aid for Scientific Research on Priority Areas in Japan. The JROB-1 features: 1) self-contained, 2) RT-Linux running on PG/AT processes vision and sensor processing, motion planning and control, 3) connected to a network via radio Ethernet as to utilize networked resources, 4) Fujitsu color tracking vision board and Hitachi general purpose vision processing board, 5) all parts are commercially available, and 6) it is extensible with respect to sensor, sensor processing hardware and software. JROB-1 is expected to be a common testbed for experiment and intelligent robotics research by integrating perception and motion.

We propose an approach "online visual feature determination through discourse", which realizes the finding and the following task in a real environment. To segment the human's image from the complex background, it is possible to prepare many finds of basic visual features, and to combine them. The proposed approach is a solution for the problem of how to select and combine these visual features for each situation. Namely, a robot and a human make a conversation to search for the suitable visual features for the current situation. This idea enables the robot to find and follow a human successfully in a changeable background environment.

View changing because of vibration while walking is one of the most fundamental problem for a vision based legged robot. To overcome this difficulty, three key issues are denoted: a) integration of color segmentation, optical flow and stereo, which is able to apply to vibrating view using correlation hardware, b) software servo loop implemented as a kernel module for the purpose of soft actuation, and c) smooth walking pattern generation using solid model and dynamics simulator. Furthermore, a quadruped legged robot "JROB-1" is developed as a platform for the research on perception-action coupling in intelligent behavior of robots.

This paper discusses a stereo vision processing system which processes color and monochrome stereo video signals on a single vision processing board. Here, we propose a "field mixing" technique for multiplexing multiple video signals. A compact color stereo vision system based on this technique is developed for a mobile robot. This system can process multiple video signals simultaneously, and realizes flexible color stereo vision processing. In order to show the feasibility of this vision system, we installed it on our mobile robot, and implemented a correlation-based EZDF method for stereo tracking of an object. The experimental result of the tracking is shown.

Previous work in vision-based mobile robotics have lacked models of the route which can be utilized for (1) localization, (2) steering angle determination, and (3) obstacle detection, simultaneously. In this paper, the authors propose a new visual representation of the route, the "view-sequenced route representation (VSRR)." The VSRR is a non-metrical model of the route, which contains a sequence of front view images along a route memorized in the recording run. In the autonomous run, the three types of recognition described above are achieved in real-time by matching between the current view image and the memorized view sequence using a correlation technique. the authors also developed an easy procedure for acquiring VSRRs, and a quick control procedure using VSRRs. VSRRs are especially useful for representing routes in corridors. Results of autonomous navigation using a two-wheeled robot in a real corridor are also presented.

Presents a practical mobile robot system, called Hyper Scooter, which a human can ride on, and share access to the environment through visual information. The advantage of these capabilities is significant. A user can give his/her instructions to the robot without skillful programming operations. This is supported by the authors' high speed tracking vision system. To show the advantage, a complex task is presented as an example. After being instructed the way to accomplish the task by a human, Hyper Scooter does it without explicit environmental models.