This paper proposes a pacing system that makes dialog robots mimic the pacing technique in order to promote trust relationships between humans and robots. Pacing is a communicative technique that synchronizes breathing or how to speak with a person to build relationships among humans. The experiment was performed by comparing the impression of human subjects' conversation with a robot when the pacing system was applied or not applied. The evaluation used both subjective and objective perspectives. From the subjective perspective, the results show significant differences for the questionnaire items concerning "feeling friendliness," "feeling fun," "feeling an emotional connection," "feeling synchrony," "motivation to use" and "listening to the user" between the with and without pacing conditions. Also, from the objective perspective, there is a significant difference in sympathetic activity in heart rate variability.

Robots deployed in large human-populated indoor environments such as shopping malls, airports etc., inadvertently communicate via wireless networks for enhanced perception and decision making capabilities. Owing to highly dynamic signal attenuation characteristics in such environments, connectivity issues may arise during robotic navigation, leading to disruption in information flow causing potential danger. Exact modeling of signal propagation for estimating spatial signal variation is usually challenging. Moreover, the presence of dynamic humans also add a layer of temporal signal variation complexities. Thus, this paper introduces a generative approach for embedding radio signal strength constraints within networked service/social robot navigation in large human-populated environments. Initially, we propose a Gaussian Process based online spatio-temporal signal strength prediction model that, as opposed to the current state of the art, also aims to take into account the temporal fading arising due to the presence of human crowds. We then devise a transient-goal driven navigation strategy to realize a sub-optimal path towards a goal, that is aimed at resolving both communication-aware and human-aware planning constraints. Evaluations of the proposed signal prediction model demonstrate the advantages of our approach with respect to the current state of the art. The efficacy of the navigation strategy in also demonstrated simulations and using hardware experiments conducted on a robotic wheelchair operating in a large shopping mall.

In this paper, we lay the algorithmic foundations of a unifying strategy for pedestrian-aware navigation that is aimed at service/social robots deployed in large human-crowded environments. In order to accommodate both modeled and learned social navigation behaviors, we formalize an approach within which the robot traverses to a specific goal (or sub-goal) via a trajectory of optimal transient-goals or optimal short-term way-points. We then evaluate an implementation of the navigation strategy, by utilizing an augmented Risk-based Rapidly Exploring Random Trees (RRT) planner, and demonstrate its efficacy for real-world deployment using discriminative simulations and by providing avenues for future work.

In human science literature, a hug interaction from a close person buffers the stress of the person being hugged. In this study, we investigated whether audio-visual stimuli during a hug with a robot change such stress buffer effects. For this purpose, we developed the MetaHug system, which consists of a huggable robot and a head-mounted-display that enables a virtual agent to reciprocate hugs. In our experiment, participants did stressful serial subtraction tasks after they experienced hug interactions with either two kinds of virtual agents: female or male appearances. Experiment results with 18 participants showed that participants showed significantly low stress when they did a hug with opposite-gender appearance agent compared to same-gender appearance agent.

Using multiple depth sensors enables us to accurately track pedestrians in real environments. Accurate pedestrian positions are essential for building an effective human-centered cyber world provided by location-based services. In particular, ceiling-mounted depth sensors can robustly track people in such environments. However, one important problem for this approach is the accurate calibration of the absolute sensor positions. This problem remains unsolved due to limited range and sensor distortions from a distance. Manual calibration is complicated and time-consuming, and the existing calibration method still has several limitations since it used a pedestrian as a movable landmark. Instead of a human landmark, we propose a method that uses a mobile robot as a movable and localized landmark to calibrate each sensor. We compared the calibration performance of the proposed and existing methods and showed that the former achieved more accurate calibration for both the absolute sensor and tracked pedestrian positions. Our proposed method with a mobile robot not only increased the accuracy of the calibration processes but also decreased human efforts.

This paper investigates whether a network robot salesclerk system increases sales in real shopping contexts. Our robot system, which consists of an autonomous virtual agent and a semi-autonomous physical agent, enables customers to interact with the virtual agents on their smartphones and reserve special character merchandise. Moreover, their virtual agent is transferred to the physical agent at the shop to physically distribute the reserved merchandise to customers. Through such cyber-physical interaction, we provided rich shopping experiences to customers to increase sales. We collaborated with an animation company, Production I.G Inc., and employed an animation character named Tachikoma from the Ghost in the Shell:Stand Alone Complex (a.k.a S.A.C. seriese) universe to design the appearance and the characteristics of both agents. We conducted field trials to investigate whether the developed system contributed to sales related to the animation merchandise of Ghost in the Shell, and the results showed our system's effectiveness.

This paper investigates the effects of a listener robot that joins a storytelling situation for children as a side-participant. For this purpose, we develop a storytelling-robot system that consists of both reader and listener robots. Our semi-autonomous system involves a human operator who makes correct responses and provides easily understandable answers to the children's questions during storytelling. We develop a gaze model for the natural and autonomous gaze behaviors of a reader robot by considering multiple listeners and a storytelling object (a display that shows images). We conducted an experiment with 16 children to investigate whether they preferred storytelling with/without the listener robot and the changes of speech activities during the storytelling. Children preferred storytelling with the listener robot to storytelling without it. Their speech activities decreased when the listener robot was involved in the storytelling.

This paper presents the effects of being hugged by a robot to encourage prosocial behaviors. In human-human interaction, touches including hugs are essential for communication with others. Touches also show interesting effects, including the “Midas touch,” which encourages prosocial behaviors from the people who have been touched. Previous research demonstrated that people who touched a robot experienced positive impressions of it without clarifying whether being hugged by a robot causes the Midas touch effect, i.e., positively influences engagement in prosocial behaviors. We developed a huge, teddy-bear-like robot that can give reciprocal hugs to people and experimentally investigated its effects on their behaviors. In the experiment, a robot first asked participants to give a hug and then asked them to make charitable donations in two conditions: with or without a reciprocated hug. Our experiment results with 38 participants showed that those who were hugged by a robot donated more money than those who only hugged the robot, i.e., without a reciprocated hug.

In this paper, we experimentally examine the relationship between visual cognition difficulty and target-tracking eye movements, which recorded during moving target cognition. Generally, such eye movements are observed when humans perceive a moving object and they vary widely due to many factors, such as target shape, backgrounds, illumination conditions, and so on. Several systems have been proposed for estimating human cognition based on gaze movements. However, since most of them employ simple thresholding techniques to classify the states of cognition, their classification performance remain insufficient. This research clarifies the relationship between multiple visual conditions and target-tracking eye movements to enhance the classification performance. We found that we can address a variety of target and background factors from the perspective of visual cognition difficulty for the targets. Observed gaze movement properties and the difficulty of target visual cognition have a linear relationship, suggesting the possibility of more precise estimation of human's target cognition based on them.

Personal mobility devices such as wheelchairs, bicycles, and compact cars are used in daily life and to runs on sidewalks. However, there are several factors that may lead to discomfort rides, such as steps, slopes, and crowded sidewalks for a passenger. This paper proposes a system that detects the factors using smartphone attached to the personal mobility device and generates a discomfort-ride map including the environment and human factors and their rating on sidewalks. In the experiment, steps (static danger factor) and the dynamic moving obstacles like human, bicycle, and car (dynamic danger factors) were estimated with attached smartphone's acceleration sensor and camera. After the process of collecting these data, verification for generating the hazard map system based on these data. Moreover, the versatility of using the proposed hazard map system for different personal mobility devices was tested with the electric wheelchair and the bicycle.

This paper presents an approach for motion sickness reduction while riding an autonomous vehicle. It proposes the Diminished Reality (DR) method for an acceleration stimulus to reduce motion sickness for the autonomous vehicle. One of the main causes of motion sickness is a repeated acceleration. In order to diminish the acceleration stimulus in the autonomous vehicle, vection illusion is used to induce the user to make a preliminary movement against the real acceleration. The Balance Wii Board is used to measure participant's movement of the center of gravity to verify the effectiveness of the method with vection. The experimental result of 9 participants shows that the proposed method of using vection could reduce acceleration stimulus compared with the conventional method.

This paper presents a two-step framework for creating the three-dimensional (3-D) sound map of an environment with a mobile robot. The first step is the creation of a map that describes the geometry of the environment. The second step is the addition of the acoustic information to the geometric map. The result is a sound map that shows the probability of emitting sound for all the structures in the environment. To build the sound map, a mobile robot equipped with a microphone array drives through the mapped environment. During this drive, the acoustic information gathered by the microphone array is accumulated in a probabilistic manner. First, the likelihood of sound source presence in a set of directions is evaluated from the acoustic power received from these directions. Then, using an estimate of the robot's pose, an acoustic ray casting procedure transfers this likelihood to the structures in the geometric map. Finally, the probability that these structures emit sound is updated accordingly to the likelihood. Experimental results show that the sound maps are: accurate as it was possible to localize sound sources in 3-D and practical as different types of environments were mapped.

Haptic interaction is a key capability for social robots that closely interact with people in daily environments. Such human communication cues as gaze behaviors make haptic interaction look natural. Since the purpose of this study is to increase human-robot touch interaction, we conducted an experiment with 20 participants who interacted with a robot with different combinations of gaze behaviors and touch styles. The experimental results showed that both gaze behaviors and touch styles influence the changes in the perceived feelings of touch interaction with a robot.

The recognition of an indicated object by an interacting person is an essential function for a robot that acts in daily environments. To improve recognition accuracy, clarifying the goal of the indicating behaviors is needed. For this purpose, we experimentally compared two kinds of interaction strategies: a robot that explicitly provides instructions to people about how to refer to objects or a robot that implicitly aligns with the people's indicating behaviors. Even though our results showed that participants evaluated the implicit approach to be more natural than the explicit approach, the recognition performances of the two approaches were not significantly different.

In human-human interaction, peer pressure is a major social influence on people's thoughts, feelings, and behaviors. The larger the group of people, the more social influence it exerts. In this paper, we investigate whether multiple robots and their synchronized behaviors exert peer pressure on people, as in human groups. We developed a multiple robot controller system that enables robots to perform precise synchronization. In the experiment, we prepared a setting that resembled previous experiments that investigated peer pressure between people and robots. The participants answered questions after hearing the robots' answers, only some of which were incorrect. Our experiment results showed that the influence of the synchronized multiple robots increased the error rates of the participants, but we found no significant effects toward conformity.

In order to solve the problems of current hearing aid devices, we make use of environment sensor network, and propose a hearing support system, where individual target and anti-target sound sources in the environment can be selected, and spatial information of the target sound sources is reconstructed. The performance of the selective sound separation module was evaluated for different noise conditions. Results showed that signal-to-noise ratios of around 15dB could be achieved by the proposed system for a 65dB babble noise plus directional music noise condition. In the same noise condition, subjective intelligibility tests were conducted, and an improvement of 65 to 90% word intelligibility rates could be achieved by using the proposed hearing support system.

The use of 3D range sensors for human position tracking has grown in recent years, especially for augmenting robotic sensing for human-robot interaction. However, extrinsic calibration of the relative positions of 3D range sensors is difficult, due to their limited range, narrow field of view, and distortion at large distances. 2D laser range finders have also been used for pedestrian tracking, providing greater accuracy and coverage at the cost of being more expensive and susceptible to occlusion. In this work, we present two novel techniques for calibrating the positions of 3D range sensors based on shared observations of pedestrians. The first technique uses 3D range sensors alone, and the second technique uses 2D and 3D range sensors together, using the high precision and long range of the 2D sensors to complement the short-range but richer sensing of 3D range sensors. We evaluate the accuracy of both automatic calibration techniques, and we furthermore show that the combination of 2D and 3D sensors gives more robust and accurate calibration than when using 3D sensors alone.

This work proposes a Human-Comfortable Path Planner (HCoPP) system for autonomous passenger vehicles. The aim is to create a path planner that improves the feeling of comfort of the passenger, this topic is different from collision free planning and it has not received much attention. For this purpose, in addition to the shortest distance constraint conventionally used in path planning, constraints related to relevant environmental features are introduced. For straight segments, the constraint is based on the lane-circulation pattern preferred by humans. In curved segments and intersections, the constraint takes into account the visibility. A multi-layered cost map is proposed to integrate these additional constraints. To compute the human-comfortable path, a graph search algorithm was implemented. The evaluation of the proposed approach was conducted by having 30 participants riding an autonomous robotic wheelchair. The paths computed by the proposed path planner were compared towards a state of the art shortest-distance path planner implemented in the navigation stack of ROS. Experimental results show that the paths computed by the proposed approach are perceived as more comfortable.

This paper presents a method for representing acoustic information with point clouds by tying it to geometrical features. The motivation is to create a representation of this information that is well suited for mobile robotic applications. In particular, the proposed approach is designed to take advantage of the use of multiple coordinate frames. As an illustrative example, we present a way to create an audio augmented point cloud by adding estimated audio power to the point cloud created by a RGB-D camera. A few applications of this method are presented.

We developed a system for detecting the speech activity intervals of multiple speakers by combining multiple microphone arrays and human tracking technologies. We also proposed a method for estimating the face orientation of the detected speakers. The developed system was evaluated in two steps: individual utterances in different positions and orientations; and simultaneous dialogues by multiple speakers. Evaluation results revealed that the proposed system could detect speech activity intervals with more than 90% of accuracy, and face orientations with standard deviations within 30 degrees, in situations excluding the cases where all arrays are in the opposite direction to the speaker's face orientation.

This paper presents a work on mapping the use of space by humans in long periods of time. Daily geometric maps with the same coordinate frame were generated with SLAM, and in a similar manner, daily affordance density maps (places people use) were generated with the output of a human tracker running on the robot. The contribution of the paper is two-fold: an approach to detect geometric changes to cluster them in similar geometric configurations and the building of geometric and affordance composite maps on each cluster. This approach avoids the loss of long term retrieved information. Geometric similarity was computed using a normal distance approach on the maps. The analysis was performed on data collected by a mobile robot for a period of 4 months accumulating data equivalent to 70 days. Experimental results show that the system is capable of detecting geometric changes in the environment and clustering similar geometric configurations.

This paper presents a study on intention communication in a navigational context using a robotic wheelchair. The robotic wheelchair uses light projection to communicate its motion intentions. The novelty of the work is threefold: the communication of robot intentions to the passenger, the consideration of passenger and robot as a group (“in-group”) [1] who share motion intentions and the communication of the in-group intentions to other pedestrians (the “out-group”). A comparison in an autonomous navigation task where the robotic wheelchair autonomously navigates the environment with and without intention communication was performed showing that passengers and walking people found intention communication intuitive and helpful for passing by actions. Evaluation results significantly show human participant preference for having navigational intention communication for the wheelchair passenger and the person passing by it. Quantitative results show the motion of the person passing by the wheelchair with intention communication was significantly smoother compared to without intention communication.

This paper reports an interactive approach to improve the recognition performance by robots of objects indicated by humans during human-robot interaction. We developed an approach based on two findings in conversations where a human refers to an object, which is confirmed by a robot. First, humans tend to use the same words or gestures as the robot in a phenomenon called alignment. Second, humans tend to decrease the amount of information in their references when the robot uses excess information in its confirmations: in other words, alignment inhibition. These findings lead to the following design; a robot should use enough information without being excessive to identify objects to improve recognition accuracy because humans will eventually use similar information to refer to those objects by alignment. If humans more frequently use the same information to identify objects, the robot can more easily recognize those being indicated by humans. To verify our design, we developed a robotic system to recognize the objects to which humans referred and conducted a control experiment that had 2 × 3 conditions; one factor was the robot's confirmation way and another was the arrangement of the objects. The first factor had two levels to identify objects: enough information and excess information. The second factor had three levels: congestion, two groups, and a sparse set. We measured the recognition accuracy of the objects humans referred to and the amount of information in their references. The success rate of the recognition and information amount was higher in the adequate information condition than in the excess condition in a particular situation. The results suggested the possibility that our proposed interactive approach improved recognition performance.

This paper investigates people's social acceptance of a childcare support robot system and compares their attitudes to two childcare technologies: anesthesia during labor and baby food (processed food and formula milk), which includes powdered milk and instant food for babies and toddlers. To investigate their social acceptance, we developed scales from three points of view: safety and trustworthy, diligence, and decreasing workload. For this paper, our participants were comprised of 412 people located through a web-based survey and 14 people who experienced the prototype of our childcare support robot system. They answered questionnaires about our three developed scales and an intention to use scale to investigate their social acceptance toward childcare support technologies. The web-based survey results indicate that our system's concept was evaluated lower than current childcare support technologies, but people who experienced our system prototype evaluated it higher than those who filled out web-based surveys.

This study developed a fall detection system for elderly people in a nursing home and investigated their acceptance of it. The system obtained their positions and heights from range sensors and used that information to correctly detect 89.5% of the falls based on data where the elderly crouched in a mockup room of a nursing home. We investigated the social acceptance with elderly people by comparing three conditions: (1) only detecting out-of-bed, (2) detecting falls in a room and always showing the human position, and (3) detecting falls in a room and only showing the human position when a fall happened. The results showed that intention to use were significantly higher in the second and third conditions than the first condition.

This study investigates whether a social robot which interacts with children via quiz-style conversations increases their understanding of science classes. We installed a social robot in an elementary school science classroom where children could freely interact with it during their breaks. The robot asks children questions related to their latest science classes to support their understanding of the classes. During interaction, the robot says children's name and distribute its gaze among the group of children by using a face recognition system and a human tracking system. Still, speech recognition is difficult in the noisy elementary school environment; therefore the operator takes over this function during interactions. In this study our result did not show significant effects of the robot for helping children's understanding, but we found several interesting interaction scenes which shows that robot had a certain effect on specific children.

This study reports a preliminary investigation of a trial to support child-care at an intelligent playroom. We installed a human tracking system and a toy-like robot into the playroom to record children/parents activities. Moreover, we interviewed nurses at nursery schools to study their attitude towards use of robotics technologies for child-care to investigate the social acceptance of robots for supporting child-care.

The importance of assistive robots for elderly people is increasing to match the needs of rapidly aging world societies. This paper develops a wheelchair robot to support the movement of the elderly to decrease caregiver workloads from the perspective of physical load and time consumption. We observed the behaviors of caregivers at a private residential care home when they are wheeling elderly people in wheelchairs. The behavior includes appropriate utterances for reducing the anxieties of senior citizens based on properties of place, e.g., "Here the path is a little narrow" while passing a narrow space or "Now let's go into elevator." They also adjusted to the wheeling speed preference of each individual because their preferred speeds were different. We implemented these two functions for elderly care using the Ubiquitous Network Robot Platform that allows the management of users and environment properties. Experimental results with elderly participants at a pseudo resident home show that the seniors prefer a wheelchair robot on which the above functions have been implemented.

This paper presents a framework for creating a 3D map of an environment that contains the probability of a geometric feature to emit a sound. The goal is to provide an automated tool for condition monitoring of plants. The map is created by a mobile platform equipped with a microphone array and laser range sensors. The microphone array is used to estimate the sound power received from different directions whereas the laser range sensors are used for estimating the platform pose in the environment. During navigation, a ray casting method projects the audio measurements made onboard the mobile platform to the map of the environment. Experimental results show that the created map is an efficient tool for sound source localization.

This work introduces a 3D visibility model for comfortable autonomous vehicles. The model computes a visibility index based on the pose of the wheelchair within the environment. We correlate this index with human navigational comfort (discomfort) and we discuss the importance of modeling visibility to improve human riding comfort. The proposed approach models the 3D visual field of view combined with a two-layered environmental representation. The field of view is modeled with information from the pose of the robot, a 3D laser sensor and a two-layered environmental representation composed of a 3D geometric map with traversale area information. Human navigational discomfort was extracted from participants riding the autonomous wheelchair. Results show that there is fair correlation between poor visibility locations (e.g., blind corners) and human discomfort. The approach can model places with identical traversable characteristics but different visibility and it differentiates visibility characteristics according to traveling direction.

This paper presents a framework for making a mobile robot aware of an entity in the blind region of its laser range finders when that entity emits sound. First in a mapping stage, a 3D description of the environment that contains information about acoustic reflection is created. Then during operation, the robot combines estimated directions of arrival of sound with this 3D description to detect entities that are not visible by line of sight sensors but could be heard because of sound reflections. Using this approach, it is possible to restrict the hypothesis about the position of a sound emitting entity in the blind region to a small set of candidate depth values.

This work proposes a model for human habituation while riding a robotic wheelchair. We present and describe the concept of human navigational habituation which we define as the human habituation to repetitively riding a robotic wheelchair. The approach models habituation in terms of preferred linear velocity based on the experience of riding a wheelchair. We argue that preferred velocity changes as the human gets used to riding on the wheelchair. Inexperienced users initially prefer to ride at a slow moderate pace, however the longer they ride they prefer to speed up to a certain comfort level and find initial slower velocities to be tediously “too slow” for their experience level. The proposed habituation model provides passenger preferred velocity based on experience. Human biological measurements, galvanic skin conductance, and participant feedback demonstrate the preference for habituation velocity control over fixed velocity control. To our knowledge habituation modeling is new in the field of autonomous navigation and robotics.

Developing interactive behaviors for social robots presents a number of challenges. It is difficult to interpret the meaning of the details of people's behavior, particularly non-verbal behavior like body positioning, but yet a social robot needs to be contingent to such subtle behaviors. It needs to generate utterances and non-verbal behavior with good timing and coordination. The rules for such behavior are often based on implicit knowledge and thus difficult for a designer to describe or program explicitly. We propose to teach such behaviors to a robot with a learning-by-demonstration approach, using recorded human-human interaction data to identify both the behaviors the robot should perform and the social cues it should respond to. In this study, we present a fully unsupervised approach that uses abstraction and clustering to identify behavior elements and joint interaction states, which are used in a variable-length Markov model predictor to generate socially-appropriate behavior commands for a robot. The proposed technique provides encouraging results despite high amounts of sensor noise, especially in speech recognition. We demonstrate our system with a robot in a shopping scenario.

This paper proposes a novel framework to estimate the amount of physical activity at a place where people stayed, by utilizing facility information and user's acceleration data. The total amount of physical activities, energy expenditure of a physical activity is a good scale. Our framework provides a physical activity scale based on typical energy expenditure of the activity given by existing researches already. To estimates the energy expenditure, we use typical value of metabolic equivalents to task (MET) which is used as practical scale. Unlike the other studies for monitoring physical activity, we estimate the type of user's activity using facility information which are obtained from road map and land-use/land-cover map. To confirm the feasibility of our approach, we have conducted a long-term experiment on monitoring the activity of elderly people living in less-populated area. As a result, our framework provides good summarization of daily activities of participants.

Small robots are being designed to recognize behaviors through playful interaction. Prior work used data from impoverished sensing devices such as inertial sensors to analyze gestures and attitude in playful interaction through time series analysis. However, the prior work did not focus on individual differences required for person identification. This research hypothesizes that person identification can be achieved by determining individual differences in playful interaction by using inertial sensor data. We propose a method that iteratively narrows down the candidates during interaction to achieve accurate person identification. This method calculates the features using a time series of the inertial sensor data. These features identify a candidate who is playfully interacting with the robot using a decision tree classifier that includes combinations of the current candidates. The system stores the results as a dataset for voting, and the voting results are used to reduce the candidates until the number of candidates is winnowed to one. Evaluation results show that our proposed method identifies persons through playful interactions with 99.1% accuracy.

This paper presents a computational model for side-by-side walking within human-robot interaction (HRI). In this work we address the importance of future motion utility (motion anticipation) of two walking partners. Previous studies only considered a robot moving alongside a person without collisions and with simple velocity-based predictions. In contrast, our proposed model includes two major considerations. First, it considers the current goal, modeling side-by-side walking as a process of moving toward a goal while maintaining a relative position with the partner. Second, it takes the partner's utility into consideration; it models side-by-side walking as a phenomenon where two agents maximize mutual utilities rather than only considering a single agent utility. The model is constructed based in a set of trajectories from pairs of people recorded in side-by-side walking; then, parameters of the model were calibrated for a mobile robot and tested in an autonomous robot walking side-by-side with participants. Finally, two evaluations were performed. The first evaluation shows that the proposed model considering mutual utilities performs better than a single utility method and a method that keeps distance from the walking partner. In the second evaluation the proposed method was used for a robot deployed in a shopping mall environment where it demonstrated to be effective.

This paper presents a multi-modal sensor approach for mapping sound sources using an omni-directional microphone array on an autonomous mobile robot. A fusion of audio data (from the microphone array), odometry information and the laser range scan data (from the robot) was used to precisely localize and map the audio sources in an environment. An audio map is created while the robot is autonomously navigating through the environment by continuously generating audio scans with a steered response power (SRP) algorithm. Using the poses of the robot, rays are cast in the map in all directions given by the SRP. Then each occupied cell in the geometric map hit by a ray is assigned a likelihood of containing a sound source. This likelihood is derived from the SRP at that particular instant. Since the localization of the robot is probabilistic, the uncertainty in the pose of the robot in the geometric map is propagated to the occupied cells hit during the ray casting. This process is repeated while the robot is in motion and the map is updated after every audio scan. The generated sound maps were reused and the changes in the audio environment were updated by the robot as it identifies these changes.

This paper presents a method for mapping the radiated sound intensity of an environment using an autonomous mobile platform. The sound intensities radiated by the objects are estimated by combining the sound intensity at the platform's position (estimated with a steered response power algorithm) and the distances to the objects (estimated using laser range finders). By combining the estimated sound intensity at the platform's position with the platform's pose obtained from a particle filter based localization algorithm, the sound intensity radiated from the objects is registered in the cells of a grid map covering the environment. This procedure creates a map of the radiated sound intensity that contains information about the sound directivity. To illustrate the effectiveness of the proposed method, a map of radiated sound intensity is created for a test environment. Then the position and the directivity of the sound sources in the test environment are estimated from this map.

We proposed a method for estimating sound source locations in a 3D space by integrating sound directions estimated by multiple microphone arrays and taking advantage of reflection information. Two types of sources with different directivity properties (human speech and loudspeaker speech) were evaluated for different positions and orientations. Experimental results showed the effectiveness of using reflection information, depending on the position and orientation of the sound sources relative to the array, walls, and the source type. The use of reflection information increased the source position detection rates by 10% on average and up to 60% for the best case.

This paper presents a method for detecting moving entities that are in the robot's path but not in the field of view of sensors like laser scanners, cameras or ultrasonic sensors. The proposed system makes use of passive acoustic localization methods which receive information from occluded regions (at intersections or corners) because of the multipath nature of sound propagation. Contrary to the conventional sensors, this method does not require line of sight. In particular, specular reflections in the environment make it possible to detect moving entities that emit sound such as a walking person or a rolling cart. This idea was exploited for safe navigation of a mobile platform at intersections. The passive acoustic localization output is combined with a 3D geometric map of the environment that is precise enough to estimate sound propagation and reflection using ray casting methods. This gives the robot the ability to detect a moving entity out of the field of view of the sensors that require line of sight. Then the robot is able to recalculate its path and waits until the detected entity is out of its path so that it is safe to move to its destination. To illustrate the performance of the proposed method, a comparison of the robot's navigation with and without the audio sensing is provided for several intersection scenarios.

The noninvasive brain-machine interface (BMI) is anticipated to be an effective tool of communication not only in laboratory settings but also in our daily livings. The direct communication channel created by BMI can assist aging societies and the handicapped and improve human welfare. In this paper we propose and experiment a BMI framework that combines BMI with a robotic house and autonomous robotic wheelchair. Autonomous navigation is achieved by placing waypoints within the house and, from the user side, the user performs BMI to give commands to the house and wheelchair. The waypoint framework can offer essential services to the user with an effectively improved information-transfer rate and is an excellent examples of the fusion of data measured by sensors in the house, which can offer insight into further studies.

Reliable autonomous navigation is an active research topic that has drawn the attention for decades, however, human factors such as navigational comfort has not received the same level of attention. This work proposes the concept of “comfortable map” and presents a navigation approach for autonomous passenger vehicles which in top of being safe and reliable is comfortable. In our approach we first extract information from users preference related to comfort while sitting on a robotic wheelchair under different conditions in an indoor corridor environment. Human-comfort factors are integrated to a geometric map generated by SLAM framework. Then a global planner computes a safe and comfortable path which is followed by the robotic wheelchair. Finally, an evaluation with 29 participants using a fully autonomous robotic wheelchair, showed that more than 90% of them found the proposed approach more comfortable than a shortest-path state of the art approach.

In this paper, we discuss how networked robot architectures can facilitate the development, deployment, management and adaptation of distributed robotic applications. Our aim is to modularize applications by factoring out environment, task-, domain-, and robot-specific knowledge components and representing them explicitly in a formal knowledge base that is shared between the robots and service applications. Robot control decisions can then be formulated in terms of inference tasks that are evaluated based on this knowledge during task execution. The explicit and modular knowledge representation allows human operators with different areas of expertise to adapt the respective parts of the knowledge independently. We implemented this concept by integrating knowledge representation methods of the RoboEarth project with the distributed task execution capabilities of the Ubiquitous Network Robot Platform.

The development of humanlike service robots which interact socially raises a new question: How can we make good interaction content for such robots? Domain experts specializing in the target service have the knowledge for making such content. Yet, while they can easily engage in good face-to-face interactions, we found it difficult for them to prepare conversational content for a robot in written form. Instead, we propose involving experts as teleoperators in an iterative development process in which the expert develops content, teleoperates a robot using that content, and then revises the content based on that interaction. We propose a software system and design guidelines to enable such an iterative design process. To validate these solutions, we conducted a comparison experiment in the field, with elderly volunteer guides teleoperating a robot at a tourist information center in Nara, Japan. The results showed that our system and guidelines enabled domain experts with no robotics background to create better interaction content and conduct better interactions than domain experts without our system.

When robots serve in urban areas such as shopping malls, they will often be required to approach people in order to initiate service. This paper presents a technique for human-robot interaction that enables a robot to approach people who are passing through an environment. For successful approach, our proposed planner first searches for a target person at public distance zones anticipating his/her future position and behavior. It chooses a person who does not seem busy and can be reached from a frontal direction. Once the robot successfully approaches the person within the social distance zone, it identifies the person's reaction and provides a timely response by coordinating its body orientation. The system was tested in a shopping mall and compared with a simple approaching method. The result demonstrates a significant improvement in approaching performance; the simple method was only 35.1% successful, whereas the proposed technique showed a success rate of 55.9%.

The teleoperation of mobile social robots requires operators to understand facial gestures and other nonverbal communication from a person interacting with the robot. It is also critical for the operator to comprehend the surrounding environment in order to facilitate both navigation and human-robot interaction. Allowing the operator to control the robot's gaze direction can help the operator observe a person's nonverbal communication; however, manually actuating a gaze increases the operator's workload and conflicts with the use of the robot's camera for navigation. To address these problems, the authors developed a teleoperation system that combines automatic control of the robot's gaze and a 3-D graphical representation of the surrounding environment, such as location of items and configuration of a shop. A study where a robot plays the role of a shopkeeper was conducted to validate the effectiveness of the proposed gaze-control technique and control interface. It was demonstrated that the combination of automatic gaze control and representations of spatial relationships improved the quality of the robot's interaction with the customer.

This study provides an in-depth analysis and practical solution to the problem of designing and implementing a human-robot team for simple conversational interactions. Models for operation timing, customer satisfaction and customer-robot interaction are presented, based on which a simulation tool is developed to estimate fan-out and robot team performance. Techniques for managing interaction flow and operator task assignment are introduced. In simulation, the effectiveness of different techniques and factors related to team performance are studied. A case study on deploying multiple robots in a shopping mall is then presented to demonstrate the usefulness of our study in helping the design and implementation of social robots in real-world settings.

Parkinson's disease (PD) is a severe disease with many symptoms, including speech disorders. Although many methods exist to treat some of PD's symptoms, therapies for speech impairment are not effective and satisfactory, resulting in an open area of research. The current project aims at taking advantage of the Lombard reflex to improve the speech loudness of PD patients. As a first step, the experience of the Lombard reflex by Japanese PD people was confirmed, and the perception of PD patients' speech was evaluated by several subjects. In a following step, methods based on masking sound will be used for intensive training and for self-training of PD patients. However, after intensive training, PD patients may be able to talk louder even without masking noise. In addition, the design and the development of a device based on masking sound that can be used by PD patients while using phone is under consideration.

We propose a multi-camera-based gaze tracking system that provides a wide observation area. In our system, multiple camera observations are used to expand the detection area by employing mosaic observations. Each facial feature and eye region image can be observed by different cameras, and in contrast to stereo-based systems, no shared observations are required. This feature relaxes the geometrical constraints in terms of head orientation and camera viewpoints and realizes wide availability of gaze tracking with a small number of cameras. In experiments, we confirmed that our implemented system can track head rotation of 120° with two cameras. The gaze estimation accuracy is 5.4° horizontally and 9.7° vertically.

Head motion occurs naturally and in synchrony with speech during human dialogue communication, and may carry paralinguistic information, such as intentions, attitudes and emotions. Therefore, natural-looking head motion by a robot is important for smooth human-robot interaction. Based on rules inferred from analyses of the relationship between head motion and dialogue acts, this paper proposes a model for generating head tilting and nodding, and evaluates the model using three types of humanoid robot (a very human-like android, "Geminoid F", a typical humanoid robot with less facial degrees of freedom, "Robovie R2", and a robot with a 3-axis rotatable neck and movable lips, "Telenoid R2"). Analysis of subjective scores shows that the proposed model including head tilting and nodding can generate head motion with increased naturalness compared to nodding only or directly mapping people's original motions without gaze information. We also find that an upwards motion of a robot's face can be used by robots which do not have a mouth in order to provide the appearance that utterance is taking place. Finally, we conduct an experiment in which participants act as visitors to an information desk attended by robots. As a consequence, we verify that our generation model performs equally to directly mapping people's original motions with gaze information in terms of perceived naturalness.

We aim to develop a shopping companion robot that can share experience with users. In this study, we focused on the shared memory acquired when a robot walks together with a user. We developed a computational model of memory recall of visited locations in a shopping mall. The model was developed with data collection from 30 participants. We found that shop size, color intensity of facade, relative visibility, and time elapsed are the influencing features for recall. The model was used in a scenario of a shopping companion robot. The robot, Robovie, autonomously follows a user while inferring the user's memory recall of shops in the visited route. When the user asks the location of other shops, Robovie replied with destination description, referring to the known locations inferred with the model of the user's memory recall. With this scenario, we verified the effectiveness of the developed computational model of memory recall. The evaluation experiment revealed that the model outputs shops that the participants are likely to recall, and makes the directions given easier to understand.

This paper presents a computational model for side-by-side walking for human-robot interaction (HRI). In this work we address the importance of future motion utility (motion anticipation) of the two walking partners.

Ambient intelligence, ubiquitous and networked robots, cloud robotics, are new research hot topics that start to gain popularity among the robotics community. They enable robots to acquire richer functionalities and open the way for the composition of a variety of robotic services with three functions: semantic perception, reasoning and actuation. This paper introduces the recent challenges and future trends of these topics.

This paper focuses on the problem of environmental noises in human-human communication and in automatic speech recognition. To deal with this problem, the use of alternative acoustic sensors -which are attached to the talker and receive the uttered speech through skin or bones- is investigated. In the current study, throat microphones and ear bone microphones are integrated with standard microphones using several fusion methods. The results obtained show that the recognition rates in noisy environments are drastically increased when these sensors are integrated with standard microphones. Moreover, the system does not show any recognition degradations in clean environments. In fact, recognition rates also increase slightly in clean environments. Using late fusion to integrate a throat microphone, an ear bone microphone, and a standard microphone, we achieved a 44% relative improvement in recognition rate in a noisy environment and a 24% relative improvement in recognition rate in a clean environment.

This paper presents an audio monitoring system for detecting and identifying people engaged in a conversation. The proposed method is hands-free as it uses a microphone array to acquire the sound. A particularity of the approach is the use of a laser range finder based human tracker system. The human tracker monitors the locations of people then local steered response power is used to detect the people speaking and localize precisely their mouths. Then an audio stream is created for each person and used to perform speaker identification. Experimental results show that the use of the human tracker has several benefits compared to an audio only approach.

Generating natural motion in robots is important for improving human-robot interaction. We developed a tele-operation system where the lip motion of a remote humanoid robot is automatically controlled from the operator's voice. In the present work, we introduce an improved version of our proposed speech-driven lip motion generation method, where lip height and width degrees are estimated based on vowel formant information. The method requires the calibration of only one parameter for speaker normalization. Lip height control is evaluated in two types of humanoid robots (Telenoid-R2 and Geminoid-F). Subjective evaluation indicated that the proposed audio-based method can generate lip motion with naturalness superior to vision-based and motion capture-based approaches. Partial lip width control was shown to improve lip motion naturalness in Geminoid-F, which also has an actuator for stretching the lip corners. Issues regarding online real-time processing are also discussed.

This study focuses on joint motion patterns of humans that move together with other humans or objects. Since this scope embraces `group motion', which relates only humans, and expands its extent of interactions accounting for various auxiliary instruments such as walking aids or pushcarts, we term this collective motion pattern as `coherent' motion. Coherence is proposed to be characterized by the distance between the moving parties, the scalar product of their velocities and the scalar product of the velocity vector and the displacement vector. The contribution of this study lies in the formulation of coherence in terms of the listed features through explicit mathematical models. The models are developed in accordance with a large database recorded in an uncontrolled environment involving a total of more than 500 mobile entities. The performance of the proposed models is evaluated qualitatively by comparing them to the empirical data and quantitatively by employing log-likelihoods. Comparison to an earlier work indicates that the proposed models improve the identification of coherence quality significantly well.

Summary form only given. This talk focuses on the ambient intelligence for network robot systems including robots and/or sensor networks, smartphones, etc. It is one of the significant functions allowing robot services over multiple sites. As examples of ambient intelligence on the human trajectory and behaviors, the "ambient intelligence map" is introduced. Next, the robotic services of shopping support for elderly, touring supports for wheelchair users in real shopping mall are developed using the map. The research trends and perspectives and the standardization for this technology are also introduced.

Teleoperation of multiple robots by a single operator has been studied extensively for applications such as search and navigation; however, this concept has never been applied to the field of social, conversational robots. In this paper, we explore the unique challenges posed by the remote operation of multiple social robots, where an operator must perform auditory multitasking to assist multiple interactions at once. It describes the general system requirements in four areas: social human-robot interaction (HRI) design, autonomy design, multirobot coordination, and teleoperation interface design. Based on this design framework, we have developed a system in which a single operator can simultaneously control four robots in conversational interactions with users. Key elements of our implementation include a control architecture enabling the scripting of conditional behavior flows for social interaction, a graphical interface enabling an operator to control one robot at a time while monitoring several others in the background, and a technique called “proactive timing control,” which is an automated method for smoothly interleaving the demands of multiple robots for the operator's attention. We also present metrics for describing and predicting robot performance, and we show experimental results demonstrating the effectiveness of our system through simulations and a laboratory experiment based on real-world interactions.

Awareness of time is particularly important for teleoperation of conversational robots, both for controlling the robot and for estimating interaction success, because people have a low tolerance for long pauses in conversation. Findings have shown that people engaged in high-workload tasks tend to underestimate the passage of time. This study confirms that this problem exists for operators controlling a conversational robot, and it investigates mechanisms for improving temporal awareness and task performance while minimizing workload. In a laboratory experiment, two approaches to helping an operator perform various information input tasks were compared: first, assisting temporal awareness by using a clock display, and second, using autonomy to assist one of the operator's time-dependent tasks. Results revealed that assisting the task itself, even without the clock, improved not only task performance but also the operator's temporal awareness. However, results regarding the effect of the clock were ambiguous: it increased workload in general and did not help temporal awareness overall, but it did improve temporal awareness for text entry tasks in particular. As text entry is an important task for teleoperation of social robots, we further investigated the problem of improving temporal awareness during text entry tasks. As the first experiment suggested the effectiveness of a clock, we further validated that the clock is specifically useful to improve temporal awareness. These results showed that the clock did not increase workload for text entry tasks; however, for touch-typing operators, the results suggested that showing a clock after the end of an interaction, rather than continuously throughout the task, could lower the operator's perceived workload.

Assistive robots can be perceived in two main ways: tools or partners. In past research, assistive robots that offer physical assistance for the elderly are often designed in the context of a tool metaphor. This paper investigates the effect of two design considerations for assistive robots in a partner metaphor: conversation and robot-type. The former factor is concerned with whether robots should converse with people even if the conversation is not germane for completing the task. The latter factor is concerned with whether people prefer a communication/function oriented design for assistive robots. To test these design considerations, we selected a shopping assistance situation where a robot carries a shopping basket for elderly people, which is one typical scenario used for assistive robots. A field experiment was conducted in a real supermarket in Japan where 24 elderly participants shopped with robots. The experimental results revealed that they prefer a conversational humanoid as a shopping assistant partner.

This paper presents the effect of a robot's active touch for improving people's motivation. For services in the education and healthcare fields, a robot might be useful for improving the motivation of performing such repetitive and monotonous tasks as exercising or taking medicine. Previous research demonstrated with a robot the effect of user touch on improving its impressions, but they did not clarify whether a robot's touch, especially an active touch, has enough influence on people's motive. We implemented an active touch behavior and experimentally investigated its effect on motivation. In the experiment, a robot requested participants to perform a monotonous task with a robot's active touch, a passive touch, or no touch. The result of experiment showed that an active touch by a robot increased the number of working actions and the amount of working time for the task. This suggests that a robot's active touch can support people to improve their motivation. We believe that a robot's active touch behavior is useful for such robot's services as education and healthcare.

This paper reports an ethnographic study on the use of a conversational robot. We placed a robot for 3.5 months in an elderly care center. Assuming a real deployment scenario, the robot was managed by a single non-programmer person during the field trial, who teleoperated the robot and updated the contents. The robot was designed to engage in daily greetings and chatting with elderly people. Through the ethnographic approach, we clarified how the elderly people interacted with this conversational robot, how the deployment process adopted to introduce the robot was designed, and how the organization's personnel involved themselves in this deployment.

Environment attributes are perceived or remembered differently according to the perspective used. In this study, two different perspectives, a survey perspective and a route perspective, are explained and discussed. This paper proposes an approach for modeling human environments from a route perspective, which is the perspective used when a human navigates through the environment. The process for route perspective semi-autonomous data extraction and modeling by a mobile robot equipped with a laser sensor and a camera is detailed. Finally, as an example of a route perspective application, a route direction robot was developed and tested in a real mall environment. Experimental results show the advantages of the proposed route perspective model compared with a survey perspective approach. Moreover, the route model is comparable to the performance of an expert person giving route guidance in the mall.

This study explores the nature of the multi-robot control problem for social robots. It begins by modeling the overall structure of a human-robot team for social interactions, and implements it for specific applications to dialog-based interactions. Operator activity during control of a social robot is studied. Customer satisfaction is proposed as an important metric for evaluating the performance of a human-robot team for social interactions with customers. Based on the modeling, fan-out of a social robot team can be calculated, and the performance of the team is estimated by simulation. A field trial was conducted in a shopping mall to demonstrate a successful deployment of social robots for a real-world application with ensured performance prior to installation using our modeling and simulation approach.

This paper proposes a method that estimates customer interests by identifying a set of areas at which a customer has stopped in our experimental shop environment and recommends items through robots located on the shop's shelves. We experimentally verified that our method changed customer behaviors. The results show that the method successfully encouraged customers to visit the shelves of recommended items.

This paper presents a novel method for solving the permutation problem inherent to frequency domain blind signal separation of multiple simultaneous speakers. As conventional methods, the proposed method exploits the direction of arrival (DOA) of the different speakers to resolve the permutation. But it is designed to exploit the information from pairs of microphones that are usually discarded because of the spatial aliasing. The proposed method is based on an explicit expression of the spatial aliasing effect on the DOA estimation. By introducing a vector of integer values in the equation used to estimate the DOA, it becomes possible to compensate the spatial aliasing by solving the equation relatively to that vector. The proposed method operates sequentially along the frequency bins. First the spatial aliasing is compensated by an iterative procedure that also detects the permutations. Then the detected permutation are suppressed and the DOA are estimated using all available pairs of microphones. Some simulation results demonstrate the effectiveness of the method.

This article introduces automatic recognition of speech without any audio information. Movements of the tongue, lips, and jaw are tracked by an Electro-Magnetic Articulography (EMA) device and are used as features to create hidden Markov models (HMMs) and conduct automatic speech recognition in a conventional way. The results obtained are promising, which confirm that phonetic features characterizing articulation are as discriminating as those characterizing acoustics (except for voicing). The results also show that using tongue parameters result in a higher accuracy compared with the lip parameters.

Recently, laser-based people-tracking systems have received increasing attention for their ability to localize human subjects precisely in crowded situations. However, in a real environment, there exist many kinds of moving objects other than people, and previous methods have focused only on humans. To design a more sophisticated system, it is necessary to distinguish humans from observed objects and recognize their individual condition-for example, the kind and amount of belongings they are carrying. However, in previous methods using 2D laser range finders (LRFs), it proved difficult to recognize the type of target since all sensors observe a common horizontal plane and only the 2D contours of their targets. In this study, to recognize the type of target, we observe the 3D shapes of objects moving in their environment by installing LRFs with an angle of inclination. So far, in the area of 3D modeling, LRFs have been used to construct 3D models of static objects by moving the sensor and registering multiple views. In contrast, our method observes moving objects by using a static LRF network in the environment. Experimental results are shown to confirm the effectiveness of the proposed method.

Small informal meetings of two to four participants are very common in work environments. For this reason, a convenient way for recording and archiving these meetings is of great interest. In order to efficiently archive such meetings, an important task to address is to keep trace of “who talked when” during a meeting. This paper proposes a new multi-modal approach to tackle this speaker activity detection problem. One of the novelty of the proposed approach is that it uses a human tracker that relies on scanning laser range finders (LRFs) to localize the participants. This choice is especially relevant for robotic applications as robots are often equipped with LRFs for navigation purpose. In the proposed system, a table top microphone array in the center of the meeting room acquires the audio data while the LRF based human tracker monitors the movement of the participants. Then the speaker activity detection is performed using Gaussian mixture models that were trained before hand. An experiment reproducing a meeting configuration demonstrates the performance of the system for speaker activity detection. In particular, the proposed hands free system maintains an good level of performance compared to the use of close talking microphone while participants are simultaneously speaking.

Pitch extraction is important for communication robots, since pitch may carry information about intention, attitude or emotion expression from the user's speech. However, current pitch extraction methods are not robust enough in real noisy environments. In the present work, we propose pitch extraction methods by combining microphone array and auditory scene analysis technologies, and evaluate pitch extraction of multiple speakers in real noisy environments. Evaluation results show that the proposed ML-PSACF (maximum likelihood adaptive beamformer with peak-pruned summary autocorrelation function) contributes to reduce the effects of interference and noise, leading to improvements of 23%±5% on pitch estimation rates, in comparison to the baseline of not using array processing.

This paper proposes a simple yet novel method for recognition of certain sorts of moving entities incorporating their shape and motion patterns. Although shape features have been commonly employed in object recognition, motion characteristics are in general not integrated to geometric models. In the interest of utilizing the motion attributes, the trajectories are investigated to extract the `coherence quality' of the entities. Besides, at every step a geometric shape model is adopted and the parameters defining the shape model are utilized in obtaining the prior probabilities of the entities being a member of a particular class of interest. The coherence quality is used to get the posterior probabilities through a Bayesian approach. The main contribution of this paper is the incorporation of coherence quality in identification of moving entities. The proposed method is tested against clutter and occlusion in an uncontrolled environment with patterns collected from over 500 entities. It is shown to yield a satisfactory performance rate of 92% over the entire dataset with significant generalization capabilities without any restrictions on the application setting and with considerable occlusion and clutter.

In daily conversation, we sometimes observe a deictic interaction scene that refers to a region in a space, such as saying "please put it over there" with pointing. How can such an interaction be possible with a robot? Is it enough to simulate people's behaviors, such as utterance and pointing? Instead, we highlight the importance of simulating human cognition. In the first part of our study, we empirically demonstrate the importance of simulating human cognition of regions when a robot engages in a deictic interaction by referring to a region in a space. The experiments indicate that a robot with simulated cognition of regions improves efficiency of its deictic interaction. In the second part, we present a method for a robot to computationally simulate cognition of regions.

Head motion naturally occurs in synchrony with speech and may carry paralinguistic information, such as intention, attitude and emotion, in dialogue communication. With the aim of verifying the relationship between head motion and the dialogue acts carried by speech, analyses were conducted on motion-captured data for several speakers during natural dialogues. The analysis results first confirmed the trends of our previous work, showing that regardless of the speaker, nods frequently occur during speech utterances, not only for expressing dialogue acts such as agreement and affirmation, but also appearing at the last syllable of the phrase, in strong phrase boundaries, especially when the speaker is talking confidently, or expressing interest in the interlocutor's talk. Inter-speaker variability indicated that the frequency of head motion may vary according to the speaker's age or status, while intra-speaker variability indicated that the frequency of head motion also differs depending on the inter-personal relationship with the interlocutor. A simple model for generating nods based on rules inferred from the analysis results was proposed and evaluated in two types of humanoid robots. Subjective scores showed that the proposed model could generate head motions with naturalness comparable to the original motions.

Tour guidance is a common task of social robots. Such a robot must be able to encourage the participation of people who are not directly interacting with it. We are particularly interested in encouraging people to overhear its interaction with others, since it has often been observed that even people who hesitate to interact with a robot are willing to observe its activity. To encourage such participation as bystanders, we developed a robot that walks backwards based on observations of human tour guides. Our developed system uses a robust human tracking system that enables a robot to guide people by walking forward/backward and allows us to scrutinize people's behavior after the experiment. We conducted a field experiment to compare the ratios of overhearing in "walking forward" and "walking backward." The experimental results revealed that in fact people do more often overhear the robot's conversation in the "walking backward" condition.

This article introduces automatic speech recognition based on Electro-Magnetic Articulography (EMA). Movements of the tongue, lips, and jaw are tracked by an EMA device, which are used as features to create Hidden Markov Models (HMM) and recognize speech only from articulation, that is, without any audio information. Also, automatic phoneme recognition experiments are conducted to examine the contribution of the EMA parameters to robust speech recognition. Using feature fusion, multistream HMM fusion, and late fusion methods, noisy audio speech has been integrated with EMA speech and recognition experiments have been conducted. The achieved results show that the integration of the EMA parameters significantly increases an audio speech recognizer's accuracy, in noisy environments.

In this article, automatic recognition of Cued Speech in French based on hidden Markov models (HMMs) is presented. Cued Speech is a visual mode, which uses hand shapes in different positions and in combination with lip-patterns of speech makes all the sounds of spoken language clearly understandable to deaf and hearing-impaired people. The aim of Cued Speech is to overcome the problems of lipreading and thus enable deaf children and adults to understand full spoken language. In this study, lip shape component is fused with hand component using also multistream HMM decision fusion to realize Cued Speech recognition, and continuous phoneme recognition experiments using data from a normal-hearing and a deaf cuer were conducted. In the case of the normal-hearing cuer, the obtained phoneme accuracy was 83.5%, and in the case of the deaf cuer 82.1%.

To provide personal and location-dependent services in public spaces such as shopping malls, it is important to be able to estimate the positions and identities of people in the environment. Sensors in the environment reliably detect their current positions, but it is difficult to identify people using these sensors. On the other hand, signals from wearable sensors can be used to identify people correctly, but precise position estimation remains problematic. In this paper, we describe a novel method of integrating laser range finders (LRFs) in the environment and wearable inertial sensors. Time sequences of angular velocities estimated from both LRFs and wearable sensors are matched to identify people. Examples of tracking individuals in the environment that confirm the effectiveness of this method are shown.

This paper presents a noise cancellation method based on the ability to efficiently cancel a close target speaker contribution from the signals observed at a microphone array. The proposed method exploits this specificity in the case of the hands-free speech interface. This method is in particular able to deal with non-stationary noise. The method can be divided in three steps. First, the steering vector pointing at the target user is estimated from the covariance of the observed signals. Then the noise estimate is obtained by cancelling the user's contribution. During this step the speech pauses are also estimated. Finally a post-filter is used to suppress this estimated noise from the observed signals. The post-filter strength is controlled by using the estimated noise during the speech pauses as reference. A 20k-words dictation task in presence of non-stationary diffuse background noise at different SNR levels illustrates the effectiveness of the proposed method.

Laser range finders are a non-invasive tool which can be used for anonymously tracking the motion of people and robots in real-world environments with high accuracy. Based on a commercial system we have developed, this paper addresses two practical issues of using networks of portable laser range finders in field environments. We first describe a technique for automated calibration of sensor positions and orientations, by using velocity-based matching of observed human trajectories to define constraints between the sensors. We then propose a mechanism for detecting when a sensor has been moved out of alignment, which can be used to alert an operator of the condition and automatically exclude erroneous data from tracking calculations. After describing our techniques for solving these problems, we demonstrate the effectiveness of our calibration and error detection systems in live trials with our real-time system, as well as offline tests based on scan data recorded from field trials.

Due to the anticipated future, extensive use of robots, human beings will probably share common spaces with them. The relationships between robots and humans will be conducted at close distances. Predicting people's future positions helps robots understand human behavior and react safely and naturally. In this paper, we propose a method for predicting people's positions in crossing behaviors, i.e. different trajectories people follow when they are crossing each other. We conducted a field experiment to gather various crossing behaviors of pedestrians in a shopping mall environment and analyzed them by focusing on “hot areas” spaces where people modify their trajectories for crossing. We clustered typical crossing behaviors in hot areas and modeled them using Hidden Markov Models for predictions. Our algorithm more accurately predicts the future positions of pedestrians by considering moving direction and speed.

Social robots that provide services to humans in real environments have been developed in recent years. Such a robot should appropriately recognize its users' orders through human-like communications because of user-friendliness. However, their styles of communicating are too diverse to achieve this goal. If the robot could shape their styles, its recognition ability would be improved. An entrainment, which is a phenomenon where human's behavior is synchronized with robot's behavior, can be useful for this shaping. Previous studies have reported the entrainment occurring in the same modality, but they have given little attention to entrainment across different modalities (e.g., speech and gestures). We need to consider this cross-modal effect because human-robot interaction is inherently multi-modal. In this paper, we defined “mutual entrainment” as the entrainment across different modalities and investigated the effect of it through a laboratory experiment. We evaluate how the frequency of human pointing gestures varies with the amount of information in robot speech, and as a result, we found that the gesture frequency increased as the amount of information decreased. The results suggest that smoother human-robot communications can be achieved by shaping human behavior through mutual entrainment.

Utterance interval detection is a bottleneck for the current speech recognition performance in robots embedded in real noisy environments. In the present work, we make use of sound localization technology using a microphone array, not only for localizing, but also for detecting sound intervals of multiple sound sources. In our previous work we have implemented and evaluated sound localization in the 3D-space using the MUSIC (MUltiple SIgnal Classification) method. In the present work, we proposed a method for detecting sound intervals based on the sound directivity information inferred from the dynamics of the MUSIC spectrogram. The proposed method achieved high sound interval detection accuracies and low insertion rates compared with the previous sound localization results.

This paper presents a development framework for social robots with which developers can easily prepare communicative behaviors based on tag-based sentences. Previous literature in human-robot interaction has revealed various useful non-verbal behaviors. But for developers, integrating such a large amount of non-verbal behaviors each time they build a social robot is not realistic. The more repertory of non-verbal behaviors acquire, the larger is the burden faced by developers to implement the non-verbal behaviors. Our software, however, only requires developers to prepare sentences with simple markup language for controlling explicit non-verbal behaviors and utterances. The backend system analyzes the tag input and adds implicit non-verbal behaviors accordingly. With the help of previous literature, the system is equipped with various communicative behaviors, so that the robot autonomously adjusts its gazes, gestures, and standing points. The system's effectiveness is demonstrated with examples on robot interaction where the system receives only simple sentences with scripting language to produce complex and communicative robot behaviors.

This paper reports the persuasion effect of a robot's whispering behavior that consists of a whispering gesture and a request made in a small voice. Whispering gestures naturally make close distance and create warmth feelings with subjects, and requests in quiet voices with whispering gestures also create familiar impressions, which are effective factors of persuasion. We believe that such physical behavior as whispering is one persuasion advantage held by real robots over computers. We conducted a between-subjects experiment to investigate the effectiveness of these two factors on persuasion. In the experiment, the robot requests an annoying task of the subjects; writing as many multiplication table equations as possible. As a result, whispering gestures significantly increased the working time and the number of equations. On the other hand, the loudness of the voice in the request had no effect. We believe the results indicate the effectiveness of physical behavior for persuasion in human-robot interaction.

We present a study on people's acceptance of robots performing tasks in a city. Three different entities (a human, a human in the costume of a mascot, and a robot) performed tasks in three different scenarios: endlessly guiding, responding to irrational complaints, and picking an accidentally discarded key out of a trash. All of these tasks involve entities interacting with visitors in situations that are troublesome, i.e. dull, stressful, and dirty, respectively. For this paper, 24 participants watched 9 videos (three tasks performed by three entities) and evaluated the appropriateness of each entity for the task and the human-likeness of the entity. The results indicate that people prefer to have a robot rather than a human perform these troublesome tasks, even though these tasks require much interaction with people. In addition, the comparison with a human in costume enables us to further speculate that people's expectations of an entity deserving human rights rather than being human-like in appearance or cognitive capability would be a reason for their judgments regarding appropriateness.

This paper reports our development of a communication robot for use in a shopping mall to provide shopping information, offer route guidance, and build rapport. In the development, the major difficulties included sensing human behaviors, conversation in a noisy daily environment, and the needs of unexpected miscellaneous knowledge in the conversation. We chose a network-robot system approach, where a single robot's poor sensing capability and knowledge are supplemented by ubiquitous sensors and a human operator. The developed robot system detects a person with floor sensors to initiate interaction, identifies individuals with radio-frequency identification (RFID) tags, gives shopping information while chatting, and provides route guidance with deictic gestures. The robot was partially teleoperated to avoid the difficulty of speech recognition as well as to furnish a new kind of knowledge that only humans can flexibly provide. The information supplied by a human operator was later used to increase the robot's autonomy. For 25 days in a shopping mall, we conducted a field trial and gathered 2642 interactions. A total of 235 participants signed up to use RFID tags and, later, provided questionnaire responses. The questionnaire results are promising in terms of the visitors' perceived acceptability as well as the encouragement of their shopping activities. The results of the teleoperation analysis revealed that the amount of teleoperation gradually decreased, which is also promising.

In this paper, we report a model that allows a robot to appropriately control its position as it presents information to a user. This capability is indispensable, since in the future, many robots will function in daily situations such as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and establish a joint view. We observed human-human interaction situations, where people presented objects, and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints to establish O-space: 1) proximity to listener; 2) proximity to object; 3) listener's field of view; and 4) presenter's field of view. We also experimentally evaluate the effectiveness of our model.

Simultaneous teleoperation of mobile, social robots presents unique challenges, combining the real-time demands of conversation with the prioritized scheduling of navigational tasks. We have developed a system in which a single operator can effectively control four mobile robots performing both conversation and navigation. We compare the teleoperation requirements for mobile, social robots with those of traditional robot systems, and we identify metrics for evaluating task difficulty and operator performance for teleoperation of mobile social robots. As a proof of concept, we present an integrated priority model combining real-time conversational demands and non-real-time navigational demands for operator attention, and in a pioneering study, we apply the model and metrics in a demonstration of our multi-robot system through real-world field trials in a shopping arcade.

To explore possible robot tasks in daily life, we developed a guide robot for a shopping mall and conducted a field trial with it. The robot was designed to interact naturally with customers and to affectively provide shopping information. It was also designed to repeatedly interact with people to build a rapport; since a shopping mall is a place people repeatedly visit, it provides the chance to explicitly design a robot for multiple interactions. For this capability, we used RFID tags for person identification. The robot was semi-autonomous, partially controlled by a human operator, to cope with the difficulty of speech recognition in a real environment and to handle unexpected situations.

During conversations, speakers establish their and others' participant roles (who participates in the conversation and in what capacity)--or "footing" as termed by Goffman-using gaze cues. In this paper, we study how a robot can establish the participant roles of its conversational partners using these cues. We designed a set of gaze behaviors for Robovie to signal three kinds of participant roles: addressee, bystander, and overhearer. We evaluated our design in a controlled laboratory experiment with 72 subjects in 36 trials. In three conditions, the robot signaled to two subjects, only by means of gaze, the roles of (1) two addressees, (2) an addressee and a bystander, or (3) an addressee and an overhearer. Behavioral measures showed that subjects' participation behavior conformed to the roles that the robot communicated to them. In subjective evaluations, significant differences were observed in feelings of groupness between addressees and others and liking between overhearers and others. Participation in the conversation did not affect task performance-measured by recall of information presented by the robot-but affected subjects' ratings of how much they attended to the task.

Human communication involves a number of nonverbal cues that are seemingly unintentional, unconscious, and automatic-both in their production and perception-and convey rich information on the emotional state and intentions of an individual. One family of such cues is called "nonverbal leakage." In this paper, we explore whether people can read nonverbal leakage cues-particularly gaze cues-in humanlike robots and make inferences on robots' intentions, and whether the physical design of the robot affects these inferences. We designed a gaze cue for Geminoid-a highly humanlike android-and Robovie-a robot with stylized, abstract humanlike features-that allowed the robots to "leak" information on what they might have in mind. In a controlled laboratory experiment, we asked participants to play a game of guessing with either of the robots and evaluated how the gaze cue affected participants' task performance. We found that the gaze cue did, in fact, lead to better performance, from which we infer that the cue led to attributions of mental states and intentionality. Our results have implications for robot design, particularly for designing expression of intentionality, and for our understanding of how people respond to human social cues when they are enacted by robots.

Providing route directions is a complicated interaction. Utterances are combined with gestures and pronounced with appropriate timing. This study proposes a model for a robot that generates route directions by integrating three important crucial elements: utterances, gestures, and timing. Two research questions must be answered in this modeling process. First, is it useful to let robot perform gesture even though the information conveyed by the gesture is given by utterance as well? Second, is it useful to implement the timing at which humans speaks? Many previous studies about the natural behavior of computers and robots have learned from human speakers, such as gestures and speech timing. However, our approach is different from such previous studies. We emphasized the listener's perspective. Gestures were designed based on the usefulness, although we were influenced by the basic structure of human gestures. Timing was not based on how humans speak, but modeled from how they listen. The experimental result demonstrated the effectiveness of our approach, not only for task efficiency but also for perceived naturalness.

This paper proposes a model of approach behavior with which a robot can initiate conversation with people who are walking. We developed the model by learning from the failures in a simplistic approach behavior used in a real shopping mall. Sometimes people were unaware of the robot's presence, even when it spoke to them. Sometimes, people were not sure whether the robot was really trying to start a conversation, and they did not start talking with it even though they displayed interest. To prevent such failures, our model includes the following functions: predicting the walking behavior of people, choosing a target person, planning its approaching path, and nonverbally indicating its intention to initiate a conversation. The approach model was implemented and used in a real shopping mall. The field trial demonstrated that our model significantly improves the robot's performance in initiating conversations.

In this paper, we present our current research on developing a model of robot behavior that leads to feelings of being together using the robot's body position and orientation. Creating feelings of"being together"will be an essential skill for robots that live with humans and adapt to daily human activities such as walking together or establishing joint attention to information in the environment. We observe people's proxemic behavior in joint attention situations and develop a model of behavior for robots to detect a partner's attention shift and appropriately adjust its body position and orientation in establishing joint attention with the partner. We experimentally evaluate the effectiveness of our model, and our results demonstrate the model's effectiveness.

Social interactions unfold over time, at multiple time scales, and can be observed through multiple sensory modalities. In this paper, we propose a machine learning framework for selecting and combining low-level sensory features from different modalities to produce high-level characterizations of human-robot social interactions in real-time.

Robust localization of robots and reliable tracking of people are both critical requirements for the deployment of service robots in real-world environments. In crowded public spaces, occlusions can impede localization using on-board sensors. At the same time, teams of service robots working together need to share the locations of people and other robots on the same global coordinate system in order to provide services efficiently. To solve this problem, our approach is to use an infrastructure of sensors embedded in the environment to provide an inertial reference frame and wide-area coverage. Based on a people-tracking system we have previously established which uses laser range finders to track people's trajectories, we have developed a technique to localize a team of service robots on a shared global coordinate system. Each robot's odometry data is associated with the observed trajectory of an entity detected by the laser tracking system, and Kalman filters are used to correct rotational offsets between the robots' individual coordinate systems and the global reference frame. We present our data association and pose correction algorithms and show results demonstrating the performance of our system in a shopping arcade.

A communication robot must recognize a referred-to object to support us in daily life. However, using our wide human vocabulary, we often refer to objects in terms that are incomprehensible to the robot. This paper focuses on lexical entrainment to solve this problem. Lexical entrainment is the phenomenon of people tending to adopt the terms of their interlocutor. While this has been well studied in human-computer interaction, few published papers have approached it in human-robot interaction. To investigate how lexical entrainment occurs in human-robot interaction, we conduct experiments where people instruct the robot to move objects. Our results show that two types of lexical entrainment occur in human-robot interaction. We also discuss the effects of the state of objects on lexical entrainment. Finally, we developed a test bed system for recognizing a referred-to object on the basis of knowledge from our experiments.

With the goal of improving human-robot speech communication, the localization of multiple sound sources in the 3D-space based on the MUSIC algorithm was implemented and evaluated in a humanoid robot embedded in real noisy environments. The effects of several parameters related to the MUSIC algorithm on sound source localization and real-time performances were evaluated, for recordings in different environments. Real-time processing could be achieved by reducing the frame size to 4 ms, without degrading the sound localization performance. A method was also proposed for determination of the number of sources, which is an important parameter that influences the performance of the MUSIC algorithm. The proposed method achieved localization accuracies and insertion rates comparable with the case where the ideal number of sources is given.

In human-robot interaction, robots often fail to lead humans to intended reactions due to their limited ability to express affective nuances. In this paper, we propose a motion modification method that combines affective nuances with arbitrary motions of humanoid robots to induce humans to intended reactions by expressing affective states. The method is applicable to various humanoid robots that differ in degrees of freedom or appearances, and the affective nuances are parametrically expressed in a two-dimensional model comprised of valence and arousal. The experimental results showed that the desired affective nuances could be expressed by our method, but it also suggested some limitations. We believe that the method will contribute to interactive systems in which robots can communicate with appropriate expressions in various contexts.

This paper reports the challenges of developing multiple social robots that operate in a shopping mall. We developed a networked robot system that coordinates multiple social robots and sensors to provide efficient service to customers. It directs the tasks of robots based on their positions and people's walking behavior, manages the paths of robots, and coordinates the conversation-performance between two robots. Laser range finders were distributed in the environment to estimate people's positions. The system estimates such human walking behaviors as ¿stopping¿ or ¿idle walking¿ to direct robots to provide appropriate tasks to appropriate people. Each robot interacts with people to provide recommendation information and route information about shops. The system sometimes simultaneously uses two robots to lead people from one place to another. The field trial, which was conducted in a shopping mall where four robots interacted with 414 people, revealed the effectiveness of the network robot system for guiding people around a shopping mall as well as increasing their interest.

The Uncanny Valley hypothesis has been widely used in the areas of computer graphics and human-robot interaction to motivate research and to explain the negative impressions that participants report after exposure to highly realistic characters or robots. Despite its frequent use, empirical proof for the hypothesis remains scarce. This study empirically tested two predictions of the hypothesis: a) highly realistic robots are liked less than real humans and b) the highly realistic robot's movement decreases its likeability. The results do not support these hypotheses and hence expose a considerable weakness in the Uncanny Valley hypothesis. Anthropomorphism and likeability may be multi-dimensional constructs that cannot be projected into a two-dimensional space. We speculate that the hypothesis' popularity may stem from the explanatory escape route it offers to the developers of characters and robots. In any case, the Uncanny Valley hypothesis should no longer be used to hold back the development of highly realistic androids.

For a robot providing services to people in a public space such as a shopping mall, it is important to distinguish potential customers, such as window shoppers, from other people, such as busy commuters. In this paper, we present a series of abstraction techniques for people's trajectories and a service framework for using these techniques in a social robot, which enables a designer to make the robot proactively approach customers by only providing information about target local behavior. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks people's positions as well as their local behaviors, such as fast walking, idle walking, wandering, or stopping. We accumulated people's trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and people's typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors as well as the probable local behaviors of individuals a few seconds in the future. In a field experiment, we demonstrate that this service framework enables the robot to serve people efficiently.

Teleoperation of multiple robots has been studied extensively for applications such as robot navigation; however, this concept has never been applied to the field of social robots. To explore the unique challenges posed by the remote operation of multiple social robots, we have implemented a system in which a single operator simultaneously controls up to four robots, all engaging in communication interactions with users. We present a user inter-face designed for operating a single robot while monitoring several others in the background, then we propose methods for characterizing task difficulty and introduce a technique for improving multiple-robot performance by reducing the number of conflicts between robots demanding the operator's attention. Finally, we demonstrate the success of our system in laboratory trials based on real-world interactions.

This paper reports an initial field trial with a prototype of a semiautonomous communication robot at a train station. We developed an operator-requesting mechanism to achieve semiautonomous operation for a communication robot functioning in real environments. The operator-requesting mechanism autonomously detects situations that the robot cannot handle by itself; a human operator helps by assuming control of the robot.

This paper reports a study about system response time (SRT) in communication robots that utilize human-like social features, such as anthropomorphic appearance and conversation in natural language. Our research purpose established a design guideline for SRT in communication robots. The first experiment observed user preferences toward different SRTs in interaction with a robot. In other existing user interfaces, faster response is usually preferred. In contrast, our experimental result indicated that user preference for SRT in a communication robot is highest at one second, and user preference ratings level off at two seconds.

This paper describes a model for a robot to appropriately control its position when it presents information to a user. This capability is indispensable since in the future many robots will be functioning in daily situations as shopkeepers presenting products to customers or museum guides presenting information to visitors. Psychology research suggests that people adjust their positions to establish a joint view toward a target object. Similarly, when a robot presents an object, it should stand at an appropriate position that considers the positions of both the listener and the object to optimize the listener's field of view and to establish a joint view. We observed human-human interaction situations where people presented objects and developed a model for an information-presenting robot to appropriately adjust its position. Our model consists of four constraints for establishing O-space: (1) proximity to listener, (2) proximity to object, (3) listener's field of view, and (4) presenter's field of view. We also present an experimental evaluation of the effectiveness of our model.

For a robot providing services to people in a public space such as a train station or a shopping mall, it is important to distinguish potential customers, such as window-shoppers, from other people, such as busy commuters. In this paper, we present a series of techniques for anticipating people's behavior in a public space, mainly based on the analysis of accumulated trajectories, and we demonstrate the use of these techniques in a social robot. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks people's positions as well as their local behaviors such as fast walking, idle walking, or stopping. We accumulated people's trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and people's typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors, and also the probable local behaviors of individuals a few seconds in the future. In a field experiment we demonstrate that this system enables the robot to serve people efficiently.

For a person to feel comfortable when approaching a robot it is necessary for the robot to behave in an expected way. Peoplepsilas behavior around a robot not being aware of them were observed during a preliminary experiment. Based on those observations people were classified into four groups depending on their interest in the robot. People were tracked with a laser range finder based system, and their positions, directions and velocities were estimated. A second classification based on that information was made and the relation between the two classifications were mapped. Different actions were created for the robot to be able to react naturally to different human behaviors. In this paper we evaluate three different robot behaviors with respect to how natural they appear. One behavior that actively tries to engage people, one that passively indicates that people have been noticed and a third that makes random gestures. During an experiment test subjects were instructed to act according to the groups from the classification based on interest, and the robotpsilas performance with regard to naturalness was evaluated. Both first and third person evaluation made clear that the active and passive behavior were considered equally natural, while a robot randomly making gestures was considered much less natural.

In the near future, robots are expected to play active roles in human communities. After this time arrives, robots will need to be socially accepted by the people in the communities to which they belong. However, it remains unknown what issues must be resolved to make robots socially accepted. In this paper, we point out the three criteria with which people in communities evaluate a robot and three types of relations people have with the robot. We conducted a six-week study in an office to statistically test the relationship between the evaluations and how well a robot is accepted depending on relation types. We probe that by improving behaviors in the criteria the robot will be more accepted by each type of people. Then, the discussion are presented about the most important issues regarding the social acceptance of robots.

This paper proposes a new server platform for supporting various robotic services that interact with people. Every robot serving people in public spaces need to know: where the person is, what the person is doing, and what is happening at the place. The proposed platform is the first integrated system that can robustly measure, recognize and provide information on human activities and spatial nature in the environment, in a standard way. In the shopping mall experiment, the system is able to track and capture activities of more than 20 people simultaneously in real time. We also describe the result of robotic service experiment utilizing the power of this platform.

Robots that are supposed to work and communicate with humans in our daily life have been developed in recent years. For this kind of robots, sensitive tactile sensors are necessary to achieve haptic interactions. However, influences of its own motion on tactile sensors sometimes become larger than the signal to be detected. This paper proposes a method to reduce such noises. It is enabled by estimating and subtracting the influences from sensor outputs. The influences of robotpsilas motions on tactile sensors are estimated by using the sequence of joint angles of the robot. In our method, a partially linear model is built to estimate the influences on a tactile sensor. The robotpsilas posture space, which is represented by its joint angles, is divided into several subspaces to fit to a linear model. We conducted an experiment with a robot covered with tactile sensors to verify the validity of our method. This paper shows that the robot is able to estimate the influences on a tactile sensor in high accuracy by the model.

This paper reports a method in which a communicative robot simultaneously interacts with two persons and identifies them with passive-type RFID and floor sensors. The difficulty emanates from the association of these inputs. A passive-type RFID reader provides very accurate person identification. However, the robot cannot specify who actually used the RFID if surrounded by many people. A floor sensor provides robust tracking of people’s positions, but it does not identify any person by itself. Thus, combining these sensors into one device that provides robust tracking of position with accurate person identification would be very beneficial. Toward this problem, in our key idea, the robot keeps multiple hypotheses about the interpretation of sensor information and gradually improves their accuracy through interaction with the people. The multimodal inputs from sensors are integrated with a Bayesian network that allows us to exploit people’s behavior patterns as parameters to estimate the accuracy of our hypotheses. Our method also considers the vagueness of current hypotheses when choosing appropriate interactive behaviors. When the robot has reservations about the situation, it prefers non-conclusive behavior and tries to solve the ambiguity by performing verification behaviors. Once the robot is certain, it prefers conclusive behavior. The developed system was tested in a real daily environment, a shopping center, where we gathered interaction data between the robot and multiple people as teaching data for the Bayesian network. The experimental results revealed that the system successfully identified 79.2% of the visitors. In particular, the hypothesis-based system for behavior control improved the successful rate by 16.7%.

Vocal fry (also called creak, creaky voice, and pulse register phonation) is a voice quality that carries important linguistic or paralinguistic information, depending on the language. We propose a set of acoustic measures and a method for automatically detecting vocal fry segments in speech utterances. A glottal pulse-synchronized method is proposed to deal with the very low fundamental frequency properties of vocal fry segments, which cause problems in the classic short-term analysis methods. The proposed acoustic measures characterize power, aperiodicity, and similarity properties of vocal fry signals. The basic idea of the proposed method is to scan for local power peaks in a ldquovery short-termrdquo power contour for obtaining glottal pulse candidates, check for periodicity properties, and evaluate a similarity measure between neighboring glottal pulse candidates for deciding the possibility of being vocal fry pulses. In the periodicity analysis, autocorrelation peak properties are taken into account for avoiding misdetection of periodicity in vocal fry segments. Evaluation of the proposed acoustic measures in the automatic detection resulted in 74% correct detection, with an insertion error rate of 13%.

The application range of communication robots could be widely expanded by the use of automatic speech recognition (ASR) systems with improved robustness for noise and for speakers of different ages. In past researches, several modules have been proposed and evaluated for improving the robustness of ASR systems in noisy environments. However, this performance might be degraded when applied to robots, due to problems caused by distant speech and the robot's own noise. In this paper, we implemented the individual modules in a humanoid robot, and evaluated the ASR performance in a real-world noisy environment for adults' and children's speech. The performance of each module was verified by adding different levels of real environment noise recorded in a cafeteria. Experimental results indicated that our ASR system could achieve over 80% word accuracy in 70-dBA noise. Further evaluation of adult speech recorded in a real noisy environment resulted in 73% word accuracy.

Human beings subconsciously adapt their behaviors to a communication partner in order to make interactions run smoothly. In human-robot interactions, not only the human but also the robot is expected to adapt to its partner. Thus, to facilitate human-robot interactions, a robot should be able to read subconscious comfort and discomfort signals from humans and adjust its behavior accordingly, just like a human would. However, most previous research works expected the human to consciously give feedback, which might interfere with the aim of interaction. We propose an adaptation mechanism based on reinforcement learning that reads subconscious body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interactions. The mechanism uses gazing at the robot's face and human movement distance as subconscious body signals that indicate a human's comfort and discomfort. A pilot study with a humanoid robot that has ten interaction behaviors has been conducted. The study result of 12 subjects suggests that the proposed mechanism enables autonomous adaptation to individual preferences. Also, detailed discussion and conclusions are presented.

This paper reports a method that uses humanoid robots as a communication medium. There are many interactive robots under development, but due to their limited perception, their interactivity is still far poorer than that of humans. Our approach in this paper is to limit robots' purpose to a non-interactive medium and to look for a way to attract people's interest in the information that robots convey. We propose using robots as a passive-social medium, in which multiple robots converse with each other. We conducted a field experiment at a train station for eight days to investigate the effects of a passive-social medium.

In this research, we realize human telepresence by developing a remote-controlled android system called Geminoid HI-1. Experimental results confirm that participants felt stronger presence of the operator when he talked through the android than when he appeared on a video monitor in a video conference system. In addition, participants talked with the robot naturally and evaluated its human likeness as equal to a man on a video monitor. At this paper's conclusion, we will discuss a remote-control system for telepresence that uses a human-like android robot as a new telecommunication medium.

This paper describes a group attention control (GAC) system that enables a communication robot to simultaneously interact with many people. GAC is based on controlling social situations and indicating explicit control to unify all purposes of attention. We implemented a semi-autonomous GAC system into a communication robot that guides visitors to exhibits in a science museum and engages in free-play interactions with them. The GAC system's effectiveness was demonstrated in a two-week experiment in the museum. We believe these results will allow us to develope interactive humanoid robots that can interact effectively with groups of people.

We have taken steps towards developing a method that enables an interactive humanoid robot to adapt its speed to a walking human that it is moving together with. This is difficult because the human is simultaneously adapting to the robot. From a case study in human-human walking interaction we established a hypothesis about how to read a human's speed preference based on a relationship between humans' walking speed and their relative position in the direction of walking. We conducted two experiments to verify this hypothesis: one with two humans walking together, and one with a human subject walking with a humanoid robot, Robovie-IV. For 11 out of 15 subjects who walked with the robot, the results were consistent with the speed-position relationship of the hypothesis. We also conducted a preferred speed estimation experiment for six of the subjects. All of them were satisfied with one or more of the speeds that our algorithm estimated and four of them answered one of the speeds as the best one if the algorithm was allowed to give three options. In the paper, we also discuss the difficulties and possibilities that we learned from this preliminary trial.

This paper introduces the possibilities of symbiosis between human and communication robots fromthe viewpoint of multi-modal interfaces. Current communication abilities of robots, such as speech recognition, are insufficient for practical use and needs to be improved. A network robot system integrating ubiquitous networking and robot technologies, has been introduced in Japan, Korea and EU countries in order to improve the abilities. Recent field experiments on communication robots based on the system were made in a science museum, a train station and a shopping mall in Japan. Results suggests that network robot systems may be used more as the next-generation communication media. The improvement of communication ability causes problems on privacy policy, since the history of human robot interaction often includes personal information. For example,when a robot askes me, "Hi, Nori. I know you," and I have never met it before, how should I respond to it? Therefore, access control method based on multi-modal interfaces would be requiredand discussed. Android science will be introduced as an ultimate human interface. The research aimsto clarify the difference between "existence" for robot-like robots and "presence" for human-like robots. Once the appearance of robots becomes more similar to that of humans, how should I respond to it? The development of communication robots at our lab, including privacy policy and android science is outlined.

This paper investigates the effect on user impressions of the body direction of a stuffed doll robot in an animation system. Many systems that combine a computer display with a robot have been developed, and one of their applications is entertainment, for example, an animation system. In these systems, the robot, as a 3D agent, can be more effective than a 2D agent in helping the user enjoy the animation experience by using spatial characteristics, such as body direction, as a means of expression. The direction in which the robot faces, i.e., towards the human or towards the display, is investigated here.

We have developed a new communication robot, Robopal, which is an indoor/outdoor robot for use in human-robot interaction research in the context of daily life. Robopal's intended applications involve leading and/or following a human to a destination. Preliminary experiments have been conducted to study nonverbal cues associated with leading and following behavior, and it has been observed that some behaviors, such as glancing towards the leader or follower, appear to be role-dependent. A system for representing these behaviors with a state transition model is described, based on four kinds of interaction roles: directive, responsive, collaborative, and independent. It is proposed that behavior modeling can be simplified by using this system to represent changes in the roles the robot and human play in an interaction, and by associating appropriate behaviors to each role

This paper reports a study that estimated visitor positions, visiting patterns, and inter-human relationships at a science museum using information from RFID readers. In the science museum, we exhibited humanoid robots. Visitors were invited to wear RFID tags to interact with the robots. Visitor behavior was simultaneously observed using 20 RFID readers, distributed throughout the entire floor, that roughly measured the distances of nearby tags. We integrated the outputs from all RFID readers to estimate visitor trajectories that were used to analyze three perspectives: space, visiting patterns, and relationships. Regarding space, we identified crowded and uncrowded areas. We found several typical visiting patterns, such as visited at every exhibit and directly going to robot area. We also identified atypical visiting behavior. Regarding relationships, for example, we estimated 68% coverage of group-member relationships with 91% reliability.

In this paper we present a method for determining body orientation and pose information from laser scanner data using particle filtering with an adaptive modeling algorithm. A parametric human shape model is recursively updated to fit observed data after each resampling step of the particle filter. This updated model is then used in the likelihood estimation step for the following iteration. This method has been implemented and tested by using a network of laser range finders to observe human subjects in a variety of interactions. We present results illustrating that our method can closely track torso and arm movements even with noisy and incomplete sensor data, and we show examples of body language primitives that can be observed from this orientation and positioning information.

With the aim of automatically generating head motions during speech utterances, analyses are conducted for verifying the relations between head motions and linguistic and paralinguistic information carried by speech utterances. Motion captured data are recorded during natural dialogue, and the rotation angles are estimated from the head marker data. Analysis results showed that nods frequently occur during speech utterances, not only for expressing specific dialog acts such as agreement and affirmation, but also as indicative of syntactic or semantic units, appearing at the last syllable of the phrases, in strong phrase boundaries. Analyses are also conducted on the dependence on linguistic, prosodic and voice quality information of other head motions, like shakes and tilts, and discuss about the potentiality for their use in automatic generation of head motions. The paper also proposes a method for controlling the head actuators of an android based on the rotation angles, and evaluates the mapping from the human head motions.

The distance between robots and users increasingly shrinks and with it the potential for robots to inadvertently harm users increasingly grows. Robots need to detect ongoing haptic interaction with users via malleable skin with flexible tactile sensors covering the whole body surface in order to reduce the risk and to realize enriched interactions. In this paper, we propose a feature space based on cross-correlation between tactile sensors, and we visualized it as a 2-dimensional map which we call Somatosensory Map. We construct a haptic interaction database using scenario based haptic interaction between humans and a robot that we equipped with distributed tactile sensors covering the whole body. In the Somatosensory Map, highly correlated sensors make clusters corresponding to distinctive haptic interactions between the robot and the subject. Each haptic interaction can thus be classified using cross-correlation between tactile sensors. To confirm the validity of our proposal, we apply the K-Nearest Neighbor (KNN) method using the feature space. As a validation test result, despite KNN being one of the simplest methods, over 60% of haptic interaction in which we may expect subject’s touches can be recognized. In conclusion, the cross-correlation includes good features to classify the interaction. As future work, the classifier should be improved to use effective cross-correlation shown as distribution of distinctive clusters in the Somatosensory Map.

A simple view of deictic communication only includes the indication process and recognition process: a person points at an object and says something about it such as “look at this,” and then the other person recognizes the pointing gesture and pays attention to the indicated object. However, this simple view lacks three important processes: attention synchronization, context focus, and believability establishment. We refer to these three processes as “facilitation processes” and implement them in a humanoid robot with a motion capturing system. An experiment with 30 subjects revealed that the facilitation processes make deictic communication natural.

Recently, remote-controlled communication robots have been studied. Our question is whether the person interacting with them feels that he/she is interacting with the robot itself or the human behind it. How do such beliefs affect interaction? We conducted an experiment to study the effect of a human presence behind an interactive humanoid robot. We used an interactive robot system based on a motion-capturing system where participants can bodily interact with the robot. They were deceptively informed that they would either be interacting with a robot controlled by a program or a human; in fact, in both conditions the robot was autonomous. Experimental results indicated that two-thirds of the participants felt that they were interacting with the robot itself. Their enjoyment was unaffected by the knowledge of whether the robot was controlled by a program or a human, although their impression of robot intelligence indicated that they distinguished between these conditions. On the contrary, since the remaining one-third of the participants felt they were interacting with a human behind the robot, they were affected by the knowledge: when they were told they were interacting with a teleoperated robot, they enjoyed it less; in contrast, their enjoyment increased when they were told they were interacting with an autonomous robot.

The uncanny valley theory proposed by Mori in 1970 has been a hot topic in human robot interaction research, in particular since the development of increasingly human-like androids and computer graphics. In this paper we describe an empirical study that attempts to plot Mori's hypothesized curve. In addition, the influence of framing on the users' perception of the stimuli was investigated. Framing had no significant influence on the measurements. The pictures of robots and humans were rated independently of whether the participants knew a particular picture showed a robot or human. Anthropomorphism had a significant influence on the measurements, but not even pictures of real humans were rated as likeable as the pictures of humanoids or toy robots. As a result we suggest the existence of an uncanny cliff model as an alternative to the uncanny valley model. However, this study focused on the perception of pictures of robots and the results, including the suggested model, may be different for the perception of movies of moving robots or the perception of standing right in front of a moving robot.

We believe that lifelikeness is important to make it possible for a communication robot to interact naturally with people. In this research, the relationship between contingency and complexity in a lifelike robot was investigated. We have developed a robot control system to control experimentally contingency and complexity in interaction by combining a humanoid robot with a motion-capturing system. Two independent experiments were conducted with different levels of interaction complexity: simple interaction and complex interaction. Experiments in the simple interaction situation indicated that subjects felt that the robot behaved autonomously when contingency was low, and there was no significant relationship between contingency and a lifelike impression. On the other hand, experiments in the complex interaction situation showed that the robot gave the subjects the impression of being more autonomous and lifelike when contingency was high.

In this article, we propose “Family Ensemble,” a piano duo support system for a musically inept parent and his/her child who is a beginner at playing the piano. The system makes it easier for parents to correctly reproduce a given sequence of pitches along with the child's performance by using score tracking and note-replacement functions. The experiments with this support system showed that the parents can immediately participate in the piano duo. Furthermore, we found that during joint practices using Family Ensemble some subjects discussed musical ideas that they would not have talked about without using the system.

The development of interactive robots is a complicated process, involving a plethora of psychological, technical, and contextual influences. To design a robot capable of operating ldquointelligentlyrdquo in everyday situations, one needs a profound understanding of human-robot interaction (HRI). We propose an approach based on integral analysis of multimodal data to pursue this understanding and support interdisciplinary research and development in the field of robotics. To adopt this approach, a software tool named interaction debugger was developed that features user-friendly navigation, browsing, searching, viewing, and annotation of data; it enables fine-grained inspection of the HRI. In four case studies, we demonstrated how our analysis approach aids the development process of interactive robots.

Along with the development of interactive robots, controlled experiments and field trials are regularly conducted to stage human-robot interaction. Experience in this field has shown that analyzing human-robot interaction for evaluation purposes fosters the development of improved systems and the generation of new knowledge. In this paper, we present the interaction debugging approach, which is based on the collection and analysis of data from robots and their environment. Considering the multimodality of robotic technology, often only audio and video are insufficient for detailed analysis of human-robot interaction. Therefore, in our analysis we integrate multimodal information using audio, video, sensory data, and intermediate variables. An important aspect of the interaction debugging approach is using a tool called Interaction Debugger to analyze data. By supporting user-friendly data presentation, annotation and navigation, Interaction Debugger enables fine-grained inspection of human-robot interaction. The main goal of this paper is to address how an integral approach to the analysis of human-robot interaction can be adopted. This is demonstrated by three case studies.

This paper investigates human behavior around an interactive robot at a science museum. To develop a communication robot that works in daily environments, it is important to investigate the available information from a robot about people's behavior. Such information will enable the robot to predict people's behavior so that the robot can optimize its interactive behavior. We analyzed visitor behavior toward a simple interactive robot exhibited at a science museum in relation to information from sound level and range sensors. We discovered factors that influence the way people approach, maintain distance, and interact both physically and verbally with the robot. This enabled us to extract meaningful information from the sensory information and apply it to communication robots.

This paper reports on a field trial with interactive humanoid robots at a science museum where visitors are supposed to study and develop an interest in science. In the trial, each visitor wore an RFID tag while looking around the museum's exhibits. Information obtained from the RFID tags was used to direct the robots' interaction with the visitors. The robots autonomously interacted with visitors via gestures and utterances resembling the free play of children [1]. In addition, they performed exhibit-guiding by moving around several exhibits and explaining the exhibits based on sensor information. The robots were highly evaluated by visitors during the two-month trial. Moreover, we conducted an experiment in the field trial to compare the detailed effects of exhibit-guiding and free-play interaction under three operating conditions. This revealed that the combination of the free-play interaction and exhibit-guiding positively affected visitors' experiences at the science museum.

The purpose of our research is to develop lifelike behavior in a communication robot, which is expected to potentially make human-robot interaction more natural. Our earlier research demonstrated the importance of a robot's contingency for lifelikeness [1]. On the other hand, perfect contingency seems to give us a non-lifelike impression. In order to explore the appropriate contingency for communication robots, we developed a robot system that allows us to adjust its contingency to an interacting person in a simple mimic interaction. As a result of an experiment, we identified the relationships between the degree of contingency and the subjective impressions of lifelikeness, autonomy, and preference. However, the experimental result also seems to suggest the importance of the complexity of interaction for investigating the appropriate contingency of communication robots.

This paper discusses communication robots as next-generation media of communication. The "Network Robots", a new framework for integrating ubiquitous network and robot technologies, is a step towards providing infrastructure to make robots into communication media. Based on this framework, communication robots have come into greater use by being networked with humans, cell-phones and ubiquitous sensors (cameras, tags, wearable computers, etc.). This paper introduces the communication robots developed at ATR. Three kinds of field experiments at elementary school, at a science museum, and a Japanese comedy conversation by two robots, called "Robot Manzai" are also introduced. The results of field experiments indicate the potential of network robots as communication media

This paper presents an analysis on the functions carried by phrase final tones in turn-taking and dialog acts, taking into account linguistic information about the part of speech (particles and auxiliary verbs) attributed to the morphemes at phrase finals. Natural conversational speech data are segmented in inter-pause units, and each utterance unit is arranged according to the phrase final morphemes. Turntaking functions are annotated, and tones of each phrase final are described by acoustic-prosodic features. Analysis results show a relationship between tones and turn-taking functions in most of the morphemes, while no clear relationship is found in some classes of morphemes which are final particles.

Aiming to realize a non-verbal communication between humans and robots, the use of acoustic parameters related with voice quality features, besides classical prosodic features, is proposed and evaluated for automatic extraction of paralinguistic information (intentions, attitudes, and emotions) in dialog speech. Experimental results indicated that prosodic features were effective for detecting groups of paralinguistic information expressing specific functions (such as affirmation, denial, and asking for repetition), accounting for 61% of the global identification rate. Voice quality features were effective for detecting part of the paralinguistic information expressing emotions or attitudes (such as surprise, disgust and admiration), leading to 12 % improvement in the global identification rate

We believe that robots will take an active part in our daily lives in the near future. Once this time arrives, robots will be expected to have the social skills necessary for interacting with people in addition to the ability to carry out their own tasks. We call an interaction that increases familiarity and makes communication smoother a "social interaction." We have recently developed a human-size humanoid, called "Robovie-IV," which will have the ability to interact with people in their daily lives. This paper discusses the design requirements for humanrobot interaction by a communication robot, then introduces an overview of Robovie-IV's hardware and software architectures. There are also preliminary results of a daily experiment that we are currently conducting with Robovie-IV in our office.

This paper describes a method for recognizing the environment and selecting behaviors for a humanoid robot that are appropriate to the environment, based on sensor histories. In order for humanoid robots to move around many kinds of environments, one of the keys is selecting the appropriate behavior. Because features of an environment that deeply influence a robot's motion, such as viscous friction or floor elasticity, are difficult to immediately measure before moving, it is difficult for the robots to plan their behaviors to take these factors into account. Therefore, in our method, a robot first records a long time series of data from sensors attached to its body while performing each behavior in each environment, and it builds decision trees based on this data for each of the behaviors, enabling it to recognize the current environment based on the sensor history. By using the trees, the robot selects the behavior which will be most effective for recognizing the environment, acquires a sensor history while doing the behavior, and selects likely candidates for the environment. By iterating over these steps, the robot can recognize the environment and select an appropriate behavior. To verify the validity of the method, we used a small-size humanoid robot and conducted an experiment to recognize environments in a family's house

This paper presents a three-layer model for generation and recognition of attention-drawing behavior. The model enables a robot to recognize people's attention-drawing behavior as well as to perform attention-drawing behavior to people. It consists of three layers: the PSM (pointing space model), the RTM (reference term model), and the OPM (object property model). The PSM associates the pointing gesture with a reference term, the RTM associates positional relationships with a reference term, and the OPM associates other supplemental verbal cues with a reference term. We implemented the model in a humanoid robot, Robovie, and verified its effectiveness through an experiment

In face-to-face communication, eyes play a central role, for example in directing attention and regulating turn-taking. For this reason, it has been a central topic in several fields of interaction study. Although many psychology findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, it is this topic we address in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we propose methods of controlling a robot's gaze responsively to its partner's gaze and confirm the effect of this on the feeling of being looked at, which is considered to be the basis of conveying impressions using gaze in face-to-face interaction experiments. Furthermore, an additional preliminary experiment with an on-screen agent shows the possibility of using blinking behaviour as another modality for responding to a partner

In this paper, we propose natural reflexive behaviors for wheeled inverted pendulum type humanoid robots and show that such behaviors change human impressions of the robot. To achieve human-like communication between humans and robots, the latter need to be recognized by the former as communication partners. We believe that the natural reflexive behaviors of robots play an important role in being human partners. To validate the evidence, we generate four types of reflexive robot behaviors when pushed by someone and conducted preliminary experiments with subjects. We verified that people change their impressions of extroversion, agreeableness, intellect, and neuroticism for the robot by changing the reflexive behaviors

This paper introduces a "teleoperated communication robot" whose unique point is that its language component is performed by an operator to avoid the automatic recognition difficulty of spoken language. On the other hand, conceptually, its nonverbal behavior will be autonomously controlled. One important point is to identify whether ordinary people will accept this type of communication robot. Therefore, we conducted a preliminary field trial at a shopping center, which revealed positive perspectives as well as technical problems to be solved

Coding can contribute to robot design by suggesting behavioral benchmarks. These, however, depend on the level of analysis. In illustration, semi-formalized rules are used to investigate child-robot encounters. By using behavior-level codes, we extract information about how children use the robot. This leads to findings about longitudinal changes in how children evaluate its behaviors. Children, we find, use the robot as a social mediator - to prompt synchronized social events. By focusing on a behavioral level, coding can benefit designers of robots, software and sensors

In face-to-face communication, eyes play a central role, for example, in directing attention and regulating turn-taking. For this reason, the eyes have been a central topic in several fields of interaction study. Although many psychological findings have encouraged previous work in both human-computer and human-robot interaction studies, so far there have been few explorations on how to move the agent's eye, including when to move it, for communication. Therefore, we investigate this topic in this study. The impression a person forms from an interaction is strongly influenced by the degree to which their partner's gaze direction correlates with their own. In this paper, we evaluate two primitive ways of controlling a robot's gaze responsively to its partner's gaze by comparison with two additional ways of non-responsive gaze control in an experiment set in a conversational situation. Consequently, we confirm that robots with different types of responsive gaze control can give subjects different impressions of its characteristics

When describing a physical object, we indicate which object by pointing and using reference terms, such as ‘this’ and ‘that’, to inform the listener quickly of an indicated object's location. Therefore, this research proposes using a three-layer attention-drawing model for humanoid robots that incorporates such gestures and verbal cues. The proposed three-layer model consists of three sub-models: the Reference Term Model (RTM); the Limit Distance Model (LDM); and the Object Property Model (OPM). The RTM selects an appropriate reference term for distance, based on a quantitative analysis of human behaviour. The LDM decides whether to use a property of the object, such as colour, as an additional term for distinguishing the object from its neighbours. The OPM determines which property should be used for this additional reference. Based on this concept, an attention-drawing system was developed for a communication robot named ‘Robovie’, and its effectiveness was tested.

Vocal fry is a voice quality that often appears in relaxed voices indicating low tension, or in pressed voices expressing attitudes/feelings of surprise, admiration and suffering. We propose a set of acoustic measures for automatically detecting vocal fry segments in speech utterances. In order to deal with vocal fry utterances with very low fundamental frequencies, where classic short-term analysis methods become problematic, a glottal pulse synchronized method is proposed. The acoustic measures are based on power, periodicity and similarity properties of vocal fry signals. The basic idea is to scan for power peaks in a “very short-term ” power contour, check for periodicity properties and evaluate a similarity measure between power peaks for deciding the possibility of vocal fry pulses. Sub-harmonic properties are also taken into account in the periodicity analysis. Evaluation of the proposed measures in automatic detection resulted in 73.3 % correct detection, with an insertion error rate of 3.9 %. 1.

In this paper, we propose an adaptation mechanism for robot behaviors to make robot-human interactions run more smoothly. We propose such a mechanism based on reinforcement learning, which reads minute body signals from a human partner, and uses this information to adjust interaction distances, gaze meeting, and motion speed and timing in human-robot interaction. We show that this enables autonomous adaptation to individual preferences by an experiment with twelve subjects.

When we talk about objects in an environment, we indicate to a listener which object is currently under consideration by using pointing gesture and such reference terms as "this" and "that". Such reference terms play an important role in human interaction by quickly informing the listener of an indicated object's location. In this research, we propose a three-layered draw-attention model for humanoid robots with gestures and verbal cues. Our proposed three-layered model consists of three sub models: reference term model (RTM), limit distance model (LDM) and object property model (OPM). RTM decides an appropriate reference term using functions constructed by an analysis of human behavior. LDM decides whether to use the object's property with a reference term. OPM decides the appropriate property for indicating the object by comparing object properties with each other. We developed an attention drawing system in a communication robot named "Robovie" based on the three layered model. We confirmed its effectiveness through the experiments.

The paper proposes a novel touch-based multi-user interaction environment for browsing images from the Internet. A large touchable screen serves as a shared data space where multiple users simultaneously interact with Internet images by using simple hand movements and gestures. Users can select, bookmark, clean up and refine the search of the incoming images. The system provides a natural and easy-to-use interface to browse and select items from large databases, with uses varying from discussion and brain storming support to edutainment and fun. Early preliminary results show the suitability of our image interactive visualization method for browsing large image collections.

The SenseWeb system is a multi-user interactive information environment. Aimed at supporting the sharing of experiences and collaboration among multiple users allowing them to simultaneously interact with digital multimedia elements by using their bare hands. By sharing a common information space on a large screen, users can cooperatively search, filter, classify and interact with multimedia data in a natural and intuitive way. This paper introduces the system as well as preliminary experiment results to assess its effectiveness as a simultaneous multi-user interaction environment in collaborative image classification tasks. Results confirm our expectations of improvements in task completion time, ease of use and user satisfaction over multi-user but one-at-the-time interaction scenarios.

We propose an intelligent environment for "co-experience communication" consisting of stationary and wearable sensors, and communication robots. The environment captures many events collaboratively from multiple viewpoints using a combination of stationary sensors that track people's behaviors and interactions, and wearable sensors that observe the viewpoints and areas of interest of individual persons. Humanoid communication robots also play a role as assistants to help visitors pay attention to significant events. First of all, the basic framework of the environment is described. Next, our developing exhibition room is introduced as an example of an intelligent environment.

This paper proposes a concept of robots called ubiquitous intelligent robot and introduces our research efforts toward establishing its fundamental technology. A ubiquitous intelligent robot is a kind of embodied intelligence that encompasses the sphere of people's activities, records these activities, and supports people. Under this concept, we have developed an intelligent environment for capturing our experiences and a wearable event recording system for medical care. These developments mainly focus on recording people's activities along with their environments.

We propose a gaze movement model that enables an embodied interface agent to convey different impressions to users. Managing one's own impression to influence the behaviors of others plays an important role in human communications. To create a new application area which involves agents in this kind of social interaction, interface agents that manage their impressions are required. For this purpose, we build the gaze movement model based on three gaze parameters picked from a large number of psychological studies: amount of gaze, mean duration of gaze, and gaze points while averted. In this paper, we introduce the gaze movement model and gaze parameters. We then present an experiment in which subjects evaluated the impressions created by nine gaze patterns produced by altering the gaze parameters. The results indicate that reproducible relations exist between the gaze parameters and impressions, which shows the validity of the model

Conventional features are robust for recognizing either deformed or degraded characters. This paper proposes a feature extraction method that is robust for both of them. Run-length compensation is introduced for extracting approximate directional run-lengths of strokes from degraded handwritten characters. This technique is applied to the conventional feature vector based on directional run-lengths. Experiments for handwritten characters with additive or subtractive noise show that the proposed feature is superior to conventional ones over a wide range of the degree of noise.

Proposes a method for adaptively acquiring templates for degraded characters in scene images. Characters in scene images are often degraded because of poor printing and viewing conditions. To cope with the degradation problem, we proposed the idea of "context-based image templates" which include neighboring characters of parts thereof and so represent more contextual information than single-letter templates. However, our previous method manually selects the learning samples to make the context-based image templates and is time-consuming. Therefore, we attempt to make the context-based image templates automatically from single-letter templates and learning text-line images. The context-based image templates are iteratively created using the k-nearest neighbor rule. Experiments with 3,467 alpha-numeric characters in nine bookshelf images show that the high recognition rates for test samples possible with this method asymptotically approach those achieved with manual selection.

This paper proposes a method for recognizing degraded characters in bookshelf images captured by a digital camera. We adopt displacement matching and templates that include neighboring characters or parts thereof to cope with the degradation. The templates are referred as context-based image templates, since they offer more contextual information than single-letter templates. Such templates are effective wherever there is a restricted word set, such as journal titles, year, month, volume, and number. Experiments with 3,468 characters in nine bookshelf images show that this method achieves a higher recognition rare (96.3%) than single-letter templates (88.4%).

Presents a multiple-dictionary method for recognizing low-quality characters in scene images. First, the environmental conditions of an input image are estimated using an initial dictionary. Then, a relevant dictionary from multiple dictionaries reflecting different environmental conditions is automatically selected from the estimation and used for recognition. Experiments are made for characters in images of bookshelves. The results show that the proposed method achieves a higher recognition rate (89.8%) than that obtained by using a single dictionary (76.4%). Furthermore, recognition accuracy improves from 89.8% to 95.2% using contextual postprocessing.

A method for recognizing characters on graphical designs is proposed. A new projection feature that separates text-line regions from backgrounds, and adaptive thresholding in displacement matching are introduced. Experimental results for newspaper headlines with graphical designs show a recognition rate of 97.7 percent.

The paper proposes a method for recognizing characters in gray-scale images with graphical designs and noise. The authors' previous method for binary images is robust against graphical designs, but it is easily affected by binarization conditions or scanning conditions. Therefore, they improve the method to be tolerant for such conditions by handling gray-scaled images directly. First, a projection profile for text-line extraction and a similarity value for character recognition are expanded for gray-scaled images. Next, a learning method of adaptive threshold values against the degree of degradation and variations of intensity levels are introduced to suppress spurious candidates during the recognition process. Experimental results for fifty Japanese newspaper headlines show that the proposed method achieves higher recognition rates than their previous method in the wide range of scanning brightness conditions.

This paper describes a two-stage recognition method that reduces the calculation load of correlation and improves recognition accuracy in statistical image recognition. It consists of an image screening and recognition stage. Image screening selects a candidate set of subimages that are similar to the object class using a lower dimensional feature vector. Since recognition is made for the selected subimages set using a higher dimensional feature vector, overall recognition efficiency is improved. The classifier in recognition designed from the selected subimages also improves recognition accuracy because selected subimages are less contaminated than the original ones. A screening criterion for measuring overall efficiency and accuracy of recognition is introduced to be exploited in designing the feature spaces of image screening and recognition. The results of experiments for the eye- and mouth-area detection in face images and text-area detection in document images show that the designed feature spaces improve recognition accuracy and more efficiency than does the conventional one-stage recognition method.

This paper describes the process model for a system that recognizes the rhythm, chords, and source-separated musical notes in monaural music signals. The model consists of multiple processing modules and a MRF (Markov random field)-based hypothesis network for integration of multiple sources of information. Because the MRP enables information to be integrated on a multiply connected hypothesis network, the results of evaluation experiments show that the present system recognizes notes better than does a system based on a singly connected Bayesian hypothesis network.

The conventional OCR fails to recognize most characters in Japanese newspaper headlines with graphical designs because of the difficulty of removing the designs. This paper proposes a method that recognizes such characters without removing the designs. First, text-line regions are extracted from a local distribution of the combination of black and white runs observed in a rectangular window while the window is shifted pixel-by-pixel in the direction of the text-line. Characters in the extracted text-line region are then recognized by displacement matching. Adaptive thresholding against the degree of degradation suppresses spurious candidates yielded by displacement matching even with graphical designs. Experimental results for fifty Japanese newspaper headlines show that the method achieves a recognition rate of 97.7%, much higher than a conventional method (17.0%).

A new preprocessing method, called image screening, is presented for improving the recognition rate and efficiency in statistical image recognition. The problem of detecting a specified object in an input image is treated as a two-class classification problem in which the image falls into a set of subimages in the target object (figure) class and the other set of subimages in the ground class. An image screening algorithm selects a candidate set of subimages which are similar to the object class and rejects the remaining set using screening filters whose design is based on projection pursuit. The classifiers of the two classes are taken from the candidate set. Experimental results for eye and mouth detection in face images and text-area detection in document images demonstrate that this method improves the recognition rate and efficiency.

A neural network model is proposed to obtain a numerical description of pain mechanisms. The modeling presented here is based on the various assumptions made by the results of physiological and anatomical studies reported in the literature and by ourselves. Those studies, especially on the neural connections between the neural units concerned in the pain mechanisms do not give conclusive evidence, and some of the results are claimed by other investigators. The assumptions used are unverified for this reason. The quantitative model presented is only a simplified one and simulates only one directional ascending and descending pathway for the pain sensation in which peripheral receptors, afferent A, A 5, and C fibers, and the receptive cells of spinal cord, brain stem, thalamus, and the cerebral cortex are involved. No interactions from the lateral adjacent fields such as lateral inhibition and facilitation have been proposed, and no analytical elucidation of spatial information processing mechanisms has been made. Only the firing characteristics of the neural cells related to pain generation are investigated and compared with the physiological results. Adaptation effect and conduction velocity of neural fibers are considered in the model, however the fibers in each neural unit are assumed to have constant conduction velocity and firing threshold. Model simulation has been carried out for the single square-wave pulse and the periodic repetitive pulse stimulation applied on peripheral receptors.