This study investigates how nudge design in robotic plates can improve children's eating habits. Establishing healthy dietary habits in childhood is crucial for their growth and development and can decrease the risk of developing diseases later in life. Traditional coercive methods for dietary change have been largely ineffective. In contrast, nudge design, which respects user autonomy and reduces negative reactions, appears to be a promising strategy for positively influencing children's dietary behaviors. We developed a robotic plate based on nudge design principles, incorporating rainbow metaphors, to encourage children to explore a variety of foods and mitigate picky eating tendencies. The study included comparative experiments with two control groups. The findings indicated that the robotic plate with rainbow metaphors attracted more interest, received better evaluations, and was deemed more appropriate and useful than the control plates. Although there were no significant differences in how the plates were used according to observational data, interviews with participants showed a preference for the robotic plate and an increased willingness to try new, healthy foods. The study concludes that nudge design, particularly when using rainbow metaphors, is more effective with younger children, potentially improving their dietary habits. Nonetheless, the influence of such designs on older children might be restricted due to their advanced cognitive development and greater susceptibility to external influences.

Nunchi is a high context communication skill which makes an individual understand the counterpart’s indirect social cues and respond appropriately. Robotic things can perceive and recognize situations, and express appropriate responses to situations. Thus, this study was conducted with the expectation that the robotic things equipped with Nunchi, which understand the situation well and behave appropriately instead of the workers, would allow the workers to focus on their work. In order to investigate the effect of robotization of the objects on the degree to which Nunchi is required for the participant, task load, and the impression of objects according to the socio-relational context, a 2 (socio-relational context: stranger group vs. friend group) X 2 (robotization: robotic things vs. baseline things) mixed-participant experiment was designed. As a result, participants evaluated robotic things as more useful and social than baseline things. In addition, Nunchi was required less for the people in the stranger group when robotic things were used in their work than when baseline things were used. Finally, participants felt their task load reduced when the robotic things were used.

How do native English speakers and native Korean speakers politely make a request to a robot? Previous human-robot interaction studies on English have demonstrated that humans use indirect speech acts (ISAs) frequently to robots to make their requests polite. However, it is unknown whether humans considerably used ISAs to robots in other languages. In addition to ISAs, Korean has other politeness expressions called honorifics, which indicate different politeness from that of ISAs. This study aimed to investigate the cultural differences in humans' politeness expressions and politeness when they make requests to robots and to re-examine the effect of conventionality of context on the use of politeness expressions. We conducted a replication experiment of Williams et al. (2018) on native Korean speakers and analyzed their use of ISAs and honorifics. Our results showed that ISAs are rarely used in task-based human-robot interaction in Korean. Instead, honorifics are more frequently used than ISAs and are more common in conventionalized contexts than in unconventionalized contexts. These results suggest that the difference in politeness expressions and politeness between English and Korean exist in both human-robot interaction and human-human interaction. Furthermore, the conventionality of context has a strong constraint on making humans follow social norms in both languages.

Social relations within a group are one of the factors that build the social context. Less attention has been paid to social relations within a group for human-robot interaction design. In this study, we designed different types of service behaviors for social relations in a mixed age group and conducted an experiment to investigate the effect of service behavior types (serving the elderly first versus serving the young first versus serving without priority) on a user's evaluation of a service robot. We only found that service evaluation and appropriateness of the robot were rated less positively under the condition of serving the young first than under other conditions. In addition, we found that the participants evaluated the robot serving the young first as impolite. The effect of the robot's behaviors on service evaluation and appropriateness was mediated by politeness. The current study provides an initial basis for human-robot interaction design for social norms in group-robot interaction.

With becoming an aging society rapidly, the digital divide among the elderly is expected to emerge as a social problem. In order to bridge the digital gap between the elderly and the general public, it is necessary to find out what difficulties the elderly have in using information and communication technology (ICT) and what technologies and services they need. In order to figure out those issues, we conducted three phases of study: ICT literacy survey, interview, and design workshop (N=10). This study has three major findings. First, from the physical view, the elderly found ICT that reduces their movement distance attractive. Second, from the cognitive perspective, services which reduce the gap between the real physical ability of the elderly and the perceived ability of them by themselves are needed. Lastly, from the psychological view, an intuitive and natural interface is needed for the elderly who are afraid of learning new technology.

This study explores the effects of the positioning strategy of domestic social robots on the purchase intention of consumers. Specifically, the authors investigate the effects of robot positioning as companions with as assistants and as appliances. The study results showed that the participants preferred the domestic social robots positioned as assistants rather than as companions. Moreover, for male participants, the positioning of domestic social robots as appliances was also preferred over robots positioned as companions. The study results also showed that the effects of positioning on the purchase intention were mediated by the participants’ perception of usefulness regarding the robot.

Due to advances in technology, various types of existing products can be robotized. The interaction design between the robotized objects which are the robotized products and users has become an emerging issue since how to interact affects user acceptance of the new products. Because the robots can perceive, recognize and assist the users, robotized objects can assist users by reading the context and guessing their intentions in advance, or can assist users in response to users’ clear requests. In order to explore the effective interaction strategy of robotized object, we conduct 2 × 2 mixed-participant experiment. As a result, participants were more surprised by and felt positive emotion toward robotized objects with proactive interaction than with reactive interaction. In addition, the robotized object with proactive interaction was evaluated as more intelligent and easier to use than that with reactive interaction. The influence of interaction strategies on satisfaction of robotized objects was mediated by surprise and positive emotions. Furthermore, the robotized object with proactive interaction was evaluated as more positive when the user’s intention was expressed only by action cue than when it was expressed by both verbal cue and action cue.

A non-lexical backchannel is known to be dependent on the conversational context, and its form can be distinguished by the social relation between the speaker and the listener in the Korean language. Thus, to investigate the effect of a non-lexical backchannel, we conducted a 2 (context: information-centric versus emotion-centric) × 3 (forms of backchannel: "ne" versus "eo" versus "eum") mixed-participant experiment (N = 96). After watching video stimuli, the participants evaluated a robot using one of the three forms of non-lexical backchannels under both contexts. No significant main effect of the forms of the non-lexical backchannel was found. By contrast, we found that task attraction and appropriateness of the robot were rated more positively under the information-centric context than under the emotion-centric context. The functions of the non-lexical backchannel of attentive listening and understanding were manifested more under the information-centric context than under the emotion-centric context. Furthermore, these functions mediated the effect of conversational context on the user perception of the robot in terms of task attraction and appropriateness.

Inducing a user’s behavior through social interaction is a goal that a social robot aims to achieve. It has been argued that empathy has a strong effect on behavior inducement. In human-human interaction, it has been verified that the influence of a nonverbal cue on empathy outweighs that of a verbal cue when those are used in a combined way. The objectives of this study are to explore if such outweighing effect of nonverbal cues is maintained in human-robot interaction (HRI) and to investigate the mechanism of communication cues’ effects by analyzing the mediation structure with the following mediator variables: perceived emotion, perceived intentionality, perceived malfunction, and empathy inducement. To this end, we conducted 2 (verbal type: positive verbal cue vs. negative verbal cue) × 2 (nonverbal type: positive nonverbal cue vs. negative nonverbal cue) within-participant experiment (N = 48). The experiment created a situation in which the social robot was harshly criticized during a conversation. The analysis of experiment results showed the outweighing effect of a nonverbal cue was maintained. When a nonverbal cue conveyed a negative, situation-accordant emotion, it had a decisive effect on perceived emotion, empathy, and behavior inducement. In contrast, a verbal cue induced participants’ empathy and behavior when it conveyed a positive, situation-discordant emotion. This inconsistency between verbal cue and nonverbal cue made the combination of positive verbal cue and negative nonverbal cue have the strongest effect. It implies that participants had different expectations for each of the two communication cues, just as they did in social interactions with human beings. It implies that participants might have applied normative expectations for social interaction with human beings to the social robot.

Smart speakers have different speech style depending on the installed artificial intelligence (AI). Furthermore, the AI agent’s appearances can make different impressions. Hence, it might give characters to the AI agent through speaking and appearance modalities. To determine how an AI agent’s speech and tactility types affect users’ perception, we designed a 2(speech types: assistant-like vs. companion-like) × 2(flexibility types: flexible vs. hard) × 2(roughness types: rough vs. smooth) mixed-participant experiment (N=48). As a result, when the speech style is like a companion, it is possible to give an impression of sociability through a flexible material. However, there was no significant difference by flexibility type when being an assistant. In addition, there was a significant interaction effect between speech types and flexibility types on usefulness. When the speech type is companion-like, the AI agent with flexible material was perceived as being more useful and providing better services than that with hard material. On the contrary, when the speech type is assistant-like, the opposite result was revealed. Regardless of the speech type, a flexible material increases the impression of sociability with higher service evaluation, and a rough finish gives the AI agent an impression of usefulness with higher service evaluation.

A social robot interacts with humans based on social intelligence, for which related applications are being developed across diverse fields to be increasingly integrated in modern society. In this regard, social intelligence and interaction are the keywords of a social robot. Social intelligence refers to the ability to control interactions or thoughts and feelings of relationships with other people; primal empathy, which is the ability to empathize by perceiving emotional signals, among the components of social intelligence was applied to the robot in this study. We proposed that the empathic ability of a social robot can be improved if the social robot can create facial expressions based on the emotional state of a user. Moreover, we suggested a framework of facial expressions for robots. These facial expressions can be repeatedly used in various social robot platforms to achieve such a strategy.

A study was conducted to investigate the effects of category labels of domestic robots on their consumer acceptance. The authors posited that compared to the label robots, a pre-existent category label such as home appliances would increase the consumers' evaluation of and purchase intention towards the products. It is suggested that the pre-existent category label helps consumers to perceive the functional values they stand to gain by consuming the product more than the label robots, which is often related to the concepts generated around cultural artifacts. The results of the study confirmed the hypotheses, and further discussions are provided in this paper.

The purpose of this study is to investigate a robot's impression perceived by users as well as the accuracy of perception of location information, which the robot provided according to the modality type of the robot. To explore this, we designed two 2 (verbal types: deictic vs. descriptive) x 2 (nose pointing: with nose vs. without nose) x 2 (eye pointing: with eyes vs. without eyes) mixed-participant studies. In the first study, we investigated the impacts of the robot's modality type in the imperative pointing situation. As a result, participants identified the robot's pointing gesture with nose as more effective, social, and positive, than the robot's pointing gesture without nose. Moreover, the descriptive speech robot was evaluated as more positive than the deictic speech robot. In terms of the accuracy of perception of location information, which the robot provided, participants identified the robot-designated chair more accurately when the robot delivered a deictic speech than when the robot delivered a descriptive speech. For the second study, we explored the effects of the robot's modality type in the declarative pointing situation. As a result, the robot's descriptive speech was rated as effective, social, natural, competent, trustworthy, and more positive than deictic speech. In the case of the robot's pointing gestures, pointing gesture with nose was evaluated as more effective, social, natural, competent, trustworthy, and positive than that without nose. In terms of the accuracy of location information perception, participants perceived the location of the object designated by the robot more accurately when the robot used descriptive speech, pointed with nose and without eyes.

In smart home environments, central interfaces that take commands from users and give orders to each relevant device appropriately are increasingly important. We investigated the type of central interface that consumers are more willing to adopt and whether these interfaces enhance the evaluation of services provided by smart home devices. This study confirms that speaker interfaces are preferred over social robots, speaker interfaces are perceived by users as more persuasive, and the adoption of central interfaces increases the overall service evaluation.

As robots and robotic things become to have more agency, IoRT which consists of robots and robotic things can be considered as a social organization. Accordingly, social organization structure of IoRT could affect users' behavior and perception of IoRT. In this study, in order to examine the effect of social organization structure on people's acceptance of IoRT, we conducted a 2 (social organization structure: flat vs. hierarchical) within-participants experiment (N=30). In the experiment, a participant was asked to take part in cooking task with the aid of a robot, a robotic measuring cup, and a robotic mixer. We executed a post-experimental survey and counted the duration of participants' following the instruction given by the platform. People gave higher scores of trustworthiness and purchase intention to the platform with flat organization structure than that with hierarchical one. On the contrary, participants were more compliant with the hierarchical IoRT service platform than a flat one. Implications for the theory and design of IoRT are discussed.

With the development of robotic technology, various types of robotic products have been developed, ranging from robotic objects that support small daily necessities (such as robotic umbrellas and frames) to robotic furniture (such as robotic chairs and drawers). Owing to consumerism, the consumer marketplace now overflows with surplus products, and storage and organizational products have been increasingly necessary. With this social stream, we focused on designing robotic storage furniture in this study. To create such a type of furniture that is acceptable by consumers, a qualitative user study was conducted with 16 subjects by analyzing the users' behaviors when using ordinary storage furniture. From this user study, we found strong user needs in terms of organizing and finding objects. Thus, we developed two types of robotic cabinets: the first one to assist users' organizing behavior and the second one to assist their finding behavior. To examine the effectiveness of the developed prototypes, we executed a 3 (behavior the robotic cabinet assists: baseline vs. organizing vs. finding) within-participants experiment. The result shows that significant effects vary depending on the type of robotic cabinet.

Robotic things and social robots have been introduced into home, and they are expected to change the relationships between humans. Our study examines whether the introduction of robotic things or social robots, and the way that they are organized, can change the social relationship between family members. To observe this phenomenon, we designed a living lab experiment that simulated a home environment and recruited two families to participate. Families were asked to conduct home activities within two different types of Internet of Robotic Things(IoRT):1)internet of only robotic things(IoRT without mediator condition), and 2)internet of robotic things mediated by a social robot(IoRT with mediator condition). We recorded the interactions between the family members and the robotic things during the experiments and coded them into a dataset for social network analysis. The results revealed relationship differences between the two conditions. The introduction of IoRT without mediator motivated younger generation family members to share the burden of caring for other members, which was previously the duty of the mothers. However, this made the interaction network inefficient to do indirect interaction. On the contrary, introducing IoRT with mediator did not significantly change family relationships at the actor-level, and the mothers remained in charge of caring for other family members. However, IoRT with mediator made indirect interactions within the network more efficient. Furthermore, the role of the social robot mediator overlapped with that of the mothers. This shows that a social robot mediator can help the mothers care for other members of the family by operating and managing robotic things. Additionally, we discussed the implications for developing the IoRT for home.

Service robots with social intelligence are starting to be integrated into our everyday lives. The robots are intended to help improve aspects of quality of life as well as improve efficiency. We are organizing an exciting workshop at HRI 2019 that is oriented towards sharing the ideas amongst participants with diverse backgrounds ranging from Human-Robot Interaction design, social intelligence, decision making, social psychology and aspects and robotic social skills. The purpose of this workshop is to explore how social robots can interact with humans socially and facilitate the integration of social robots into our daily lives. This workshop focuses on three social aspects of human-robot interaction: (1) technical implementation of social robots and products, (2) form, function and behavior, and (3) human behavior and expectations as a means to understand the social aspects of interacting with these robots and products.

Human has an ability to adjust utterance depending on the state of interlocutor. In this study, we explore the verbal behaviors of human through interaction with two smart speakers that have different level of task competence. We analyzed (1) linguistic behaviors appeared in user's utterance, (2) length of the uttered speech, and (3) required pragmatics skills to understand the user's intent. As a result, there were no significant difference in linguistic behaviors and length of the speech while user interacts with speakers with different task competence. In addition, various pragmatics elements were equally utilized and especially, implied intentions were frequently observed in user's short utterance even under simple interaction scenarios.

In this paper, we introduce an ontology-based home care service robot that can provide personalized care for people who are in the early stage of dementia. The hardware and software framework encompassed in the proposed service robot was developed to carry out care services in their daily life at home. Specifically, to generate adaptive task plans in diverse caring situation, context reasoner and ontological model of dementia are included. Ontology includes various concepts that are related with the knowledge of caring dementia patient: dementia, dementia symptom, environment of around patient, and situation during patient's daily life. To evaluate if the proposed service robot could provide appropriate care service or not, experimental care scenario for helping a person with dementia take medicine was tried in the lab environment. Although tasks of the robot required for the experiment are rather simple, we have demonstrated that the robot could provide a personalized service that may be beneficial to dementia patient, family members and caregivers. In the future, we will add more care knowledge in the ontology and further develop a variety of care services. Additionally, we are going to test the care service robot in a real environment with actual dementia patient.

In this study, we propose a probabilistic approach to overcome time delay for a mobile robot. The robot receives sporadic user input when using brain-computer interface, and related studies have been considered about various strategies to control the robot as intended. However, the most of experiments heavily depended on the driving environment because of the low information transfer rate. To solve this problem, we propose a probabilistic local goal determination method. Our approach accumulates user input sequence, and specifies it as a feasible path. Simulations in the room and on the track are provided. As a result, we show that the proposed method is robust to long time delay, and achieves the mission with fewer inputs.

Since the appearance of humanoid robots is similar with real human, a method is proposed to discriminate these robots from real human in a perception sensor network (PSN) which uses multiple Kinects and PTZ cameras to provide the location and name of tracked human to these robots. To this end, the global map which is acquired by fusing and calibrating one local map built by a robot and the other local map by the PSN system is used to provide the location of the robot. Since the PSN updates periodically the location of the robot by subscribing the ROS (Robot Operating System) topics, the robot which is wrongly detected as human but yields the closest distance to the location can be correctly identified as a robot in the system. Therefore, the misunderstood cases when robots are recognized as human in the PSN system are removed effectively. The experimental results demonstrate the outperforming of the proposed method in various scenarios.

Social skills training of children with autism spectrum disorders (ASD) is not an easy problem enough to typically spend a lot of time and repetitive efforts. As means of assistance, we propose and examine the feasibility of a robotic behavioral intervention system to facilitate social development of the children with ASD and relieve operational burden during training process. To this end, based on well-known behavioral treatment protocols, the robot system incorporates robotic stimulation design, recognition modules for human activities, and reinforcement procedure in the interaction scheme design. Using these configuration, it repeatedly perform a role of training eye contact and reading emotions targeted at preschoolers with a high functioning level in the planned training process. Through showing an advantage in a comparative analysis with control group taught by humans, we verified that the proposed system can contribute to evoke positive response of children with ASD and provide labor-saving effect on the clinical environment.

This paper presents the integration of audio-visual perception components for human robot interaction in the Robot Operating System (ROS). Visual-based nodes consist of skeleton tracking and gesture recognition using a depth camera, and face recognition using an RGB camera. Auditory perception is based on sound source localization using a microphone array. We present an integration framework of these nodes using a top-down hierarchical messaging protocol. On the top of the integration, a message carries information about the number of persons and their corresponding states (who, what, where), which are updated from many low-level perception nodes. The top message is passed to a planning node to make a reaction of the robot, according to the perception about surrounding people. This paper demonstrates human-robot interaction in multi-persons scenario where robot pays its attention to the speaking or waving hand persons. Moreover, this modularization architecture enables reusing modules for other applications. To validate this approach, two sound source localization algorithms are evaluated in real-time where ground-truth localization is provided by the face recognition module.

In this paper, we introduce a simulation approach for emulating a real-world human recognition system called a Perception Sensor Network (PSN). The proposed PSN system has fusion components for automatic localization and facial recognition that uses multiple Kinect sensors and pan-tilt-zoom (PTZ) cameras. We verified that the generic vision schemes utilized in human detection and facial recognition algorithms were interoperable in a scenario that consisted of virtual avatar humans. In experiments, we tested a perception scenario in which multiple humans in a 3D simulation space are automatically recognized in both location and identification with real-world system parameters.

In this paper, we propose with a robot-assisted behavioral intervention system to easily improve children's social capability. In particular, the system for children with autism spectrum disorders (ASD) is basically achieved through the discrete trial teaching (DTT) protocol with three task modes of therapy, encouragement, and pause in social training scenarios. In child-robot interaction architecture, the robot firstly offers therapeutic training elements of mutual greeting and interplay game, and evaluates the level of children's reactivity by recognition modules for frontal face and touch features. Thence, the system in the decision-making process determines the task mode to perform subsequent action by grasping behavioral state of the children, and then it copes with individual response appropriately by using the robotic stimuli with the combination of kinesic acts and displayable contents. From the experiments of clinical trials with children with non-ASD and ASD in each robotic stimulus, the system showed the potential to increase their attention and activeness for social training, and we believe that the proposed system has some positive effect on developing children's social skills.

Recently, human-robot interaction, or human-robot communication, has been extended to practical application in real environments. One of the important factors in human-robot interaction is sound source localization. Moreover, communication is more complex in sound-corrupted environments than in controlled environments. In the present study, the a priori SNR Wiener Scalart algorithm for noise reduction is utilized and integrated in a sound source localization system. The system is evaluated in the sound-corrupted context of a vacuum cleaning robot performing cleaning operations.

In this work, we introduce a 3D simulation framework for multi-human detection using multi-sensor. We build a 3D indoor space by modeling the real dimensions of a classroom using the 3D creation tool Blender. We also create human animations like sitting down and sleeping using its game engine. The multi-sensor frame consists of two RGB and two depth cameras simulated with the aid of the simulator MORSE. In addition, we examine the feasibility of simulation by testing the baseline vision method of face and human detection on a textured human animation.

The proposed algorithm in this paper is capable of classifying not only unusual speech when people get anger, surprised, or excited but also unusual noise such as clashing, hitting, or clapping in real-time without depending on particular speaker voices or utterances. Also, it does not require a prior learning process to construct acoustic models. This algorithm, therefore, allows a surveillance camera system to effectively monitor quarrel or violent situations regardless of object shields and light conditions. To realize our approach, we analyze the variance and change of spectral densities and pitches when unusual speech and noise occur. We then propose new methods (SEBNI, USDF, and UNDF) to classify unusual sounds in real-time. Moreover, to improve performance, we apply a noise suppression system based on MMSE-STSA and a statistic model-based VAD to our algorithm in order to extract reliable voice features and segment only voice-related periods in noisy environments. We confirm that our proposed method achieves an 87% accuracy performance for classifying unusual speech.

This paper presents a general framework for localization of multiple sound sources in Cartesian coordinate with perception sensor network (PSN). PSN consists of Kinect sensor that has a color camera, a depth camera, and an internal microphone array and our experimental pan-tilt-zoom camera with attached microphone array. Sound localization with PSN is based on three-stage analysis. Short-time narrowband directional sound localization based on Phase difference of arrival (PDOA) is obtained in every time-frequency points, utilizing the sparseness assumption of audio mixtures. Multi-sensor directional localizations are transformed to Cartesian coordinate by a simple triangulation. The results are accumulated in all frequency bins for a block of frames and then clustered to obtain mid-term broadband localization. Furthermore, the framework is able to integrate any Bayesian filtering algorithms for long-term localization. Simulation results with four arrays (each has four microphone) show that the proposed framework successfully localize three simultaneous sources where the distance among sources is about one meter.

In realistic environments, mismatches between the calculated angle-TDOA map with its real exact values are the major reason of performance degradation in sound localization. Usually, those mismatches come from some certain configuration errors or deviations caused by the change of environments. To reduce those mismatches, in this paper we proposed an angle-TDOA map adaptation method which can achieve the robust sound localization in various robot platforms (i.e., various types of microphone array configuration). Especially, the proposed method can be easily applied to the sound localization system by using only several sound sources which generated from some known directions. As a result, the proposed method not only showed a good localization performance, but also saved processing time.

Recently, many researchers have studied about adaptation methods of various environments since intelligent service robots generally have to interact with human beings in any environments. In this paper, an adaptation method of angle-TDOA map is proposed which can be applied to a sound localization system when a certain configuration error or deviation is brought up by the change of environments. In addition, two cases of situation are defined along with two suggested methods respectively to solve the problem. The first method is building an angle-TDOA map with a test signal generated at know positions when a microphone array is randomly placed. The second one is an on-line adaptation method for the real-time sound localization. In this method, we will use a vision system to find out the real direction of sound sources and the angle-TDOA map is learned from the difference between map and actual value of TDOAs. We use a face detection module to update an angle-TDOA map and assume the detected person is generating sounds. Finally experiments show the effect of the adaptation in both cases.

Sound source localization is an important part of intelligent robot auditory system. It makes a robot to respond naturally to human user's call. In the ordinary situations, there always exist multiple sound sources including user's call. Since localized outputs from each source are mixed in distribution, clustering is an important issue in the multi-source sound localization. In this work, we propose a new k-means clustering algorithm for unknown number of clusters, which is the competitive k-means. We compared its performance to the adaptive k-means++ algorithm and verified its effectiveness. Finally, we applied it to our sound source localization for multi-source sound localization and achieved satisfying results.

This paper describes a DSP integration of sound source localization (SSL) and multi-channel Wiener filter (MWF). To develop a robot audition system, we integrated SSL module and MWF module into a DSP system. SSL is a module to perceive the direction of a human user's call. It measures time delay of arrival among microphones and estimates the direction of sound source. Also, it post-processes the resulted estimations of direction by histogram to perceive the direction robustly under noisy environment. MWF is a module to reduce background noises from raw voice signal to enhance the performance of robot's speech recognition. It gathers information of background noises during noise-period and then reduces noises during voice-period. This SSL-MWF combination system will be a cheap, high-performing and convenient solution for robot audition.

Recently, many researchers have carried out works on audio-video integration. It is worth exploring because service robots are supposed to interact with human beings using both visual and auditory sensors. In this paper, we propose an audio-video method for sound source localization in reverberant environment. Using visual information from a vision camera, we could train our audio localizer to distinguish a real source from fake sources and improved the performance of audio localizer in reverberant environment.

Many kinds of sound source localization systems have been developed for detecting a direction of sound source. They are commonly using time delay of arrival (TDOA) or interaural time difference (ITD) algorithm for sound source localization where, especially, the ITD is the difference in arrival time of a sound between two ears. It is largely changed depending on frequency components of sound even though the sound source is located in the same place. In this paper we propose a binaural sound localization system using sparse coding based ITD (S-ITD) and self-organizing map (SOM). The sparse coding is used for decomposing given sounds into three components: time, frequency and magnitude. Moreover we estimate the azimuth angle through the SOM. This localization system is installed in our robot that has two ears, head and body. We use PeopleBot as a body of the robot.

Many kinds of sound source localization systems have been developed for detecting a direction of sound source. They are commonly using time delay of arrival (TDOA) or interaural time difference (ITD) algorithm for sound source localization where, especially, the ITD is the difference in arrival time of a sound between two ears. It is largely changed depending on frequency components of sound even though the sound source is located in the same place. In this paper we propose a binaural sound localization system using sparse coding based ITD (SITD) and self-organizing map (SOM). The sparse coding is used for decomposing given sounds into three components: time, frequency and magnitude. Moreover we estimate the azimuth angle through the SOM. This localization system is installed in our robot that has two ears, head and body. We use PeopleBot as a body of the robot.

In this paper, data association method using audio and video data together to localize targets in a cluttered environment and detect who is speaking to the robot is presented. Particle filter is applied to find an optimal association between target and measurements efficiently. State variables are composed of position and speaking status. To update the speaking state, we first evaluate the upcoming sound signal based on the cross-correlation between microphones and then calculate the likelihood for audio information. Visual measurement is applied to find an optimal association between target and visual observations. The number of targets that robot should interact is updated based on the vision and audio information. Several experimental data were collected beforehand and simulated on computer to verify the performance of proposed data association method for speaker selection problem in a cluttered environment.

Several kinds of indoor localization systems have been developed for estimating object’s position lately. They are classified according to the environment, the purpose of application, the media type and so on. In this paper, we propose an improved beacon system, having the ultrasonic media type, which can be used for indoor localization. Moreover, it can estimate the direction of a mobile object by using a digital compass. However it is very hard for the indoor localization system using ultrasonic sensors to achieve much highly precise localization for mobile objects since the ultrasonic sensor is highly sensitive to noises and external shocks. In order to overcome this problem, we adopt a bandpass-filter to protect ultrasonic sensors form external shocks or another noisy signals. In addition, the multi-modulation of ultrasonic sensors - using ultrasonic sensors with different frequency - improves the sampling rate of this system. And we present the application of UKF (Unscented kalman Filter) to improve the localization performance of our indoor localization system. Consequently, the positioning accuracy and sampling rate of this system has been increased through bandpass-filter and multi-modulation.

In this paper, a single speaker tracking mechanism with particle filter is presented for a reliable interaction with robot. To have a smooth tracking result, we use not only audio data but also video data. For sound observation, we use Time Delay Of Arrival (TDOA) between two microphones, and in order to expect high reliability in the likelihood of localization, we apply variable sensitivity values according to the angle of arrival for a triangle arrangement of microphones to calculate the likelihood for audio observation. For video observation, we use face detection routine of OpenCV which is widely available through the Internet. Together with audio and video data, we found out that more reliable source tracking is possible if we apply data fusion mechanism at every possible environment on a robot platform than single information.

In this paper, we proposed a probabilistic localization of service robot using laser scanner and indoor GPS system. This scheme is a sample based algorithm as an application of Monte Carlo localization. Samples are scattered by referring to odometry data and position data of indoor GPS system. And those are evaluated by map-matching method using laser sensor and distance error matching using indoor GPS system. We are able to get more robust localization results by using these processes, even though the floor condition is deteriorated and there are many or huge obstacles And it is possible that remove errors which are caused when the robot is blocked by unexpected obstacles for a long time. Experimental results demonstrate the evaluation of the robustness of our algorithm fusing range image sensor and indoor GPS system data.

In this paper, we not only developed a reliable sound localization system including VAD (voice activity detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we proposed a way to integrate these systems in the human-robot interaction to compensate the errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from the undesired directions effectively. For the purpose of verifying our system's performances, we installed the proposed audition and vision system to the prototype robot, called IROBAA (Intelligent ROBot for Active Audition), and showed how to integrate an audio-visual system

In this paper, we have developed not only a probabilistic sound localization system including VAD (voice activity detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we have proposed a way to integrate these systems to compensate the errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from the undesired directions effectively. For the purpose of verifying our system's performances, we have installed the proposed audition and vision system to the prototype robot, called IROBAA (intelligent robot for active audition), and showed how to integrate an audio-visual system

In this paper, we propose reliable detection of sound's direction for human robot interaction. Compared with previous researches, this system comprises simpler algorithm and a proposed nonlinear amplifier which has advantages to increase a detectible distance of sound signal in spite of simple circuits. Moreover, we propose the new performance index using cross-correlation to make reliable detection of sound's direction. Also, since this system include a function to detect a period of voice signals, the robot can know when to start finding sound's direction and performing speech recognition automatically. In order to verify our system's performances, we install the proposed audition system to the prototype robot, called IROBAA (intelligent robot for active audition), and describe the experimental results.

We derive a near-time-optimal trajectory for wheeled mobile robots (WMRs) satisfying the following: 1) initial and final postures/velocities; and 2) battery voltage and armature current constraints, under assumptions of simplified dynamics and constant translational/rotational velocity sections. We use a simplified dynamic model for WMRs, neglecting inductances of motor armatures and divide our trajectory generation algorithm for cornering motion into three sections. We specify a path-deviation requirement for obstacle avoidance. Transforming dynamics into uncorrelated form with regard to translational and rotational velocities, we make extreme control possible. By separating the rotational section with translational sections and determining the velocity scale factor a near-time-optimal trajectory can be obtained. Simulation results along with inverse control of path-following are given to validate the generated trajectory.

In this paper, we describe the development of our BEST nano-robot soccer team composed of 5 robots, a vision system, and communication modules. Each nano-robot is designed with an all-in-one type microcontroller and two DC motors fixed in parallel due to its small size, 4(cm)/spl times/4(cm)/spl times/4.5(cm). A vision system capable of 30 recognitions/sec provides each robot's position feedback via RF communication. A three level hierarchical software control scheme is adopted: role allocation, target generation, and visual servoing. Dynamic role allocation is done in the highest layer (the role allocation layer) regarding the context and role importance. In the target generation layer, target positions of robots are generated based on 6 roles, 2-2-1 formation, and libero system. In the visual servoing layer, path planning is achieved using a newly proposed line-circle algorithm with linear and circular paths as our basic strategy. Normally, 2-2-1 formation is adopted. When the both attackers are blocked, one of defenders is assigned to continue the attack as libero (temporary 3-1-1 formation).