This paper presents a process to build a classifier in a data-driven way for recognizing engagement of children in a robot-based math quiz game. The process consists of collecting video recordings from HRI experiments; annotating the social signals and engagement states via video analysis; extracting feature vectors from the annotations and training classifiers. We conducted an experiment with 7 participants of 10 -- 11 years of age using an android robot EveR-4. With three coders annotating the video recordings and extracting features by snapshot model with 1-second time window, we achieved 84.83% recall performance.

In collective intelligent robots, communication capability is considered as a key feature to decide how to design the collaboration policy. A cooperative routing robot system is suggested to maintain the communication capability in between human and a robot and an algorithm to establish a seamless global communication is proposed. Generating the radio signal map is suggested to estimate an area where communication is possible. If the robots only move in the area, they can prevent the disconnection of communication in different radio environments. The proposed algorithm is composed of two steps. At the first step, Extended Kalman Filter is designed to estimate state of the radio environments. Then, the communication link a radio signal map is generated using k-NN(Nearest Neighbors)algorithm. Experiments with routing robots and sensing system verify our proposed system and algorithm are valid in real environment.

In collective intelligent robots, communication capability is considered as a key feature to decide how to design the collaboration policy. A cooperative routing robot system is suggested to maintain the communication capability in between human and robots and an algorithm to establish a seamless global communication is proposed. In this paper, implementation of seamless communication between human and the robots using cooperative routing robots is shown in real environment.

Providing robots with social intelligence is critical for making entertaining and sustainable human-robot interactions. The first step to get good social intelligence is to appropriately understand the meaning of social signals emitted by interactors. In this paper, we introduce a preliminary study on identifying principal social signals in interpreting participant's engagement and confirmation intention in 1:1 interactions. We annotated 6 video recordings of private teacher-student interactions with 20 social signals and their interpretations, and built pattern data sets with different subsets of social signals. C4.5 based decision trees were generated using the pattern data sets and the recall rates were compared. Also attribute selection was performed to find principal social signals. The results showed that verbal signal was the most principal for determining engagement, and the combination of gaze and verbal signal for confirmation intention.

Robotic person-following is an essential component for natural human robot interaction. To follow a person, the robot should track the target person robustly and in real time. Object tracking algorithms in the computer vision field typically require abundant features and heavy computing power, and thus cannot be directly applied to person-following robots due to the problems arising in practical robotic environments. This paper proposes a robotic person-tracking algorithm based on modified multiple instance learning. In order to resolve the problems raised by the rearward view of the target person, the tracker is modified to be guided by color histogram back-projection. Additionally, the search area model is modified from circle to ellipse and the number of features is reduced so that the tracker should adapt the robotic environment in real-time. The algorithm is validated through system integration and experiments.

The vision of an ecology of physically embedded intelligent systems, or PEIS-Ecology, combines insights from the fields of autonomous robotics and ambient intelligence to provide a new approach to building robotic systems in the service of people. In this paper, we present this vision, and we report the results of a four-year collaborative research project between Sweden and Korea aimed at the concrete realization of this vision.We focus in particular on three results: a robotic middleware able to cope with highly heterogeneous systems; a technique for autonomous self-configuration and reconfiguration; and a study of the problem of sharing information of both physical and digital nature.

Abstract — Reinforcement learning (RL) has been used as a learning mechanism for a mobile robot to learn state-action relations without a priori knowledge of working environment. However, most RL methods usually suffer from slow conver-gence to learn optimum state-action sequence. In this paper, it is intended to improve a learning speed by compounding an existing Q-learning method with a shortest path finding algorithm. To integrate the shortest path algorithm with Q-learning method, a stochastic state-transition model is used to store a previous observed state, a previous action and a current state. Whenever a robot reaches a goal, a Stochastic Shortest Path(SSP) will be found from the stochastic state-transition model. State-action pairs on the SSP will be counted as more significant in the action selection. Using this learning method, the learning speed will be boosted when compared with classical RL methods. To show the validity of our proposed learning technology, several simulations and experimental results will be illustrated. I.

The robot space is a ubiquitous environment in which the networked robot plays the role of main mediator that accepts the sensor data from the environment, interprets the data to understand the situation, and decides the actions to perform. Semantic robot space provides the high-level data model of the robot space that enables context-aware service execution. We designed the semantic robot space based on the semantic Web technology, which provided crucial advantages in architecting dynamic reconfiguration mechanism. The standard nature of the technology leverages the interaction among stake-holders in building robot spaces, which makes it possible to establish public registry-based construction of the semantic robot space. We conclude that semantic robot space based on the semantic Web not only embellishes the intelligence of robots but also reduces the cost of installing and maintaining robot spaces.

This paper presents a text-independent speaker identification system using multiple microphones on the robot, which is intended for use in human-robot interaction. For the purpose of the best possible classification rate in speaker identification, the individual identification results obtained from multiple microphones on the robot are combined by various combination schemes. The performance improvement has been achieved. Our ultimate goal is to enhance human-robot interaction by improving the recognition performance of speaker identification with multiple microphones on the robot side in adverse distant-talking environments. Various combination schemes obtained high classification accuracy in the ubiquitous robot companion (URC) environment, where the robot is connected to a server through extremely high broadband penetration rate. In conclusion, our speaker identification system can provide human-robot interaction with a reliable basic interface with high classification accuracy.

URC (ubiquitous robotic companion) is a new concept for a network-based robotic system, in which a different kinds of robots are connected to a server system to provide various robotic services. In this paper, we introduce the client/server framework to provide context-aware services to network-based robots. We call the framework as CAMUS (context-aware middleware for URC systems). The CAMUS is mainly composed of three parts: service agent managers, CAMUS server and planet. The service agent manager is the robot-side framework to send robot's sensing data to the CAMUS server and receive control commands from the CAMUS server. The CAMUS server is the server-side framework to manage information from service agent managers, generate and disseminate appropriate events according to the context changes, and finally execute server-based robot tasks. The planet is the communication framework between service agent managers and the CAMUS server. This paper also discusses the implementation of the URC field test applications.

We introduce the concept of ecology of physically embedded intelligent systems, or PEIS-ecology. This is a network of heterogeneous robotic devices (PEIS) pervasively embedded in the environment. A PEIS can be as simple as a toaster and as complex as a humanoid robot. PEIS can exchange information at different levels of abstraction, and share both physical and virtual functionalities to perform complex tasks. By putting together insights from the fields of autonomous robotics and of ambient intelligence, the PEIS-ecology approach explores a new road to building assistive, personal, and service robots. In this paper, we discuss this concept, describe a first realization of it, and show an implemented use-case scenario

The localization is one of the most important issues for mobile robot. We describe a novel localization sensor suite for the development of a wireless location sensing network. The sensor suite comprises wirelessly controlled infrared landmarks and an image sensor which detects the pixel positions of infrared sources. The proposed sensor suite can operate irrespective of the illumination condition in the indoor environment. We describe the operating principles of the developed sensor suite and report the performance for mobile robot localization and navigation. The advantage of the developed sensor suite lies in its robustness and low cost to obtain localization information as well as simplicity of deployment to build a robot location sensing network. Experimental results show that the developed sensor suite outperforms the state-of-the-art localization sensor

This video is concerned with a scenario for developing speech-based Human-Robot Interaction (sHRI) components for Ubiquitous Robot Companion (URC) Intelligent Service Robots as one of the next-generation growth engine industries in Korea. Here the URC means that it will provide the necessary services at any time and place to meet the user's requirements. Thus, it combines the network function with the current concept of a robot in order to enhance mobility and human interface. The main characteristics of this video are to combine of text-independent speaker recognition and Korean-based spontaneous speech recognition from two speakers in scenario. Especially each speaker communicates with service robot through spontaneous speech recognition with continuous words to provide useful information such as daily life schedule and TV program suitable to the speaker recognized. On the basis of these components, consumers will be able to utilize various speech-based services of the robot.

In this video, we present a scenario for developing vision-based Human-Robot Interaction (vHRI) components for Ubiquitous Robot Companion (URC) Intelligent Service Robots, which exploit strong Information Technology (IT) infrastructure. Here the URC means that it will provide the necessary services at any time and place to meet the user's requirements. Thus, it combines the network function with the current concept of a robot in order to enhance mobility and human interface. The vHRI components used in this video consist of caller identification, face recognition/verification, and gesture recognition. Firstly we perform face verification and recognition for the imposter and a member of family from face images of two users, respectively. After that, we use gesture recognition to provide the selection of TV channels under noise or long-distance environments. Finally, we demonstrate the usefulness through vision-based scenario shown from video. The robot platform used in this video is WEVER, which is a URC intelligent service robot developed at Intelligent Robot Research Division in Electronics and Telecommunication Research Institute (ETRI).

In this paper, an algorithm for human head detection over a distance exceeding 2.5 m between a camera and an object is described. This algorithm is used for the control of a robot, which has the additional limits of a moving camera, moving objects, various face orientations, and unfixed illuminations. With these circumstances, only the assumption that human head and body contours have an omega (Omega) shape is made. In order to separate the background from these omega shapes, the three basic features of gray, color and edge are used, and combined. The skin color cue is very useful when the image stream is frontal and has large face regions, and additionally has no background objects similar to the skin color. The gray cue is also important when captured faces have a lower gray level than background objects. The edge cue is helpful when captured background objects have similar gray levels and colors to those of a head, but can be discriminated by edges. Since these three methods have roughly orthogonal failure results, they serve to complement each other. The next step is a splitting method between the head and body region using the proposed method. The final step is an ellipse fitting and a head region verification algorithm. The results of this algorithm provide robustness for head rotation, illumination changing, and variable head sizes. Furthermore, it is possible to carry out real time processing

This paper proposes a novel real-time robust hand tracking algorithm, integrating multi-cues, and a limb's degree of freedom. For this purpose, we construct a limb model and maintain the model obtained from KLT-AR methods with respect to second-order auto-regression model and Kanade-Lucas-Tomasi (KLT) features, respectively. Furthermore, this method provides directivity of a target, enabling us to predict the next motion. Thus, we can develop a method of hand tracking for gesture and behavior recognition techniques frequently used in conjunction with human-robot interaction (HRI) components. The experimental results show that the proposed method yields a good performance in the intelligent service robots, so called Wever developed in ETRI

The ability to recognize people is positively necessary for interaction between robots and people. In this paper, we introduce intelligent security system that is able to recognize, track and remember user's face. System uses moving camera that has pan/tilt and zoom function like robot's eyes. In order to construct system, we propose the face identification method based on combining multiple features and applying multiple matching algorithms. Also, we address the benefits of multiple instances and show that identification ability can be improved by using proper combination strategy

An approach to identifying a caller or callers by a service robot is presented for a natural interaction with people in a home/office environment. The problem addressed specifically in this paper is how to successfully identify a caller in a cluttered environment with large uncertainties involved in the sensed audio-visual cues. The proposed approach is based on a proposition that the dependability of perceptual recognition may come unlikely from "the effort to make individual sensing perfect", but likely from "the effort to self-generate perceptual behaviors of integrating individual sensing that lead to mission accomplishment, no matter how imperfect and uncertain individual sensing may be". We implement the above proposition in terms of a novel robotic architecture, referred to here as "cognitive robotic engine (CRE)." CRE implemented for the case of a robot identifying a caller in a crowded and noisy environment, including its experimental results, are shown

A localization algorithm for topological maps with dynamics is proposed in this paper. Especially, this algorithm considers a localization when some nodes are deleted by door closing and has two main features. First, lots of edge information are used in efficient way. Second, the algorithm calculates probability of a node to be the current location in a systematic and general way. Experiments in a topological map with dynamics are conducted to show the performance of the proposed algorithm.

In recent years, motivated by the emergence of ubiquitous computing technology, a new class of networked robots - ubiquitous robots - has been introduced. The URC (ubiquitous robotic companion) is our conceptual vision of ubiquitous robot which provides us with the services we need, anytime and anywhere. To realize the vision of URC, it is one of the most important requirements for the robotic systems to support seamlessness of services even though service environments changes. Specifically, it is needed for robotic systems to be interoperable with the sensors and devices in current service environments automatically, rather than statically preprogrammed for them. In this paper, we present SURF (service-oriented ubiquitous robotic framework) which enables automated integration of networked robots into ubiquitous computing environments based on the semantic Web services technology. In SURF approach, we implement interfaces for robots, networked sensors and devices as Web services. And we describe knowledge about them in OWL-S the semantic Web services ontology and register the knowledge to KB, so that a SURF agent can automatically discover required knowledge and compose a feasible service plan for the service environments. And then the agent controls robots, sensors and devices through SOAP the Web services execution protocol according to the service plan.

The user authentication problem is discussed in the context of robot environments. Although authentication is indispensable for useful robot services, the traditional approaches that have been applied in strictly regulated environments cannot be applied as they are when both the robot and humans are freely moving around. In this paper, a new authentication system redesigned to be adapted to robot environments is described. In the proposed system, semi-biometrics and user tracking are incorporated to cope with the limited applicability of traditional authentication techniques. Semi-biometrics indicates a set of features useful for discriminating persons, but only in interested group of persons and in interested frame of time. As a representative semi-biometric feature, body height and color characteristics of the clothes are investigated. Especially, a novel method to measure body height with single camera is proposed. Also by incorporating tracking functionality, the system could maintain the user status information continuously, which is useful not only for recognition but also for finding a designated person.

The two most important abilities for a robot to survive in a given environment are selecting and learning the most appropriate actions in a given situation. Historically, they have also been the biggest problems in robotics. To help solve this problem, we propose a two-layered action selection mechanism (ASM) which designates an action pattern layer and a reactive behavior plan layer. In the reactive behavior plan layer, a task is selected by comparing behavior motivation values that, in an animal, correspond to external stimuli as well as internal states due to hormones. After a task is selected, its corresponding reactive behavior plan is executed as a set of sequential dynamic behavior motivations (DBMs), each of which is associated with an action pattern. In the action pattern layer, each action pattern can be functionally decomposed into primitive motor actions. Shortest path-based Q-learning (SPQL) is incorporated into both the reactive behavior plan and action pattern layers. In the reactive behavior plan layer, relationships between perceptions and action patterns are learned to satisfy a given motivation, as well as the relative priorities of these relationships. In the action pattern layer, the relations between sensory states and primitive motor actions can be learned. To establish the validity of our proposed ASM, experiments with our real designed robot was illustrated together with simulations.

Presents a situation-driven reconfigurable architecture for a schema-based robot controller which is operating in partially known indoor environments. Instead of the purely repulsive forces from the obstacles, we employ the linear combination of the repulsive and the detouring forces around the obstacle, so that the obstacles generate virtual forces pushing the robot toward the goal rather than just blocking it. By reconfiguring the detouring direction and the activation level of each schema based on the geometric and/or topological information provided by the variable resolution maps, the robot can navigate through the environments in very natural way. Simulation results show that our approach can remarkably improve the intrinsic defects of traditional potential field-bused schemas.

Underwater robotic vehicles (URV) have been employed for mobility, and robot manipulators attached to the underwater vehicle play the role of manipulation. URV motions are determined by inherent dynamic couplings between active and passive joints. Furthermore, the control problem becomes complex since there should be considered many hydrodynamic terms as well as intrinsic model uncertainties. To cope with these difficulties, we propose a disturbance observer-based robust control algorithm for underwater manipulators with passive joints. The proposed control algorithm is able to treat an underactuated system as a pseudo-active system in which passive joints are eliminated. Also, to realize a robust control method, a nonlinear feedback disturbance observer is applied to each active joint. A four-jointed underwater robotic system with one passive joint is considered as an illustrative example. Through simulation, it is shown that the proposed control algorithm has good position tracking performances even in the presence of several external disturbances and model uncertainties.

This paper deals with target tracking control of unicycle type mobile robots. Target tracking function is essential for autonomous robots such as guide robots, security guard robots, etc. In the field of mobile robot control, many control schemes for posture stabilization and trajectory tracking problem have been proposed. Target tracking control, however, cannot be achieved using these kinds of control laws. Therefore, a new global asymptotic stable controller for this problem is designed using the backstepping method. The stability of the system is proved using the Lyapunov function. Various simulation results validate the performance and theoretical analysis.

A compliance control strategy for robot manipulators is proposed by employing a self-adjusting stiffness function. To be specific, each entry of the diagonal stiffness matrix corresponding to task coordinate in Cartesian space is adaptively adjusted during contact along the corresponding axis based on the contact force with its environment. It can also be used for both unconstrained and constrained motions without any switching mechanism which often causes undesirable instability and/or vibrational motion of the end-effector. The experimental results show the effectiveness of the proposed method by employing a two-link direct drive manipulator interacting with an unknown environment.

Proposes a hybrid control architecture which combines the key features of the two well-known robot control architectures; hierarchical and behavioral-based. The overall control architecture consists of three layers, i.e. the high-level planner, the middle plan executor, and the lower level monitor and behavior-based controller. In the planned task, only one behavior module is chosen by the logical coordinator in the plan executor according to the way point bin. In the exceptional situation, the central controller in the plan executor issues an additional control command to reach the planned way point. Several simulations and experiments with an autonomous mobile robot show that the proposed architecture enables the robot controller to achieve the multiple sequential goals even in a dynamic and uncertain environment.

Focusing on the manipulation tasks to be executed by humanoid robots, principal requirements which are to be satisfied by hardware/software of the control system are considered. In order to meet the requirements, a novel type of hardware structure and software architecture is proposed. Since the target humanoid robot consists of multiple subsystems such as a central controller for brain, a vision controller for eye, and five motion sub-controllers for two arms, two hands, one spine, the on-board hardware control system is designed to have a distributed control structure connected by pseudo real-time Ethernet interfaces. A goal-achieving software architecture is also proposed which meets the requirements of semi-autonomy, reactivity, expandability, and object-orientedness. Specifically, in order to achieve reactivity, a coordination method is proposed to configure three kinds of executive modules, primitive module, flow-control module, and goal module, which have multiple exit states. The control architecture proposed has been implemented for performing toy-block assembly tasks on a humanoid robot as well as on the graphic simulator.

An object-oriented approach to behavior-based control architecture for autonomous mobile robots as presented. Instead of traditional implementations of the architecture with proprietary programming languages which resulted in slow execution speed and poor hardware compatibility, the C++ language is used and an optimal scheduling algorithm is proposed for fast and guaranteed response time. Each behavior module is compiled separately and loaded at run-time using dynamic binding to improve modularity and expandability. A vector valued representation of output signals is also proposed to arbitrate conflicting signals properly without losing any information. The proposed method is implemented and applied to control a laboratory developed autonomous mobile robot and tested in real-time.

A sequence replanning/control method is proposed to overcome geometric uncertainties of the slave environment in our a priori knowledge of a discrete-event-based tele-task model. A class of controlled Petri net (CPN) is adopted as a modeling tool of tele-robotic part-mating process and the discrete-event control policy of the process is formulated on the basis of the CPN model. In the case when an unexpected state transition occurs due to the uncertainty of the slave environment, recovery paths and their costs are extracted from the a priori knowledge and one of them to minimize the given cost is chosen as the replanned sequence in order for the slave process to escape from the unexpected state and resume the nominal plan.

This paper presents a new planning/control method to integrate the master and slave site in a teleprogramming system in the framework of a discrete-event dynamic system model. Specifically, a simple telerobotic part-mating task environment is modelled based on a class of controlled Petri net (CPN) by associating contact states between objects with state places, and the model is used as the source of a priori knowledge for successful accomplishment of the given task in the slave site. The a priori knowledge is transmitted from the master to the slave site and interpreted into the a priori knowledge base before the teleprogramming starts. Referring to the a priori knowledge base, the slave site interprets the succeeding symbolic commands from the master side and successfully executes the sequential commands to reach the final goal even in the case when an unexpected state transition occurs due to geometric uncertainties in the actual environment.