The facial expression recognition has been a topic of active researches given a result the proposal of several efficient algorithms; however, in most cases they remain limited to controlled conditions situations. In this study, we tackle the challenge of recognizing emotions through the facial expression into activities in-the-wild adding the accuracy rate for each expression. To this end we an algorithm that allows accurate face expression recognition in an uncontrolled environment, that means different kind of illumination, backgrounds, occlusions, face's profiles, etc. Proposed system firstly detects different profile of face (left, frontal and right), Then it uses only the frames in which the face profile is frontal, in the next step the face regions of interest (ROI) are segmented automatically to carry out the feature extraction. We use a classifier based on clustering, it has the advantage that if a new class (emotion) is added, it is not necessary to train this completely. Proposed system was evaluated using short video clips of several pictures together with description sentences describing the main activity in the video. The evaluation results show that the proposed scheme is able to recognize the face's profiles with the recognition rate to approximately 93% and principal emotions in unconstrained video sequences.

In this paper, we present a method of estimating a child's attention, one of the more important human mental states, in a free-play scenario of child-robot interaction. First, we developed a system that could sense a child's verbal and non- verbal multimodal signals such as gaze, facial expression, proximity, and so on. Then, the observed information was used to train a Support Vector Machine (SVM) to estimate a human's attention level. We investigated the accuracy of the proposed method by comparing with a human judge's estimation, and obtained some promising results which we discuss here.

Recent studies have shown that human beings unconsciously use signals that represent their thoughts and/or intentions when communicating with each other. These signals are known as “honest signals.” This study involves the use of a sociometer to capture multimodal data resulting from the interaction between humans. These data are then used to model the interaction using a multimodal hierarchical Dirichlet process hidden Markov model, which is then implemented in the robot. The model enables robots to generate “honest signals” and to interact in a natural manner.

Muhammad Attamimi, Yuji Ando, Tomoaki Nakamura, Takayuki Nagai, Daichi Mochihashi, Ichiro Kobayashi, Hideki Asoh. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015.

It is difficult to design robotic playmates for introverted children. Therefore, we examined how a robot should play with such shy children. In this study, we hypothesized and tested an effective play strategy for building a good relationship with shy children. We conducted an experiment with 5- to 6-year-old children and a humanoid robot teleoperated by a preschool teacher. We developed a valid play strategy for shy children.

In the field of intelligent robotics, object handling by robots can be achieved by capturing not only the object concept through object categorization, but also other concepts (e.g., the movement while using the object), as well as the relationship between concepts. Moreover, capturing the concepts of places and people is also necessary to enable the robot to gain real-world understanding. In this study, we propose multi-layered multimodal latent Dirichlet allocation (mMLDA) to realize the formation of various concepts, and the integration of those concepts, by robots. Because concept formation and integration can be conducted by mMLDA, the formation of each concept affects others, resulting in a more appropriate formation. Another issue to be addressed in this paper is the language acquisition by the robots. We propose a method to infer which words are originally connected to a concept using mutual information between words and concepts. Moreover, the order of concepts in teaching utterances can be learned using a simple Markov model, which corresponds to grammar. This grammar can be used to generate sentences that represent the observed information. We report the results of experiments to evaluate the effectiveness of the proposed method.

The influence of holding hands on the relationship building process between children and robots is investigated in this study. In particular, at the first meeting, it is difficult for a child to be open if he/she starts to rebuff the robot partner. This significantly degrades the possibility of the child forming a friendship relationship with the robot. Thus, the initial approach of the robot to the child in the early stage of the relationship building process should be appropriate. We hypothesize that physical embodied communication, such as walking hand in hand, improves the relationship between children and robots. A holding hands system was implemented in a real robot, and an experiment was conducted at a kindergarten to validate our hypothesis. The results strongly support our hypothesis.

In this study, we propose a system to enable robots to navigate more efficiently to target objects in order to carry out mobile manipulation tasks. Because a robot arm has limited reach, a target object cannot be grasped if the distance to the object exceeds the reach. At the same time, navigation errors may increase as the robot moves around in an environment. To mitigate these problems, we propose the use of maps that express the arm reachability and navigation reachability. Places with minimum probability of navigation errors and maximum tolerable navigation errors for the reach of the arm can be determined by integrating these two maps. We also consider the fact that the target object position is not always known. To cope with this problem, we introduce an object existence map that represents the ambiguity of the target object position, considering the past position. If the ambiguity is large, it is possible to select a place from where the object searching task can be performed with a wide angle of view by the robot. Efficient object searching can be achieved by changing the task objective from searching to grasping and vice versa. We conducted some mobile manipulation tasks to evaluate the proposed method. The results showed that the proposed system can perform such tasks effectively.

This paper presents a method for learning novel objects from audio-visual input. Objects are learned using out-of-vocabulary word segmentation and object extraction. The latter half of this paper is devoted to evaluations. We propose the use of a task adopted from the RoboCup@Home league as a standard evaluation for real world applications. We have implemented proposed method on a real humanoid robot and evaluated it through a task called “Supermarket”. The results reveal that our integrated system works well in the real application. In fact, our robot outperformed the maximum score obtained in RoboCup@Home 2009 competitions.

This paper presents a novel 3D measurement system, which yields both depth and color information in real time, by calibrating a time-of-flight and two CCD cameras. The problem of occlusions is solved by the proposed fast occluded-pixel detection algorithm. Since the system uses two CCD cameras, missing color information of occluded pixels is covered by one another. We also propose a robust object recognition using the 3D visual sensor. Multiple cues, such as color, texture and 3D (depth) information, are integrated in order to recognize various types of objects under varying lighting conditions. We have implemented the system on our autonomous robot and made the robot do recognition tasks (object learning, detection, and recognition) in various environments. The results revealed that the proposed recognition system provides far better performance than the previous system that is based only on color and texture information.