This paper proposes a new dentity, gender, and age recognition convergence system for robot environments. In a robot environment, it is difficult to apply deep learning based methods because of various limitations. To overcome the limitations, we propose a shallow deep-learning fusion model that can calculate identity, gender, and age at once, and a technique for improving recognition performance. Using convergence network, we can obtain three pieces of information from a single input through a single operation. In addition, we propose a 2D / 3D augmentation method to generate virtual additional datasets for learning data. The proposed method has a smaller model size and faster computation time than existing methods and uses a very small number of parameters. Through the proposed method, we finally achieved 99.35%, 90.0%, and 60.9% / 94.5% of performance in identity recognition, gender recognition, and age recognition. In all experiments, we did not exceed the state-of-the-art results, but compared to other studies, we obtained performance similar to the previous study using only less than 10% parameters. In some experiments, we also achieved state-of-the-art result.

In this paper, we present an affect recognition system for measuring the engagement level of children using the Kinect while performing a multiple intelligence test on a computer. First of all, we recorded 12 children while solving the test and manually created a ground truth data for the engagement levels of each child. For a feature extraction, Kinect for Windows SDK provides support for a user segmentation and skeleton tracking so that we can get 3D joint positions of an upper-body skeleton of a child. After analyzing movement of children, the engagement level of children’s responses is classified into two classes: High or Low. We present the classification results using the proposed features and identify the significant features in measuring the engagement.

This paper describes a novel RGB-D-based visual target tracking method for person-following robots. We enhance a single-object tracker, which combines RGB and depth information, by exploiting two different types of distracters. First set of distracters includes objects existing near-by the target, and the other set is for objects looking similar to the target. The proposed algorithm reduces tracking drifts and wrong target re-identification by exploiting the distracters. Experiments on real-world video sequences demonstrating a person-following problem show a significant improvement over the method without tracking distracters and state-of-the-art RGB-based trackers. A mobile robot following a person is tested in real environment.

This paper describes the reason that perception technology in HRI has not given high performance enough to be used for commercial service robots. As a practical solution for better performance of perception technology, we propose a perception framework with a dedicated perception engine called a perception demon. The demon in the proposed framework constantly collects evidences and analyses them better by combining various types of individual perception components. The proposed framework enables robot makers to easily get more reliable information on humans without concerns about optimizing perception components to their robots.

We propose a user pose verification technique using depth information to compare their pose with expert riders. Xtion sensor by Asus is used for gathering a depth data in real world. The user pose verification algorithm is divided into two categories: user segmentation and user pose verification. In user segmentation step, body parts are segmented based on the region growing algorithm from a head point (i.e. seed point). Then, a simple algorithm is used to generate skeletal joints in the segmented body parts. Finally, the user pose is investigated by the standard pose of experts in the same situation.

In everyday lives, so many events happen around us. Although not all of them are important to us, we have to process them all to be able to isolate the significant events. Consciousness is a constant awareness of the environment, which can be acquired only by such an exhaustive approach. As an autonomous agent, a robot is also required to be conscious, which is not yet properly realized. In this paper, we discuss the strategy to organize various perception technologies to achieve the consciousness of the robots. Especially, an independent process that constantly monitors its surroundings without the intervention of higher processes is proposed. This process, which is called a perception daemon, is discriminated from the traditional component-based approach, where individual perception technology is provided as a passive function so that the service applications should call whenever it is necessary.

This paper presents a person following robot equipped a RGB-D imaging sensor. We introduce three modules of a visual target tracking, target detection, and robot control. For the tracking module, distracters existing near-by the target are explicitly tracked to support target tracking. Preliminary tests showed that the robot robustly follows the target in uncontrolled environment with cluttered background and uneven illumination.

Robotic person-following is an essential component for natural human robot interaction. To follow a person, the robot should track the target person robustly and in real time. Object tracking algorithms in the computer vision field typically require abundant features and heavy computing power, and thus cannot be directly applied to person-following robots due to the problems arising in practical robotic environments. This paper proposes a robotic person-tracking algorithm based on modified multiple instance learning. In order to resolve the problems raised by the rearward view of the target person, the tracker is modified to be guided by color histogram back-projection. Additionally, the search area model is modified from circle to ellipse and the number of features is reduced so that the tracker should adapt the robotic environment in real-time. The algorithm is validated through system integration and experiments.

This paper presents a character segmentation method to address automatic number plate recognition problem. The method considered pixel intensity, character appearance, and arrangement of characters altogether to segment character regions. The method firstly discovers candidate blobs of characters by using connected component analysis and appearance-based character detection. A character recognizer is used for removing redundant and noisy blobs. Then, a trained classifier selects character blobs among the candidates by examining arrangement of the blobs. Experimental results show an achievement of 98.3% of segmentation rate, which prove the effectiveness of our method.

This paper presents a user recognition system, using face, height, and clothes color features under the special assumption that is a user is monitored and tracked. In real human-robot interaction situation, all information cannot be provided at the same time and some parts of frames in a video have no clues at all. In the proposed system, tracking is an important feature to recognize a user because data in the previous frames can be utilized. We propose an information update method that efficiently updates similarity results. This system is tested using the movie clips acquired under the unconstrained environment including illumination variation, several distance from a camera to the user, and various view types of human body.

A character segmentation algorithm for automatic license plate recognition is presented in this paper. Character regions are selected through binarization, connected component analysis, and character recognition. A blob analysis operation excludes noisy blobs, merges fragmented blobs, and splits clumped blobs. A character segmentation module achieved an accuracy rate of 97.2%. The recognition accuracy of the complete system with license plate localization was 90.9%. In depth analysis of failure cases is also provided for better understanding of the algorithm and a future development direction.

This paper investigates image transformations as preprocessing steps that can be applied toward a state-of-the-art age classification framework of manifold learning. We report on the experimental results of four different preprocessing methods in terms of classification accuracy using a large training and test face database. The use of histograms of oriented gradient (HOG) descriptors increases the classification accuracy from 46 to 75%. Robustness to the rotation and translation is also tested.

The fully automatic face identification on the robot system usually meet challenging environments due to the motion blur, illumination change, and erroneous face normalization caused by imperfect face detection. In the face identification system, multiple images of same individual in gallery set can be used as useful sources for good identification. This paper introduces the face identification system using efficient comparison method between gallery and probe set using HOG features. The comparison method based on blocks makes possible that the system could effectively use the limited registration images in the classification step. The whole process is validated on ETRI-HRI Database acquired in uncontrolled environment.

Recently, some 2D based algorithms were proposed for image classification. In this paper, we propose two-dimensional logistic regression for two-class problem. This method has an advantage of memory saving and better performance than other 2D based approaches. The experimental results show two dimensional logistic regression is superior to one dimensional logistic regression and other popular two-dimensional methods.

In this study, we present a simple on-line speaker enrollment and identification among human-robot interaction (HRI) with intelligent service robots. For this purpose, speaker enrollment is performed through rhythmical singing voices or a simple game such as paper-scissors-rock. While the conventional enrollment methods frequently used in the security area should be cooperative, the proposed approach can be enrolled in a very natural way. After enrolling, the text-independent speaker recognition is accomplished by using the well-known mel-frequency cepstral coefficients (MFCC) and Gaussian mixture models (GMM). The experimental results reveal that the proposed approach yields better recognition performance in comparison to the results obtained by the conventional enrollment method.

This paper describes a method to recognize the age and gender of a user on the basis of human speech. Using voice source characteristics of the Mel frequency cepstral coefficients (MFCCs), a Gaussian mixture model (GMM) technique is applied in an effort to discover the age, gender, and other information as regards a user. On the basis of this information, service applications for robots can satisfy users by offering services adaptive to the special needs of specific user groups that may include adults and children as well as females and males. The major aim of this paper is to discover the voice source parameters of age and gender and to classify these two characteristics simultaneously. ETRI-VoiceDB2006 was employed to evaluate the proposed method.

User recognition is one of the most fundamental functionalities for intelligent service robots. However, in robot applications, the conditions are far severer compared to the traditional biometric security systems. The robots should be able to recognize users non-intrusively, which confines the available biometric features to face and voice. Also, the robots are expected to recognize users from relatively afar, which inevitably deteriorates the accuracy of each recognition module. In this paper, we tried to improve the overall accuracy by integrating the evidences issued by independently developed face and speaker recognition modules. Each recognition module exhibits different statistical characteristics in representing its confidence of the recognition. Therefore, it is essential to transform the evidences to a normalized form to integrate the results. This paper introduces a novel approach to integrate mutually independent multiple evidences to achieve an improved performance. Typical approach to this problem is to model the statistical characteristics of the evidences by well-known parametric form such as Gaussian. Using Mahalanobis distance is a good example. However, the characteristics of the evidences often do not fit into the parametric models, which results in performance degradation. To overcome this problem, we adopted a discrete PDF that can model the statistical characteristics as it is. To confirm the validity of the proposed method, we used a multi-modal database that consists of 10 registered users and 550 probe data. Each probe data contains face image and voice signal. Face and speaker recognition modules are applied to generate respective evidences. The experiment showed an improvement of 11.27% in accuracy compared to the individual recognizers, which is 2.72% better than the traditional Mahalanobis distance approach.

Ambient noise caused by room acoustics usually degrades quality and reliability of speech recognition for the service robot application. In this paper, we propose enhancing the robustness of speech recognition for home service robots by combining blind equalization with classification. In particular, the linear predictive code (LPC) is used for feature extraction. The constant modulus algorithm (CMA) is combined with the radial basis function (RBF) neural network to form a robust classifier with automatic equalization capability to reduce the room service effect. Using real speech data collected by WEVER-R2 acoustic robot, it is shown that the proposed method can increase the speech recognition rate from 74.46% to 87.23% for age recognition and from 80.95% to 95.09% for gender recognition.

In this paper, an algorithm for human head detection over a distance exceeding 2.5 m between a camera and an object is described. This algorithm is used for the control of a robot, which has the additional limits of a moving camera, moving objects, various face orientations, and unfixed illuminations. With these circumstances, only the assumption that human head and body contours have an omega (Omega) shape is made. In order to separate the background from these omega shapes, the three basic features of gray, color and edge are used, and combined. The skin color cue is very useful when the image stream is frontal and has large face regions, and additionally has no background objects similar to the skin color. The gray cue is also important when captured faces have a lower gray level than background objects. The edge cue is helpful when captured background objects have similar gray levels and colors to those of a head, but can be discriminated by edges. Since these three methods have roughly orthogonal failure results, they serve to complement each other. The next step is a splitting method between the head and body region using the proposed method. The final step is an ellipse fitting and a head region verification algorithm. The results of this algorithm provide robustness for head rotation, illumination changing, and variable head sizes. Furthermore, it is possible to carry out real time processing

The ability to recognize people is positively necessary for interaction between robots and people. In this paper, we introduce intelligent security system that is able to recognize, track and remember user's face. System uses moving camera that has pan/tilt and zoom function like robot's eyes. In order to construct system, we propose the face identification method based on combining multiple features and applying multiple matching algorithms. Also, we address the benefits of multiple instances and show that identification ability can be improved by using proper combination strategy

The user authentication problem is discussed in the context of robot environments. Although authentication is indispensable for useful robot services, the traditional approaches that have been applied in strictly regulated environments cannot be applied as they are when both the robot and humans are freely moving around. In this paper, a new authentication system redesigned to be adapted to robot environments is described. In the proposed system, semi-biometrics and user tracking are incorporated to cope with the limited applicability of traditional authentication techniques. Semi-biometrics indicates a set of features useful for discriminating persons, but only in interested group of persons and in interested frame of time. As a representative semi-biometric feature, body height and color characteristics of the clothes are investigated. Especially, a novel method to measure body height with single camera is proposed. Also by incorporating tracking functionality, the system could maintain the user status information continuously, which is useful not only for recognition but also for finding a designated person.

The purpose of this study is to prove that using gesture recognition through implementing the method of touring into the picture makes navigation very convenient and easy compared to navigation using mouse and keyboard. The method of touring into the picture is a simple technique that converts an 2D image into 3D animation. This study shows how to convert 2D image into 3D navigation, implementing the method previously proposed by Horry and Phy on the personal computer. Hand shape gesture recognition which functions as a user interface has been developed into three types: Type-1 means pause, Type-2 means showing directions on 2D; Type-3 is designed to assign upper and lower depth in space. These three types play the role as virtual mouse and keyboard in 3D-rendering image. Above all, using hand-shaped gestures contributes to the ease of user's navigation of 3D space..