Recently, many attempts have been made to apply real-time simultaneous localization and mapping (SLAM) technology to augmented reality (AR) applications. Such AR systems based on SLAM technology are generally implemented by augmenting virtual objects onto a diorama or three-dimensional sculpture. However, a new SLAM map needs to be generated if the space or lighting where the diorama is installed changes. This leads to the problem of updating the coordinate system each time a new SLAM map is generated. Updates to the coordinate system signify that the positions of the virtual objects placed in the AR space change as well. Therefore, we proposed a SLAM map regeneration technique in which the existing coordinate system is maintained even if a new map is generated.

We describe a miniature diorama AR system called ‘Miniature AR’ which can be applied to a mechanical diorama to extend the content’s feasibility by overlapping virtual objects on the complex diorama structure. The previous AR researches for the diorama are usually based on 2D planar recognition, and multiple user experience cannot be considered due to multi-devices synchronization. To overcome these constraints, in this paper, we show a new diorama AR system suitable for a tiny complex structure. The contributions of our work are i) design of the diorama AR system, ii) AR space generation and 6DOF view device tracking for diorama, iii) multiple view and event synchronization for multiple users. The utility of the approach has been demonstrated under a real diorama environment (miniature of a ski slope) using mobile devices.

A system control technique (changing the interaction mode) is needed to provide various spatial interaction techniques for virtual, augmented and mixed reality (VR/AR/MR) applications. We propose HandPoseMenu, a hand posture-based virtual menus system in 3D space. HandPoseMenu recognizes the non-dominant hand's posture using a depth sensor that is attached to a wearable display and presents the virtual menu corresponding to the hand posture beside the non-dominant hand. When the user selects a desired function among the presented menu as a dominant hand, the interaction mode changes. The combination of the two system controls (graphical menu and hand posture) makes it possible to display the menu that the user wants to use in 3D space. This paper describes the implementation of HandPoseMenu and its application.

For live television broadcast such as the educational program for children conducted through viewer participation, the smooth integration of virtual contents and the interaction between the casts and them are quite important issues. Recently there have been many attempts to make aggressive use of interactive virtual contents in live broadcast due to the advancement of AR/VR technology and virtual studio technology. These previous works have many limitations that do not support real-time 3D space recognition or immersive interaction. In this sense, we propose an augmented reality based real-time broadcasting system which perceives the indoor space using a broadcasting camera and a RGB-D camera. Also, the system can support the real-time interaction between the augmented virtual contents and the casts. The contribution of this work is the development of a new augmented reality based broadcasting system that not only enables filming using compatible interactive 3D contents in live broadcast but also drastically reduces the production costs. For the practical use, the proposed system was demonstrated in the actual broadcast program called “Ding Dong Dang Kindergarten” which is a representative children educational program on the national broadcasting channel of Korea.

Visual surveillance finds increasing deployment for monitoring urban environments. Operators need to be able to determine identity from surveillance images and often use face recognition for this purpose. In surveillance environments, it is necessary to handle pose variation of the human head, low frame rate, and low resolution input images. We describe the first use of gait to enable face acquisition and recognition, by analysis of 3-D head motion and gait trajectory, with super-resolution analysis. We use region- and distance-based refinement of head pose estimation. We develop a direct mapping to relate the 2-D image with a 3-D model. In gait trajectory analysis, we model the looming effect so as to obtain the correct face region. Based on head position and the gait trajectory, we can reconstruct high-quality frontal face images which are demonstrated to be suitable for face recognition. The contributions of this research include the construction of a 3-D model for pose estimation from planar imagery and the first use of gait information to enhance the face extraction process allowing for deployment in surveillance scenarios.

In the last decade, face analysis, e.g. face recognition, face detection, face tracking and facial expression recognition, is a very lively and expanding research field. As computer animated agents and robots bring a social dimension to human computer interaction, interest in this research field is increasing rapidly. In this paper, we introduce an artificial facial expression mimic system which can recognize facial expressions of human and also imitate the recognized facial expressions. We propose a classifier that is based on weak classifiers obtained by using modified rectangular features to recognize human facial expression in real-time. Next, we introduce our robot that is manipulated by a distributed control algorithm and that can make artificial facial expressions. Finally, experimental results of facial expression recognition and facial expression generation are shown for the validity of our artificial facial expression imitator

For face tracking in a video sequence, various face tracking algorithms have been proposed. However, most of them have a difficulty in finding the initial position and size of a face automatically. In this paper, we present a fast and robust method for fully automatic multi-view face detection and tracking. Using a small number of critical rectangle features selected and trained by Adaboost learning algorithm, we can detect the initial position, size and view of a face correctly. Once a face is reliably detected, we can extract face and upper body color distribution from the detected facial regions and upper body regions for building a robust color modeling respectively. Simultaneously, each color modeling is performed by using k-means clustering and multiple Gaussian models. Then, fast and efficient multi-view face tracking is executed by using several critical features and a simple linear Kalman filter. Our proposed algorithm is robust to rotation, partial occlusions, and scale changes in front of dynamic, unstructured background. In addition, our proposed method is computationally efficient. Therefore, it can be executed in real-time.

In this paper, we propose a method of selecting new types of rectangle features that are suitable for facial expression recognition. The basic concept in this paper is similar to Violar's approach, which is used for face detection. Instead of previous Haar-like rectangle features, we choose rectangle features for facial expression recognition among all possible rectangle types in a 3/spl times/3 matrix form using the AdaBoost algorithm. Also, the facial expression recognition system constituted with the proposed rectangle features is compared to that with previous rectangle features with regard to its capacity. The results show that the proposed approach has better performance in facial expression recognition in terms of simulation and experimental results.