Avatar is the virtual representation of user's facial, body, and motion characteristics in computer game, social network, and augmented reality. Facial modeling needs enormous efforts to achieve immersive experience in applications like avatar chatting or online makeover. Great challenge exists in robust detection of 2D facial prominent points and mapping them to 3D models in a parameterized manner. Another challenge is how to characterize semantic components of eyes, mouth, nose, and cheek rather than low level mesh geometries. In this paper, we proposed an augmented makeover framework to deal with aforementioned challenges. Aiming to provide amateurs with flexible customizations, morphable model is constructed from a set of scanned 3D face data set. Appearance personalization is carried out in the offline phase where single image and multiple views are discussed respectively to generate deformative shape in a progressive manner. Augmentation is implemented in the online phase where a fast and robust 3D tracking is used to balance the tradeoff between accuracy and real-time requirements. By this means, immersive Human Computer Interaction such as virtual makeover and photo-realistic avatar chatting could be achieved.

Mobile Augmented Reality, as an emerging application for handheld devices, explores more natural interactions in real and virtual environments. For the purpose of accurate system response and manipulating objects in real-time, extensive efforts have been made to estimate six Degree-of-Freedom and extract robust feature to track. However there are still quite a lot challenges today in achieving rich user experience. To allow for a seamless transition from outdoor to indoor service, we investigated and integrated various sensing techniques of GPS, wireless, Inertial Measurement Units, and optical. A parallel tracking and matching scheme is presented to address the speed-accuracy tradeoff issue. Two prototypes, fine-scale mirror world navigation and context-aware trouble shooting, have been developed to demonstrate the suitability of our approach.

Finding a destination in unfamiliar environments of complex office building or shopping mall has been always bothering us in daily life. Fine-scale directional guidance, a combination of location based service and context aware service, is emerging along with the technology developments of mobile computing, wireless communication, and augmented reality. This paper addresses the core problem of how to align what we see with where we are. In order to achieve robust and accurate navigation by 2D/3D alignment, we developed a hybrid solution by fusing GPS, inertial measurement units and computer vision methods together. We have implemented a prototype of Mobile Augmented Reality and 3D navigation on the manually created virtual environment of Intel China Research Center office and surroundings.

Detection of salient moving object has great potentials in activity recognition, scene understanding, etc. However techniques to characterizing the object in fine granularity have not been well developed in real applications due to the computational intensity. The emerging multi-core technology in hardware design provides an opportunity for the compute intensive algorithms to boost speed in parallel. This paper proposed a scalable approach to detecting salient moving object which is designed inherently for parallelization. To characterize the object in fine granularity, we extract color-texture homogenous regions as the basic processing unit by image segmentation. To identify salient object, we generate probabilistic template by learning the space-time context. The parallel algorithm is implemented using OpenMP. Evaluations have been carried out on sports, news, and home video data. For the CIF size image, we get processing speed of 51.1 frames per second and near linear speed up on an eight-core machine. It indicates that the algorithm parallelization is a promising solution for practical applications in the multimedia field.

Bi-layer video segmentation (segmentation of videos into foreground layer and background layer) has attracted a lot of research interests recently. The algorithm can be applied into many vision and multimedia applications, such as human computer interaction, gesture recognition, object detection/tracking and personal video editing. Traditional approaches cluster pixels into homogenous regions based on color distributions or motion patterns. However, spatial or temporal clues alone are insufficient to distinguish objects because different objects may share similar colors or motions. Nowadays, the availability of stereo cameras provides the possibility to recover certain 3D depth information from videos. In this paper, we propose bi-layer segmentation from stereo video sequences by fusing multiple cues, including 3D depth information, color/texture distribution, and motion vectors. We also explore temporal-spatial coherence among a consecutive sequence of frames in order to reduce noises from a single frame.

Consumers are making increasing use of digital camera, phone camera, and camcorder every minute. Normal users have desires to produce high-quality photos although they are less of photography techniques. Consumers hunger for online instructing on how to capture a good photograph in terms of camera pose (location, orientation, field of view, etc.) Making it ubiquitously accessible is essential which requires excessive computation and communication capabilities over existing portable devices. The emerging Mobile Internet Devices (MIDs) and Ultra Mobile PCs (UMPCs) have been equipped with enabling features and turn into an alternative. This paper presents a new Location Based Service for photography assistance. It consists of three major components, focus of attention modeling, image quality assessment, and camera pose matching, to deal with the geo-referenced photos retrieved from Internet. Those techniques have been widely studied in the research domains of computer vision and pattern recognition. Having the MID / UMPC platforms at hand, richer experiences on Internet, such as instant travel blogging and photo sharing are more convenient.

In automated home video editing, selecting out the most informative contents from the redundant footage is challenging. This paper proposes an information-theoretic approach to content selection by exploring the dependence relations between who (characters) and where (scenes) in the video. First the footage is segmented into basic units about the same characters at the same scene. To compactly represent the dependence relations between scenes and characters, contingency table is used to model their co-occurrence statistics. Suppose the contents about which characters at which scene are dominating by two random variables, an optimal selection criterion is proposed based on joint entropy. To improve the computation efficiency, a pruned N-Best heuristic algorithm is presented to search the most informative video units. Experimental results demonstrated the proposed approach is flexible and effective for automated content selection.

In order to help the user to grasp the long video content quickly, this paper proposes a novel video summarization approach based on redundancy removal and content ranking. By video parsing and cast indexing, the approach first constructs a story board to let user know about the main scenes and the main actors in the video. Then it generates a "story-constraint summary" by key frame clustering and repetitive segment detection. To shorten the video summary length to a target length, our approach constructs a "time-constraint summary" by important factor based content ranking. Extensive experiments are carried out on TV series, movies, and cartoons. Good results demonstrate the effectiveness of the proposed method.

In the field of automated home video editing, exploring the dependence relations between who (character) and where (scene) makes great sense to end-users for content selection. However, such techniques have not been well developed in real applications due to their computational intensity. The emerging multi-core architectures provide an opportunity to speed up those compute expensive algorithms if shift from serial thinking to parallelism. This demonstration presents a scalable parallel system for home video editing. In a realtime processing speed, the system analyzes how many characters and scenes are captured and provides end-users with flexible preference customization. Through kernel module optimization and data-level parallelization, evaluations on a real 8-core machine indicates a near linear speed up could be achieved along with the increasing number of cores.