Point cloud registration is a popular research topic and has been widely used in many tasks, such as robot mapping and localization. It is a challenging problem when the overlap is small, or the rotation is large. The problem has not been well solved by existing methods such as the iterative closest point (ICP) and its variants. In this paper, a novel method named principal coordinate alignment with overlap tuning (PCAOT) is proposed based on the Manhattan world assumption. It solves two key problems together, the transformation estimation and the overlap estimation. The overlap is represented by a 3D cuboid and the transformation is computed only within the overlap region. Instead of finding point correspondence as in traditional methods, we estimate the rotation by principal coordinates alignment, which is faster and less sensitive than ICP and its variants to small overlaps and large rotations. Evaluations demonstrate that our method achieves much better results than the ICP and its variants when the overlap ratio is smaller than 50%, or the rotation angle is larger than 60°. Especially, it is effective when the overlap ratio is less than 30%, or the rotation angle is larger than 90°.

Avatar is the virtual representation of user's facial, body, and motion characteristics in computer game, social network, and augmented reality. Facial modeling needs enormous efforts to achieve immersive experience in applications like avatar chatting or online makeover. Great challenge exists in robust detection of 2D facial prominent points and mapping them to 3D models in a parameterized manner. Another challenge is how to characterize semantic components of eyes, mouth, nose, and cheek rather than low level mesh geometries. In this paper, we proposed an augmented makeover framework to deal with aforementioned challenges. Aiming to provide amateurs with flexible customizations, morphable model is constructed from a set of scanned 3D face data set. Appearance personalization is carried out in the offline phase where single image and multiple views are discussed respectively to generate deformative shape in a progressive manner. Augmentation is implemented in the online phase where a fast and robust 3D tracking is used to balance the tradeoff between accuracy and real-time requirements. By this means, immersive Human Computer Interaction such as virtual makeover and photo-realistic avatar chatting could be achieved.

This paper proposes a novel patch-wise image inpainting algorithm using the image signal sparse representation over a redundant dictionary, which merits in both capabilities to deal with large holes and to preserve image details while taking less risk. Different from all existing works, we consider the problem of image inpainting from the view point of sequential incomplete signal recovery under the assumption that the every image patch admits a sparse representation over a redundant dictionary. To ensure the visually plausibility and consistency constraints between the filled hole and the surroundings, we propose to construct a redundant signal dictionary by directly sampling from the intact source region of current image. Then we sequentially compute the sparse representation for each incomplete patch at the boundary of the hole and recover it until the whole hole is filled. Experimental results show that this approach can efficiently fill in the hole with visually plausible information, and take less risk to introduce unwanted objects or artifacts.

This paper presents a novel audio-visual fusion method for speech detection, which is an important front-end for content-based video processing. This approach aims to extract homogeneous speech segments from the accompanying audio stream in real-world movie/TV videos with the help of video captions. Note that captions are mainly created to help viewers to follow the dialog, rather than to accurately locate the speech regions. We propose a caption-aided speech detection approach, which makes use of both caption information and audio information. The inaccurate positions of the captions are refined through using audio features (pitch and MFCCs) and BIC-based acoustic change detection. Comparison experiments against several other traditional speech detection approaches are conducted, showing that the proposed approach improves the speech detection performance greatly.

For video summarization and retrieval, one of the important modules is to group temporal-spatial coherent shots into high-level semantic video clips namely scene segmentation. In this paper, we propose a novel scene segmentation and categorization approach using normalized graph cuts(NCuts). Starting from a set of shots, we first calculate shot similarity from shot key frames. Then by modeling scene segmentation as a graph partition problem where each node is a shot and the weight of edge represents the similarity between two shots, we employ NCuts to find the optimal scene segmentation and automatically decide the optimum scene number by Q function. To discover more useful information from scenes, we analyze the temporal layout patterns of shots, and automatically categorize scenes into two different types, i.e. parallel event scenes and serial event scenes. Extensive experiments are tested on movie, and TV series. The promising results demonstrate that the proposed NCuts based scene segmentation and categorization methods are effective in practice.

This paper addresses the problem of audio segmentation in practical media (e.g. TV series, movies and etc.) which usually consists of segments in various lengths with quite a portion of short ones. An unsupervised audio segmentation approach is presented, including a segmentation-stage to detect potential acoustic changes, and a refinement-stage to refine these candidate changes by a tri-model Bayesian information criterion. Experiments show that the proposed approach has good detectability of short segments and the novel tri-model BIC effectively improves the overall segmentation performance.

In this paper, a fast and robust multi-view lip localization algorithm in video is proposed. We consider lip localization as a binary classification problem, where a classifier is learned to distinguish between the lip and the region surrounding it. The classifier we use here is a histogram-based one which exploits the anthropometrical properties of the human face with the help of face scale normalization. Due to the perceptual uniformity and robustness for lip/skin color variations across different people, we adopt CIELUV color model to represent the color of lip. After classification, we propose a novel projection-cut algorithm by spatial deviation analysis (SDA) to locate the lip, which is effective to deal with the background clutters. Experimental results on teleplay videos demonstrate that the proposed approach is efficient and robust for lip localization.

Replay is one of the key cues indicating highlights in sports videos. A replay is usually sandwiched by two identical logos which prompt the start and end of a replay. A logo transition usually contains 10-30 frames, describes a flying or varying object(s). In this paper, a reliable logo and replay detecting approach is proposed. It contains two main stages: first, a logo transition template is unsupervised learned, a key frame (K-frame) and a set of pixels that describes logo object (logo pixels, L-pixels) accurately are also extracted; second, the learned information are used jointly to detect logos and replays in the video. In addition to traditional color analysis, optical flow feature is employed to depict the movement of the logo object(s). Extensive experiments show that the proposed approach can reliably detect logos and replays regardless of the types of sports videos.

In this paper, we present a novel approach which employs the temporal redundancy of video texts to improve the performance of text segmentation and stroke extraction from complex background. We first demonstrate how to obtain the required aligned text image sequences from video frames via a robust corners-set matching scheme. Then the changing background pixels can be identified and removed by exploiting the statistics of the temporal redundancy of video. The text stroke pixels can be clearly separated from the complex background. Experiments on TV programs and movies videos show the proposed approach can generate clean text strokes image which can efficiently improve the OCR's performance compare to some traditional approaches.

In this paper, a real time and robust multi-view lip localization in video frames is proposed. The objective of this research is to exactly localize lip surrounding box with sub-pixel accuracy in low quality video frames. This is achieved by multi-view face detection in video frames, lip ROI extraction, color space transformation, vertical edge feature detection and lip localization by a feature-spatial profile analysis on a joint color-edge plane. One possible application of talking face detection in video is used to evaluate our work. Experimental results on TV programs and movies demonstrate that the proposed approach is efficient and robust in lip localization.

In order to help the user to grasp the long video content quickly, this paper proposes a novel video summarization approach based on redundancy removal and content ranking. By video parsing and cast indexing, the approach first constructs a story board to let user know about the main scenes and the main actors in the video. Then it generates a "story-constraint summary" by key frame clustering and repetitive segment detection. To shorten the video summary length to a target length, our approach constructs a "time-constraint summary" by important factor based content ranking. Extensive experiments are carried out on TV series, movies, and cartoons. Good results demonstrate the effectiveness of the proposed method.

Soccer highlight detection is an active research topic in recent years. One of the difficult problems is how to effectively fuse multi-modality cues, i.e. audio, visual and textual information, to improve the detection performance. This paper proposes a novel two-dependence Bayesian network (2d-BN) based fusion approach to soccer highlight detection. 2d-BN is a particular Bayesian network which assumes that each variable depends on two other variables at most. Through this assumption, 2d-BN can not only characterize the relationships among features but also be trained efficiently. Extensive experiments demonstrate the effectiveness of the proposed method

In this paper, we present a tree-partition algorithm for parallel mining of frequent patterns. Our work is based on FP-Growth algorithm, which is constituted of tree-building stage and mining stage. The main idea is to build only one FP-Tree in the memory, partition it into several independent parts and distribute them to different threads. A heuristic algorithm is devised to balance the workload. Our algorithm can not only alleviate the impact of locks during the tree-building stage, but also avoid the overhead that do great harm to the mining stage. We present the experiments on different kinds of datasets and compare the results with other parallel approaches. The results suggest that our approach has great advantage in efficiency, especially on certain kinds of datasets. As the number of processors increases, our parallel algorithm shows good scalability.

As an extension of Bayesian network, module network is an appropriate model for inferring causal network of a mass of variables from insufficient evidences. However learning such a model is still a time-consuming process. In this paper, we propose a parallel implementation of module network learning algorithm using OpenMP. We propose a static task partitioning strategy which distributes sub-search-spaces over worker threads to get the tradeoff between load-balance and software-cache-contention. To overcome performance penalties derived from shared-memory contention, we adopt several optimization techniques such as memory pre-allocation, memory alignment and static function usage. These optimizations have different patterns of influence on the sequential performance and the parallel speedup. Experiments validate the effectiveness of these optimizations. For a 2,200 nodes dataset, they enhance the parallel speedup up to 88%, together with a 2X sequential performance improvement. With resource contentions reduced, workload imbalance becomes the main hurdle to parallel scalability and the program behaviors more stable in various platforms.

Single nucleotide polymorphisms (SNPs) are loci on the genome where different alleles are observed in the population. It has been observed that there might be some patterns or context dependences in the sequence segments adjacent to SNPs sites. Discovering such dependences is very important for understanding possible origins of SNPs in evolution. We collected 519,767 bi-allelic SNPs of human in gene regions from HGBASE and separated them in 6 groups according to the types of alleles at the SNP loci. Bayesian network structure learning technique is applied to discovery of possible dependences in sequence segments around these sites as well as in reference sequences collected as comparison. Noticeable probabilistic correlations among some loci were detected in all the 6 SNP groups and nothing significant was found in the reference sequences. The dependence relations found with different SNP groups are different. These putative context dependences around SNP sites provide important hints for further analyzing SNP-related sequences patterns. The work also illustrates the powerfulness of the Bayesian network method as a tool for biological sequence analysis.

We propose dynamic Bayesian network (DBN) based synchronous and asynchronous multi-stream models for noise-robust automatic speech recognition. In these models, multiple noise-robust features are combined into a single DBN to obtain better performance than any single feature system alone. Results on the Aurora 2.0 noisy speech task show significant improvements of our synchronous model over both single stream models and over a ROVER based fusion method.

DBNs (Dynamic Bayesian Networks) [1] are powerful tool in modeling time-series data, and have been used in speech recognition recently [2,3,4]. The “decoding” task in speech recognition means to find the viterbi path [5](in graphical model community, “viterbi path” has the same meaning as MPE “Most Probable Explanation”) for a given acoustic observations. In this paper we describe a new algorithm utilizes a new data structure “backpointer”, which is produced in the “marginalization” procedure in probability inference. With these backpointers, the viterbi path can be found in a simple backtracking. We first introduce the concept of backpointer and backtracking; then give the algorithm to compute the viterbi path for DBNs based on backpointer and backtracking. We prove that the new algorithm is correct, faster and more memory saving comparison with old algorithm. Several experiments are conducted to demonstrate the effectiveness of the algorithm on several well known DBNs. We also test the algorithm on a real world DBN model that can recognize continuous digit numbers.