Most of the syntax-based metrics obtain the similarity by comparing the sub-structures extracted from the trees of hypothesis and reference. These sub-structures cannot represent all the information in the trees because their lengths are limited. To sufficiently use the reference syntax information, a new automatic evaluation metric is proposed based on the dependency parsing model. First, a dependency parsing model is trained using the reference dependency tree for each sentence. Then, the hypothesis is parsed by this dependency parsing model and the corresponding hypothesis dependency tree is generated. The quality of hypothesis can be judged by the quality of the hypothesis dependency tree. Unigram F-score is included in the new metric so that lexicon similarity is obtained. According to experimental results, the proposed metric can perform better than METEOR and BLEU on system level and get comparable results with METEOR on sentence level. To further improve the performance, we also propose a combined metric which gets the best performance on the sentence level and on the system level.

In this paper we bring forth a novel stroke-based method which is simple and effective to detect texts in natural scenes. We first introduce a general mathematical model to describe character strokes from the perspective of the scale space along with difference of Gaussian filters. Then we detail a text line aggregation approach utilizing the inherent text layout. Afterwards, we set up the whole scheme with three main steps, i.e. stroke extraction, text line aggregation and verification. Finally, experiments show the advantage of our method. As strokes are considered to be the fundamental component of characters, compared to edge- or other connected-component-based methods, our method is much more reasonable.

This paper addresses the problem of affine distortions caused by viewpoint changes for the application of image retrieval. We study how to expand the visual words from a query image for better retrieval recall without the sacrifice of retrieval precision and efficiency. Our main contribution is the building of visual dictionaries that retain the mapping relationships between visual words extracted from different viewpoints of the same object. Additionally, in each mapping rule we record the affine transformation in which the two visual words are related, as a compact code of viewpoints relationships. By analogizing the concepts of verb stem and verb tense in text, we use Visual Stems to denote visual words extracted from robust local patches, and record the relationships between their affine variants as visual stem mapping rules, including the geometric relationships coded as Geometric Tenses. In this way, our method augments original visual vocabulary with sufficient and accurate expansion information. In query phase, only the objects corresponding to the same visual stems and coherent geometric tense codes will be regarded as similar ones. Moreover, the mapping rules can be learned offline with only one sample for each object. Experiments show that our method can support efficient object retrieval with high recall, requiring little extra time and space cost over traditional visual vocabularies.

Summary form only given. In this paper we propose a video coding optimization strategy for H.264/AVC encoder which uses HVS (Human Visual System) theory to improve the visual quality. It is based on a basic rule that sensitivity of distortion is different according to the image property. Most existing coding strategies which allocate fixed parameters to each unit within a frame are not reasonable because visual redundancy cannot be fully used. Instead, our scheme dynamically changes the coding strategy of different regions. It enhances the quality of regions which is more sensible and slightly decreases the quality of regions which is less sensible. The overall perceptual quality can be improved.

Human skin detection in images is desirable in many practical applications, e.g., adult-content filtering. However, existing methods are mainly pixel-based and ignore that human skin is region-based. In this paper, we introduce a successful region detector, i.e., MSER, into the skin detection by regarding the skin region as the maximally stable extremal region (MSER). We extend the original MSER to both color and texture analysis to reduce the skinlike regions1. Furthermore, to be adaptive to the dynamic illumination and chrominance, face detection is used to customize the skin color model to each image. The proposed method has achieved promising performance over our dataset, which is a challenging set with a great part of hard images. Our True Positive Rate is 81.2% under False Positive Rate 8.2%, which outperforms all of eight state-of-the-art algorithms.

Portraits, also known as images of people, constitute an important part of consumer photos. Existing methods manage portraits based on either explicit objectives, e.g., a specified person or event, or aesthetics, i.e., the aesthetic quality of portraits. This paper presents a novel system for personalized portraits ranking. First, four kinds of personalized features, i.e., composition, clothing style, affection and social relationship are proposed to quantify users' intent. Then, example-based and sketch-based user interfaces (UI) are developed, which are capable of capturing users' personal intent hardly described by queries or aesthetics. Finally, portraits ranking is implemented by combing these features together with the developed user interfaces. Experimental results show that the system performs well in providing personalized preferences and the proposed features are effective for portraits ranking. From the user study, our system gets promising results.

With the exponential growth of social media, web video retrieval based on contextual information associated with videos has attracted wide attention recently. However, state-of-the-art methods mainly focus on limited kinds of context cues and lack of unified exploration towards multiple heterogeneous contexts. In this paper, we propose a novel web video ranking framework called CommunityRank by leveraging the collective wisdom from a community perspective. Firstly, it formulizes various social relations among users, videos and tags in a heterogeneous context network and further detects its latent community structure. Then the algorithm maps videos into the community space and performs a community-oriented re-ranking through a bipartite graph model. By aggregating the multiple relations, CommunityRank can make the most of textual, visual and social contexts and leads to better search results. The encouraging performances of the proposed method on YouTube video collection demonstrate that the discovered communities reveal topics of interest emerging in collective behaviors and can facilitate web video retrieval.

Realistic human action recognition in videos has been a useful yet challenging task. Video shots of same actions may present huge intra-class variations in terms of visual appearance, kinetic patterns, video shooting, and editing styles. Heterogeneous feature representations of videos pose another challenge on how to effectively handle the redundancy, complementariness and disagreement in these features. This paper proposes a localized multiple kernel learning (L-MKL) algorithm to tackle the issues above. L-MKL integrates the localized classifier ensemble learning and multiple kernel learning in a unified framework to leverage the strengths of both. The basis of L-MKL is to build multiple kernel classifiers on diverse features at subspace localities of heterogeneous representations. L-MKL integrates the discriminability of complementary features locally and enables localized MKL classifiers to deliver better performance in its own region of expertise. Specifically, L-MKL develops a locality gating model to partition the input space of heterogeneous representations to a set of localities of simpler data structure. Each locality then learns its localized optimal combination of Mercer kernels of heterogeneous features. Finally, the gating model coordinates the localized multiple kernel classifiers globally to perform action recognition. Experiments on two datasets show that the proposed approach delivers promising performance.

Most current research on human action recognition in videos uses the bag-of-words (BoW) representations based on vector quantization on local spatial temporal features, due to the simplicity and good performance of such representations. In contrast to the BoW schemes, this paper explores a localized, continuous and probabilistic video representation. Specifically, the proposed representation encodes the visual and motion information of an ensemble of local spatial temporal (ST) features of a video into a distribution estimated by a generative probabilistic model such as the Gaussian Mixture Model. Furthermore, this probabilistic video representation naturally gives rise to an information-theoretic distance metric of videos. This makes the representation readily applicable as input to most discriminative classifiers, such as the nearest neighbor schemes and the kernel methods. The experiments on two datasets, KTH and UCF sports, show that the proposed approach could deliver promising results.

We propose a compressive video sensing scheme based on user attention model (UAM) for real video sequences acquisition. In this work, for every group of consecutive video frames, we set the first frame as reference frame and build a UAM with visual rhythm analysis (VRA) to automatically determine region-of-interest (ROI) for non-reference frames. The determined ROI usually has significant movement and attracts more attention. Each frame of the video sequence is divided into non-overlapping blocks of 16 × 16 pixel size. Compressive video sampling is conducted in a block-by-block manner on each frame through a single operator and in a whole region manner on the ROIs through a different operator. Our video reconstruction algorithm involves alternating direction l1- norm minimization algorithm (ADM) for the frame difference of non-ROI blocks and minimum total-variance (TV) method for the ROIs. Experimental results showed that our method could significantly enhance the quality of reconstructed video and reduce the errors accumulated during the reconstruction.

Query expansion is an effective method to improve the usability of multimedia search. Most existing multimedia search engines are able to automatically expand a list of textual query terms based on text search techniques, which can be called textual query expansion (TQE). However, the annotations (title and tag) around web videos are generally noisier for text-only query expansion and search matching. In this paper, we propose a novel multi-modal query expansion (MMQE) framework for web video search to solve the issue. Compared with traditional methods, MMQE provides a more intuitive query suggestion by transforming tex-tual query to visual presentation based on visual clustering. Paral-lel to this, MMQE can enhance the process of search matching with strong pertinence of intent-specific query by joining textual, visual and social cues from both metadata and content of videos. Experimental results on real web videos from YouTube demon-strate the effectiveness of the proposed method.

Word alignment plays an important role in many NLP tasks as it indicates the correspondence between words in a parallel text. Although widely used to align large bilingual corpora, generative models are hard to extend to incorporate arbitrary useful linguistic information. This article presents a discriminative framework for word alignment based on a linear model. Within this framework, all knowledge sources are treated as feature functions, which depend on a source language sentence, a target language sentence, and the alignment between them. We describe a number of features that could produce symmetric alignments. Our model is easy to extend and can be optimized with respect to evaluation metrics directly. The model achieves state-of-the-art alignment quality on three word alignment shared tasks for five language pairs with varying divergence and richness of resources. We further show that our approach improves translation performance for various statistical machine translation systems.

This paper presents a system for automatically detecting and analyzing complex player actions in moving background sports video sequences, aiming at action-based sports videos indexing and providing kinematic measurements for coach assistance and performance improvement. The system works in a coarse-to-fine fashion. For an input video, in the coarse granularity level, we automatically segment the highlights, that is, the video clips containing the desired action as summaries for general user viewing purposes; in the middle granularity level, we recognize the action types to support action-based video indexing and retrieval; and finally in the fine granularity level, the critical kinematic parameters of player action are obtained for sports professionals' training purposes. However, the complex and dynamic background of sports videos and the complexity of player actions bring considerable difficulty to the automatic analysis. To fulfill such a challenging task, robust algorithms including global motion estimation with adaptive outliers filtering, object segmentation based on adaptive background construction, and automatic human body tracking are proposed in this paper. Two visual analyzing tools: motion panorama and overlay composition, are also introduced. Real diving and jump game videos are used to test the proposed system and algorithms, and the extensive and encouraging experimental results show their effectiveness.

Event-related query is playing a more and more important role in video retrieval. However, it is still a challenge to the existing video retrieval engines for lacking the effective motion analysis. In this paper, we propose a novel re-ranking scheme for video retrieval based on motion region trajectory analysis. By focusing on the changes of the primary moving regions, we construct an intuitive motion region-based trajectory descriptor (MRTD) to depict the shot activities. In the re-ranking phase, the proposed approach takes the MRTD as a motion cue and re-ranks the baseline results by motion-related query selection and MRTD-based weighting. We evaluate our method in TRECVID2007 and 2008 datasets, and observe consistent improvement over all the baselines, leading to a greatest performance gain of 42.9%, and an average gain of 17%. The experiments also show that the motion descriptor of MRTD is fruitful for a variety of features.

Logo detection is important for brand advertising and surveillance applications. The central issues of this technology are fast localization and accurate matching. Based on key traits analysis of common logos, this paper presents a two-stage detection scheme based on spatialspectral saliency (SSS) and partial spatial context (PSC). SSS speeds up logo location and avoid the impact of cluttered background. PSC filters false matching using spatial consistency of local invariant points. The integration of SSS and PSC result in faster localization and increased accuracy. Experiments on a dataset of nearly 10,000 web images containing several popular logo types are presented. The results indicate that our method is applicable and precise for different logo detection scenarios.

This paper proposes a novel content-based copy retrieval scheme for video copy identification. Its goal is to detect matches between a doubtful video and the ones stored in the database of the legal holders of the videos. Due to various transformations the copy may has, we use visual words vector as a representation of a frame which is based on SIFT descriptor. Unlike traditional bag-of-words (BoW) based approach applied in semantic retrieval, in which the temporal variation during the video is always neglected, our matching algorithm takes into account spatial and temporal distances between a query clip and the one in database. Experiments show robustness and effectiveness of our approach according to various single and compound transformations.

Action recognition has attracted much attention for human behavior analysis in recent years. Local spatial-temporal (ST) features are widely adopted in many works. However, most existing works which represent action video by histogram of ST words fail to have a deep insight into a fine structure of actions because of the local nature of these features. In this paper, we propose a novel method to simultaneously localize and recognize action units (AU) by regarding them as 3D (x,y,t) objects. Firstly, we record all of the local ST features in a codebook with the information of action class labels and relative positions to the respective AU centers. This simulates the probability distribution of class label and relative position in a non-parameter manner. When a novel video comes, we match its ST features to the codebook entries and cast votes for positions of its AU centers. And we utilize the localization result to recognize these AUs. The presented experiments on a public dataset demonstrate that our method performs well.

The local classification methods try to simplify the complex global modeling problem by decomposing it into a set of local classification sub-problems, which is a potential key to overcome the semantic gap in multimedia content analysis. In this paper we proposed a Sample-Balancing Clustering segmentation method and an effective local classification framework named K-Nearest Sub-classifiers (KNSC). In KNSC the final prediction is an ensemble of the predictions made by K nearest local classifiers. We experimentally compare the effect of different sub-domain segmentation methods, different types of sub-classifiers and different classification/ensemble strategies. The applications on semantic analysis of TRECVID data show the good performance of our method.

Spatial relation of local image patches plays an important role in object-based image retrieval. An approach called spatial frequent items is proposed as an extension of Bag-of-Words method by introducing spatial relations between patches. Spatial frequent items are defined as frequent pairs of adjacent local image patches in polar coordinates, and exploited using data mining. Based on these frequent configurations, we develop a method to encode patches and their spatial relations for image indexing and retrieval. Besides, to avoid the interference of background patches, informative patches are filtrated based on their local entropy and self-similarity in the preprocess stage. Experimental results demonstrate that our method can be 8.6% more effective than the state-of-art object retrieval methods.

Personalization especially in the domain of information retrieval is essentially important, as users might pose the same query even when they are searching for different information. It is thus necessary to create a retrieval engine which takes into consideration the dynamic information needs of different users. This paper presents our personalized news video retrieval engine, which exploits the individual userpsilas previous browsing history to customize and enhance their future search results. Specifically, the system utilizes the news topic hierarchy, a hierarchical news topic structure derived from unsupervised clustering on the news video corpus and event entities from news video and online news articles. We then dynamically project userpsilas browsing history onto this topic hierarchy to provide the basis for re-ranking relevant news videos. This system is tested on one month of TRECVID 2006 dataset consisting of 80 hours news video and found to return results in a more intuitive and personalized manner.

Automatic detection of commercials in digital multimedia material is a challenging task with many applications. This paper presents a novel approach to fast commercial detection based on audio retrieval. It is based on the idea of segmenting energy envelope of audio into units, using only audio signal for matching on a commercial database. Fast searching and matching can be performed with high accuracy, by searching and by novel similarity function based on units. Experimental results show that 96.8% recall rate and 98.7% precision rate can be achieved under 0.125 real-time.

A novel statistical framework for replay detection is presented in this paper. Unlike current methods, the proposed framework exploits both inherent characters and transition relations of replay and non-replay scenes based on annotation of the video, which realizes segments and classifies video stream into replay and non-replay shots simultaneously. After annotation, the detected replay segment is further verified and its boundaries are adjusted to get more accurate replay segment considering probability distribution of lengths of replay and non-replay shots. Experimental results on soccer video are promising, demonstrating the effectiveness of the proposed framework.

Cylinder panoramic videos are a 360deg representation in horizontal of a certain scene. The users can navigate interactively through the scene and change their view angels. Because of large field of view, panoramic videos are often high-resolution and consume a significant amount of bandwidth for transmission. To resolve the problem, tile-based panoramic video coding and transmission is applied in some systems. With tile-based panoramic video coding and transmission, the panorama is divided into tiles and only the tiles involved with perspective view are transmitted and decoded. In the tile-based video coding, the selection of tile size is a problem. Different tile sizes will bring different quality of perspective view with the same bandwidth. In this paper, we proposed a method to select the most efficient tile size. With this method, the most efficient tile size could be fast selected and users can build the best quality perspective view from cylinder panoramic videos with the given bandwidth.

The massive amount of multimedia information especially video available on the Web requires a more precise and interactive retrieval. Current operational video retrieval systems do not make use of the implicit visual features but rely only on textual metadata supplied by the user during uploading. This greatly affects the retrieval performance as the metadata may not be comprehensive or consistent. In this paper, we describe the use of a spatio-temporal visual map (STVM) model to supplement Web video retrieval. This is done by employing the spatio-temporal visual similarity to rerank the text-retrieval results and find new results. Experimental results on a dynamic Web video corpus show significant improvement based on STVM model, with good usability scores based on human users.

Existing video research incorporates the use of relevance feedback based on user-dependent interpretations to improve the retrieval results. In this paper, we segregate the process of relevance feedback into 2 distinct facets: (a) recall-directed feedback; and (b) precision-directed feedback. The recall-directed facet employs general features such as text and high level features (HLFs) to maximize efficiency and recall during feedback, making it very suitable for large corpuses. The precision-directed facet on the other hand uses many other multimodal features in an active learning environment for improved accuracy. Combined with a performance-based adaptive sampling strategy, this process continuously re-ranks a subset of instances as the user annotates. Experiments done using TRECVID 2006 dataset show that our approach is efficient and effective.

We present a novel translation model based on tree-to-string alignment template (TAT) which describes the alignment between a source parse tree and a target string. A TAT is capable of generating both terminals and non-terminals and performing reordering at both low and high levels. The model is linguistically syntax-based because TATs are extracted automatically from word-aligned, source side parsed parallel texts. To translate a source sentence, we first employ a parser to produce a source parse tree and then apply TATs to transform the tree into a target string. Our experiments show that the TAT-based model significantly outperforms Pharaoh, a state-of-the-art decoder for phrase-based models.

We propose a novel reordering model for phrase-based statistical machine translation (SMT) that uses a maximum entropy (MaxEnt) model to predicate reorderings of neighbor blocks (phrase pairs).The model provides content-dependent, hierarchical phrasal reordering with generalization based on features automatically learned from a real-world bitext.We present an algorithm to extract all reordering events of neighbor blocks from bilingual data.In our experiments on Chineseto-English translation, this MaxEnt-based reordering model obtains significant improvements in BLEU score on the NIST MT-05 and IWSLT-04 tasks.

In this paper, a fast transform method is proposed to convert MPEG-2 8-tap discrete cosine transform (DCT) coefficients to H.264/AVC 4-tap integer transform coefficients directly in the transform domain. The proposed transform method saves 16 operations for each 8times8 DCT block by utilizing a novel transform kernel matrix and a fast computing method for multiplication of this new matrix. The simulation results show that the proposed method causes only a very little quality degradation, which is completely negligible in practice with the maximum value lower than 8times10-33dB, as compared with Jun Xin' s method. Hence, it can be efficiently used in the transform-domain MPEG-2 to H.264 transcoding

For the rate control of H. 264/AVC, one of the most important things is to get the statistics of the current frame accurately. To achieve this, a novel adaptive coding characteristics prediction scheme is presented to improve the accuracy of R-D modeling, by exploiting spatio-temporal correlations. With the proposed prediction scheme, we present a novel rate function and a linear distortion model, and then deduce a simple close-form solution to the problem of optimum bit allocation, just in a TMN-8-alike way. Extensive experiments show that improvements with gains up to 0.92dB per frame over JVT-G012, the current standardized rate control scheme, are achieved by the proposed scheme for a variety of test sequences with less demanding bandwidth.

We present a framework for word alignment based on log-linear models.All knowledge sources are treated as feature functions, which depend on the source langauge sentence, the target language sentence and possible additional variables.Log-linear models allow statistical alignment models to be easily extended by incorporating syntactic information.In this paper, we use IBM Model 3 alignment probabilities, POS correspondence, and bilingual dictionary coverage as features.Our experiments show that log-linear models significantly outperform IBM translation models.

The new video coding standard, H.264 allows motion estimation performing on tree-structured block partitioning and multiple reference frames. This feature improves the prediction accuracy significantly, but the cost of which is the complexity and computation load of video coding increase drastically. The reference software of H.264 adopts a full search scheme and its complexity increases linearly with specified the number of reference frames and the number of modes respectively. In this paper, we propose a fast mode decision algorithm in H.264 which eliminates searching unnecessary modes. The proposed algorithm takes full use of available information obtained from the previous searching process, e.g., macroblock's best mode in searching the previous reference frames. In the algorithm implementation, multiple half-stop conditions have been set in motion estimation so as to decrease encoder's complexity. Simulation results show that this proposed algorithm can effectively reduce complexity of inter-frame encoding, and the quality degradation is tiny compared with full search scheme. Furthermore, our algorithm is adaptive to variant test sequences, no need to set specific experimental threshold.

A new algorithm under multi-constraints for automatic segmentation of video moving objects is proposed. First, the temporal segmentation separates the initial areas including moving objects accurately from the background by continuous frame difference. Then, the spatial segmentation segments the initial areas into spatially consistent regions by the watershed algorithm based on a color gradient. Finally, regions are classified as foreground/background by maximizing the a posterior probability (MAP) of the MRF with spatial, temporal and adjacent constraints. Experimental results demonstrate that the algorithm is not sensitive to objects' irregular movement and illumination, and it can extract moving video objects accurately.

We propose a fast intra-prediction mode selection method for JVT video coding standard H.264. Based on local edge information obtained by calculating edge feature parameters, the method can reduce the computational complexity considerably while maintaining similar PSNR and bit rate. Experimental results are presented to show the effectiveness of the proposed method.

In this paper, an efficient block size mode selection algorithm for the variable-sizes block-matching (VSBM) in the MPEG-2 to H.264 transcoding is presented. Depending on leveraging the available motion information carried by the MPEG-2 bit-streams, the proposed algorithm is used to determine which one of the 16x16, 16x8, 8x16, and 8x8 block size modes should be used for each macroblock (MB). The simulation results show that the performance of the proposed algorithm is close to that of a cascaded pixel-domain transcoder (CPDT) when all the seven block size modes are enabled and the exhaustively full search method is used to determine the best block size modes. The whole transcoding time can be efficiently reduced by 22% on the average while the bit rate is slightly increased (2.9%).

There is no good solution of data management for CSCW applications in synchronous work mode. The choice now is no management or duplicate. In the former case, it is common that phone tool discards audio data that is worth to store. In the latter, the performance is still not high, especially when there involves large amount of multimedia data. We take advantages of main memory database technology to support synchronous cooperative work. After analyzes the cooperative design process based an cooperative concept design instance, the paper provides a main memory-based distributed data management model. The model schedules memory according to the features of synchronous cooperative application, greatly improving the performance of data access.

In many CSCW systems, mobile agents are used to do some important work. However, mobile agents add a notable amount of complexity to all common problems of distributed systems, and cause a couple of new ones, such as location management, frequent disconnection and migration, etc. In this article, we focus on a technique for strong migration of mobile agents. Capturing and re-establishing the agent's state is the most important problem in strong migration of mobile agents. We describe a way to save and restore the state of a running Java-based agent. We achieve this at the language level, without modifying the Java Virtual Machine, by instructing the Java source programs with a preprocessor which is used in a mobile agent scenario to offer strong migration for mobile agents based on Java. The automatically inserted code saves the runtime information when the program requests state saving and re-establishes the program's runtime state on restart.

Consistency control is an important research field in CSCW. We present an improved algorithm of concurrency control based on operation transformation. Our fastAdOPT algorithm improves other algorithms in two ways: first, we improve the transformation efficiency by saving the original form of operations in a request log and by the special process of "intimate preceding" operations. Secondly, it can realize a natural interactive mode by not using the undo/redo operation.