This paper improves spatial pyramid kernel (SPK) and proposes a relevance learning approach to compare performer's poses in a large dance archive, the NRCD collection1. Domain knowledge of Choreutics is exploited to define pose topics and a selection operator is developed for pose topic matching. The visual structure descriptor of self similarity (SSF) is extended to hierarchical self similarity (HSSF) to keep shape context. The framework of Bag-of-Visual Words (BOVW) is applied to encode as well as to speed up the matching on pose topics/topic combinations. This alleviates the complexity in limb allocation which is infeasible in our data. Extensive experiments show that the new approach outperforms the original SPK in both precision and robustness.

We describe a system for matching human posture (pose) across a large cross-media archive of dance footage spanning nearly 100 years, comprising digitized photographs and videos of rehearsals and performances. This footage presents unique challenges due to its age, quality and diversity. We propose a forest-like pose representation combining visual structure (self-similarity) descriptors over multiple scales, without explicitly detecting limb positions which would be infeasible for our data. We explore two complementary multi-scale representations, applying passage retrieval and latent Dirichlet allocation (LDA) techniques inspired by the text retrieval domain, to the problem of pose matching. The result is a robust system capable of quickly searching large cross-media collections for similarity to a visually specified query pose. We evaluate over a cross-section of the UK National Research Centre for Dance's (UK-NRCD), and the Siobhan Davies Replay's (SDR) digital dance archives, using visual queries supplied by dance professionals. We demonstrate significant performance improvements over two base-lines: classical single and multi-scale bag of visual words (BoVW) and spatial pyramid kernel (SPK) matching .

A graphic representation is proposed to facilitate the analysis of temporal saliency in a video document. This representation visualizes temporal saliency at multiple resolutions and introduces efficient image operations to the computationally intensive task of saliency analysis. Algorithms for graphic-based saliency fusion and attended area detection are presented. We evaluate the effectiveness by the application of general highlight detection in football videos. The experimental collection includes six full games from FIFA World Cup and European Champion.

Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.

This paper proposes an aggregative query generation which exploits a media document representation called feature term to create a query from multiple media examples, e.g. images. A feature term denotes an interval of one media feature dimension, such as a bin in colour histogram. This approach (1) can easily accumulate features from multiple query examples to generate an efficient query; (2) enables the exploration of text-based retrieval models for multimedia retrieval. Two criteria, minimised chi2and maximised entropy, are proposed to optimise feature term selection. Two ranking functions, KL divergence and tf-idf based BM25 model, are used for relevance estimation. Experiments on the Corel photo collection demonstrate the effectiveness of feature terms.