In this article, a novel method for personalized item recommendation based on social tagging is presented. The proposed approach comprises a content-based tag propagation method to address the sparsity and “cold start” problems, which often occur in social tagging systems and decrease the quality of recommendations. The proposed method exploits (a) the content of items and (b) users' tag assignments through a relevance feedback mechanism in order to automatically identify the optimal number of content-based and conceptually similar items. The relevance degrees between users, tags, and conceptually similar items are calculated in order to ensure accurate tag propagation and consequently to address the issue of “learning tag relevance.” Moreover, the ternary relation among users, tags, and items is preserved by performing tag propagation in the form of triplets based on users' personal preferences and “cold start” degree. The latent associations among users, tags, and items are revealed based on a tensor factorization model in order to build personalized item recommendations. In our experiments with real-world social data, we show the superiority of the proposed approach over other state-of-the-art methods, since several problems in social tagging systems are successfully tackled. Finally, we present the recommendation methodology in the multimodal engine of I-SEARCH, where users' interaction capabilities are demonstrated.

In this paper, a novel framework for rich-media object retrieval is described. The searchable items are media representations consisting of multiple modalities, such as 2-D images, 3-D objects and audio files, which share a common semantic concept. The proposed method utilizes the low-level descriptors of each separate modality to construct a new low-dimensional feature space, where all media objects can be mapped irrespective of their constituting modalities. While most of the existing state-of-the-art approaches support queries of one single modality at a time, the proposed one allows querying with multiple modalities simultaneously, through efficient multimodal query formulation, and retrieves multimodal results of any available type. Finally, a multimedia indexing scheme is adopted to tackle the problem of large scale media retrieval. The present framework proposes significant advances over existing methods and can be easily extended to involve as many heterogeneous modalities as possible. Experiments performed on two multimodal datasets demonstrate the effectiveness of the proposed method in multimodal search and retrieval.