With the diversified sources satisfying users' needs, many online service platforms collect information from multiple sources in order to provide a set of useful information to the users. However, existing recommendation systems are mostly designed for single-source data, and thus fail to recommend multi-source review packages since the interplay between the reviews of different sources is not properly modeled. In fact, modeling the interplay between different sources is challenging because 1) two reviews may conflict with each other, 2) different users have different preferences on review sources, and 3) users' preferences to each source may change under different scenarios. To address these challenges, we propose Self-Attentive Recommendation for multi-source review Package (SARP), for predicting how useful the user feels to the package, while simultaneously reflecting how much the user is affected by each review. Specifically, SARP jointly considers the relationships of every user, purpose, and review source to learn better latent representations. A self-attention module is further used for integrating source representations and the review ratings, following a multi-layer perceptron (MLP) for the prediction tasks. Experimental results on the self-constructed dataset and public dataset demonstrate that the proposed model outperforms the state-of-the-art approaches.

Existing approaches for egocentric activity recognition mainly rely on a single modality (e.g., detecting interacting objects) to infer the activity category. However, due to the inconsistency between camera angle and subject's visual field, important objects may be partially occluded or missing in the video frames. Moreover, where the objects are and how we interact with the objects are usually ignored in prior works. To resolve these difficulties, we propose multiple mid-level representations (e.g., objects manipulated by a user, background context, and motion patterns of hands) to compensate the insufficiency of a single modality, and jointly consider what, where, and how a subject is interacting with. To evaluate the method, we introduce a new and challenging egocentric activity dataset (ADL+) that contains video and wrist-worn accelerometer data of people performing daily-life activities. Our approach significantly outperforms the state-of-the-art method on the public ADL dataset (i.e., 36.8% to 46.7%) and our ADL+ dataset (i.e., 32.1 % to 60.0%) in terms of classification accuracy. In addition, we also conduct a series of analyses to explore relative merits of each modality to egocentric activity recognition.

With the popularity of social media nowadays, tons of photos are uploaded everyday. To understand the image content, image classification becomes a very essential technique for plenty of applications (e.g., object detection, image caption generation). Convolutional Neural Network (CNN) has been shown as the state-of-the-art approach for image classification. However, one of the characteristics in social media photos is that they are often applied with photo filters, especially on Instagram. We find that prior works do not aware of this trend in social media photos and fail on filtered images. Thus, we propose a novel CNN architecture that utilizes the power of pairwise constraint by combining Siamese network and the proposed adaptive margin contrastive loss with our discriminative pair sampling method to solve the problem of filter bias. To the best of our knowledge, this is the first work to tackle filter bias on CNN and achieve state-of-the-art performance on a filtered subset of ILSVRC2012.

The advent of touch panels in mobile devices has provided a good platform for mobile sketch search. However, most of the previous sketch image retrieval systems usually adopt an inverted index structure on large-scale image database, which is formidable to be operated in the limited memory of mobile devices. In this paper, we propose a novel approach to address these challenges. First, we effectively utilize distance transform (DT) features to bridge the gap between query sketches and natural images. Then these high-dimensional DT features are further projected to more compact binary hash bits. The experimental results show that our method achieves very competitive retrieval performance with MindFinder approach [3] but only requires much less memory storage (e.g., our method only requires 3% of total memory storage of MindFinder in 2.1 million images). Due to its low consumption of memory, the whole system can independently operate on the mobile devices.