Given an input query, a recommendation model is trained using user feedback data (e.g., click data) to output a ranked list of items. In real-world systems, besides accuracy, an important consideration for a new model is novelty of its top-k recommendations w.r.t. an existing deployed model. However, novelty of top-k items is a difficult goal to optimize a model for, since it involves a non-differentiable sorting operation on the model's predictions. Moreover, novel items, by definition, do not have any user feedback data. Given the semantic capabilities of large language models, we address these problems using a reinforcement learning (RL) formulation where large language models provide feedback for the novel items. However, given millions of candidate items, the sample complexity of a standard RL algorithm can be prohibitively high. To reduce sample complexity, we reduce the top-k list reward to a set of item-wise rewards and reformulate the state space to consist of tuples such that the action space is reduced to a binary decision; and show that this reformulation results in a significantly lower complexity when the number of items is large. We evaluate the proposed algorithm on improving novelty for a query-ad recommendation task on a large-scale search engine. Compared to supervised finetuning on recent pairs, the proposed RL-based algorithm leads to significant novelty gains with minimal loss in recall. We obtain similar results on the ORCAS query-webpage matching dataset and a product recommendation dataset based on Amazon reviews.

Since air pollution seriously affects human heath and daily life, the air quality prediction has attracted increasing attention and become an active and important research topic. In this paper, we present AccuAir, our winning solution to the KDD Cup 2018 of Fresh Air, where the proposed solution has won the 1st place in two tracks, and the 2nd place in the other one. Our solution got the best accuracy on average in all the evaluation days. The task is to accurately predict the air quality (as indicated by the concentration of PM2.5, PM10 or O3) of the next 48 hours for each monitoring station in Beijing and London. Aiming at a cutting-edge solution, we first presents an analysis of the air quality data, identifying the fundamental challenges, such as the long-term but suddenly changing air quality, and complex spatial-temporal correlations in different stations. To address the challenges, we carefully design both global and local air quality features, and develop three prediction models including LightGBM, Gated-DNN and Seq2Seq, each with novel ingredients developed for better solving the problem. Specifically, a spatial-temporal gate is proposed in our Gated-DNN model, to effectively capture the spatial-temporal correlations as well as temporal relatedness, making the prediction more sensitive to spatial and temporal signals. In addition, the Seq2Seq model is adapted in such a way that the encoder summarizes useful historical features while the decoder concatenate weather forecast as input, which significantly improves prediction accuracy. Assembling all these components together, the ensemble of three models outperforms all competing methods in terms of the prediction accuracy of 31 days average, 10 days average and 24-48 hours.

In this paper we share our solution to the Biomedia ACM MM Grand Challenge 2019, which focuses on the gastrointestinal tract with an aim to detect and classify abnormalities. We firstly identify the challenges in this task, including the scarce and imbalanced data, and the subtle inter-category variances. Based on these analysis, we propose a solution which leverages the 10-fold cross validation approach to alleviate the over-fitting problem, and design a model to adaptively ensemble all sub-models belonging to all component models. Based on extensive offline evaluations, we verify the performance of the proposed technique under various settings. In the competition, we eventually receive the MCC(Matthews correlation coefficient) score of 0.9480.

Automatic image annotation has been extensively studied in the recent decades. Nevertheless, existing methods usually assume a properly labeled training set, which greatly inhibits their application to real-world datasets with incomplete labels. Due to their lack of special treatments for noisy data, most existing methods simply consider the missing labels as strictly negative ones, leading to the degradation of tagging accuracy. In light of such challenges, we propose a novel model in this paper, called ranking-preserving low-rank factorization. Specifically, we construct a local training set for each test image, and conduct low-rank matrix factorization on the model coefficient matrix, to simultaneously capture the label dependency and reduce the model complexity. Furthermore, to alleviate the ambiguity introduced by missing labels, the prediction model is learnt via tag ranking regularized by sample similarities and tag correlations, and both regularization terms are incorporated into our factorization scheme. By assembling all the aforementioned components together, our method obviates the need for making binary decisions based on unreliable data, and thus is more robust towards missing labels. Extensive empirical evaluations conducted on four datasets demonstrate the effectiveness of the proposed method.

Recently, very deep two-stream ConvNets have achieved great discriminative power for video classification, which is especially the case for the temporal ConvNets when trained on multi-frame optical flow. However, action recognition in videos often fall prey to the wild camera motion, which poses challenges on the extraction of reliable optical flow for human body. In light of this, we propose a novel method to remove the global camera motion, which explicitly calculates a homography between two consecutive frames without human detection. Given the estimated homography due to camera motion, background motion can be canceled out from the warped optical flow. We take this a step further and design a new architecture called Saliency-Context two-stream ConvNets, where the context two-stream ConvNets are employed to recognize the entire scene in video frames, whilst the saliency streams are trained on salient human motion regions that are detected from the warped optical flow. Finally, the Saliency-Context two-stream ConvNets allow us to capture complementary information and achieve state-of-the-art performance on UCF101 dataset.

The recent decade has witnessed remarkable developments of SIFT-based approaches for image retrieval. However, such approaches are inherently insufficient in handling the semantic gap and large viewpoint changes, leading to inferior performance. To address these limitations, this paper extends SIFT-based match kernels by integrating the match functions for SIFT and CNN features. Specifically, a thresholded exponential match kernel for CNN features is proposed to calculate their semantic similarity of images. Thus, images having semantic similarity lower than the threshold will be filtered out, and the remaining images will be assigned semantic weights to adjust the similarity scores measured by SIFT. Enhanced discriminability is achieved in the proposed method by effectively encoding the complementary cues from SIFT and CNN features into the match kernels. Extensive evaluations on benchmark datasets demonstrate the effectiveness of our method, which outperforms existing methods by a considerable margin.

Many visual applications have benefited from the outburst of web images, yet the imprecise and incomplete tags arbitrarily provided by users, as the thorn of the rose, may hamper the performance of retrieval or indexing systems relying on such data. In this paper, we propose a novel locality sensitive low-rank model for image tag completion, which approximates the global nonlinear model with a collection of local linear models. To effectively infuse the idea of locality sensitivity, a simple and effective pre-processing module is designed to learn suitable representation for data partition, and a global consensus regularizer is introduced to mitigate the risk of overfitting. Meanwhile, low-rank matrix factorization is employed as local models, where the local geometry structures are preserved for the low-dimensional representation of both tags and samples. Extensive empirical evaluations conducted on three datasets demonstrate the effectiveness and efficiency of the proposed method, where our method outperforms pervious ones by a large margin.

In this paper, motivated by the superior performance of sparse representation based dictionary learning for application of image classification and the usage of nonlinearity property in improving performance of image representation, we propose a locality sensitive dictionary learning algorithm with global consistency and smoothness constraint to overcome the restriction of linearity at relatively low cost. Specifically, the image features are partitioned into several groups in a locality sensitive way and a global consistency regularizer is embedded into locality sensitive dictionary learning algorithm. The proposed algorithm is efficient to capture complex nonlinear structure. Experimental results on several benchmark data sets demonstrate the efficiency of our proposed locality sensitive dictionary learning algorithm.

A novel tag completion algorithm is proposed in this paper, which is designed with the following features: 1) Low-rank and error s-parsity: the incomplete initial tagging matrix D is decomposed into the complete tagging matrix A and a sparse error matrix E. However, instead of minimizing its nuclear norm, A is further factorized into a basis matrix U and a sparse coefficient matrix V, i.e. D = UV + E. This low-rank formulation encapsulating sparse coding enables our algorithm to recover latent structures from noisy initial data and avoid performing too much denoising; 2) Local reconstruction structure consistency: to steer the completion of D, the local linear reconstruction structures in feature space and tag space are obtained and preserved by U and V respectively. Such a scheme could alleviate the negative effect of distances measured by low-level features and incomplete tags. Thus, we can seek a balance between exploiting as much information and not being mislead to suboptimal performance. Experiments conducted on Corel5k dataset and the newly issued Flickr30Concepts dataset demonstrate the effectiveness and efficiency of the proposed method.

In this work, a novel method is proposed for recognizing human actions in still images, which incorporates both pose and context information. Poselet-based action classifiers are learned using Poselet Activation Vector as features, which contain pose information for each action. And context-based action classifiers for each action are learned on contextual information, which is obtained by sparse coding on foreground and background. The confidences of an image belonging to each action are obtained through summing up the probability outputs of the poselet-based and the context-based classifiers. The contribution of this work is three folded. Firstly, sparse coding is adopted to find compact patterns of the original features. Secondly, a block coordinate descent algorithm is proposed for sparse coding, which can be performed very fast in practice. Thirdly, both pose and context information are taken into consideration for action recognition. The experimental results show the proposed method achieves the state-of-the-art performance on several benchmarks.

In this paper, we propose a novel method that is capable of detecting and removing moving cast shadows robustly. Four properties of the moving cast shadows are observed. To make full use of the four properties, pixel level processing is adopted to extract the color, texture and luminance features of the moving foreground, and region level processing is introduced to detect shadows. Another innovation is that we also propose a method to check the reliability of the color feature that will be used only when it is reliable. Furthermore, by introducing the geometric center detection as a prejudge step, the efficiency of the method gets improved. The experimental results prove that the proposed method is effective and robust.