Web-scale search systems typically tackle the scalability challenge with a two-step paradigm: retrieval and ranking. The retrieval step, also known as candidate selection, often involves extracting entities, creating an inverted index, and performing term matching for retrieval. Such traditional methods require manual and time-consuming development of retrieval models. In this paper, we propose a framework for constructing a graph that integrates human knowledge with user activity data analysis. The learned links are utilized for retrieval purposes. The model is easy to explain, debug, and tune. The system implementation is straightforward and can directly leverage existing inverted index systems. We applied this retrieval framework to enhance the job search and recommendation systems on a large professional networking portal, resulting in significant performance improvements.

The goal of Jobs Marketplace at LinkedIn is to match members to promoted job postings such that both job posters' ROI is optimized (amount of money spent per job clicks and applications) and the members are presented with relevant jobs that they are interested in and qualified for. This is achieved via a first-price auction mechanism where each job provides a bid for the member that comes to the job recommendations page. This bid depends on the match of the member to the job, as well as the daily budget that remains for the job, and its capability to spend it via clicks (e.g. some jobs might have more demand and have it easier to spend their budgets via clicks than others). In such a scheme, budget pacing, i.e. the capability of a job to spend its daily budget evenly, or according to a preset plan, is extremely important towards efficient utilization of its budget via reaching a higher number of candidates, and obey a variety of spending plans optimizing for different events such as clicks and applications.

Current computer involvement in adolescent social networks (youth between the ages of 11 and 17) provides new opportunities to study group dynamics, interactions amongst peers, and individual preferences. Nevertheless, most of the research in this area focuses on efficiently retrieving information that is explicit in large social networks (e.g., properties of the graph structure), but not on how to use the dynamics of the virtual social network to discover latent characteristics of the real-world social network. In this paper, we present the analysis of a game designed to take advantage of the familiarity of adolescents with online social networks, and describe how the data generated by the game can be used to identify bullies in 5th grade classrooms. We present a probabilistic model of the game and using the in-game interactions of the players (i.e., content of chat messages) infer their social role within their classroom (either a bully or non-bully). The evaluation of our model is done by using previously collected data from psychological surveys on the same 5th grade population and by comparing the performance of the new model with off-the-shelf classifiers.

Traffic flow prediction is a basic function of Intelligent Transportation System. Due to the complexity of traffic phenomenon, most existing methods build complex models such as neural networks for traffic flow prediction. As a model may lose effect with time lapse, it is important to update the model on line. However, the high computational cost of maintaining a complex model puts great challenge for model updating. The high computation cost lies in two aspects: computation of complex model coefficients and huge amount training data for it. In this paper, we propose to use a nonparametric approach based on locally weighted learning to predict traffic flow. Our approach incrementally incorporates new data to the model and is computationally efficient, which makes it suitable for online model updating and predicting. In addition, we adopt wavelet analysis to extract the periodic characteristic of the traffic data, which is then used for the input of the prediction model instead of the raw traffic flow data. The primary experiments on real data demonstrate the effectiveness and efficiency of our approach.

Many text processing applications adopted the bag of words (BOW) model representation of documents, in which each document is represented as a vector of weighted terms or n-grams, and then the cosine distance between two vectors is used as the similarity measurement. Although the great success in information retrieval and text categorization, the conventional BOW model ignores the detailed local text information, i.e. the co-occurrence pattern of words at sentence or paragraph level. In this paper, we propose a novel approach to represent a document as a set of local tf-idf vectors, or what we called local word bags (LWB). By encapsulating local information distributed around a document into multiple LWBs, we can measure the similarity of two documents via the partial match of their corresponding local bags. To perform the matching efficiently, we introduce the local word bag kernel (LWB kernel), a variant of VG-Pyramid match kernel. The new kernel enables the discriminative machine learning methods like SVM to compute the partial matching between two sets of LWBs in linear time after an one time hierarchical clustering procedure over all local bags at the initialization stage. Experiments on real world datasets demonstrate the effectiveness of our new approach.

Outlier detecting is one of the most important data analysis technologies in data mining, which can be used to discover anomalous phenomena in huge dataset. Many literatures on spatial outlier detecting and time series outlier detecting have appeared, while the area of spatio-temporal outliers considering both spatial and temporal dimensions has still rarely been touched. Defining outliers in traditional dataset is more explicit because the data structure we need to focus on is very straightforward (e.g., a spatial point or a transaction record). However, it is much more difficult to give outlier a definite characterization in spatio-temporal lattice data, since there are so many data structures we can pay attention to. With the aim of detecting useful and meaningful outliers in climate dataset, we introduce a formalized way to define outliers in spatio-temporal lattice data, in which the importance of clarifying basic data structure (we call it basic element in our paper) is stressed. As a case study, we define two kinds of spatio-temporal outliers based on a global climate dataset, according to the three aspects we propose in defining an outlier. The introduction of basic element and the formulation of outlier definition process make it easier and clearer to define meaningful outliers. Thus outlier detecting in spatio-temporal lattice data will provide us with really interesting and useful knowledge.

Spatial sampling is a significant part of query processing in spatial database. In this paper, an adaptive data-driven sampling for selectivity estimation is proposed in spatial database. This technique presents an efficient sampling for spatial data, especially for two-dimensional line and polygon data, and it make the sample size fit the limit of time and memory, or a user-defined parameter. The data-driven sampling technique is compared with various techniques on different type of datasets in our experimental study, and it out outperforms the other techniques over a broad range of query workloads and datasets.