Learning to rank arises in many data mining applications, ranging from web search engine, online advertising to recommendation system. In learning to rank, the performance of a ranking model is strongly affected by the number of labeled examples in the training set; on the other hand, obtaining labeled examples for training data is very expensive and time-consuming. This presents a great need for the active learning approaches to select most informative examples for ranking learning; however, in the literature there is still very limited work to address active learning for ranking. In this paper, we propose a general active learning framework, expected loss optimization (ELO), for ranking. The ELO framework is applicable to a wide range of ranking functions. Under this framework, we derive a novel algorithm, expected discounted cumulative gain (DCG) loss optimization (ELO-DCG), to select most informative examples. Then, we investigate both query and document level active learning for raking and propose a two-stage ELO-DCG algorithm which incorporate both query and document selection into active learning. Furthermore, we show that it is flexible for the algorithm to deal with the skewed grade distribution problem with the modification of the loss function. Extensive experiments on real-world web search data sets have demonstrated great potential and effectiveness of the proposed framework and algorithms.

In Web search and vertical search, recency ranking refers to retrieving and ranking documents by both relevance and freshness. As impoverished in-links and click information is the the biggest challenge for recency ranking, we advocate the use of Twitter data to address the challenge in this article. We propose a method to utilize Twitter TinyURL to detect fresh and high-quality documents, and leverage Twitter data to generate novel and effective features for ranking. The empirical experiments demonstrate that the proposed approach effectively improves a commercial search engine for both Web search ranking and tweet vertical ranking.

Mobile search is still in infancy compared with general purpose web search. With limited training data and weak relevance features, the ranking performance in mobile search is far from satisfactory. To address this problem, we propose to leverage the knowledge of Web search to enhance the ranking of mobile search. In this paper, we first develop an equivalent page conversion between web search and mobile search, then we design a few novel ranking features, generated from the click-through data in web search, for estimating the relevance of mobile search. Large scale evaluations demonstrate that the knowledge from web search is quite effective for boosting the relevance of ranking on mobile search.

Recency ranking refers to the ranking of web results by accounting for both relevance and freshness. This is particularly important for "recency sensitive" queries such as breaking news queries. In this study, we propose a set of novel click features to improve machine learned recency ranking. Rather than computing simple aggregate click through rates, we derive these features using the temporal click through data and query reformulation chains. One of the features that we use is click buzz that captures the spiking interest of a url for a query. We also propose time weighted click through rates which treat recent observations as being exponentially more important. The promotion of fresh content is typically determined by the query intent which can change dynamically over time. Quite often users query reformulations convey clues about the query's intent. Hence we enrich our click features by following query reformulations which typically benefit the first query in the chain of reformulations. Our experiments show these novel features can improve the NDCG5 of a major online search engine's ranking for "recency sensitive" queries by up to 1.57%. This is one of the very few studies that exploits temporal click through data and query reformulations for recency ranking.

One of the tasks in machine learning is to build a device that predicts each next input symbol of a sequence as it takes one input symbol from the sequence. We studied new approaches to this task. We suggest that deterministic finite automata (DFA) are good building blocks for this device, together with genetic algorithms (GAs), which let these automata "evolve" to predict each next input symbol of the sequence. Moreover, we study how to combine these highly fit automata so that a network of them would compensate for each others' weaknesses and predict better than any single automaton. We studied the simplest approaches to combine automata: building trees of automata with special-purpose automata, which may be called switchboards. These switchboard automata are located on the internal nodes of the tree, take an input symbol from the input sequence just as other automata do, and predict which subtree will make a correct prediction on each next input symbol. GAs again play a crucial role in searching for switchboard automata. We studied various ways of growing trees of automata and tested them on sample input sequences, mainly note pitches, note durations and up/down notes of Bach's Fugue IX. The test results show that DFAs together with GAs seem to be very effective for this type of pattern learning task.