Machine learning offers a fantastically powerful toolkit for building useful complexprediction systems quickly. This paper argues it is dangerous to think ofthese quick wins as coming for free. Using the software engineering frameworkof technical debt, we find it is common to incur massive ongoing maintenancecosts in real-world ML systems. We explore several ML-specific risk factors toaccount for in system design. These include boundary erosion, entanglement,hidden feedback loops, undeclared consumers, data dependencies, configurationissues, changes in the external world, and a variety of system-level anti-patterns.

Localized query prediction (LQP) is the task of estimating web query trends for a specific location. This problem subsumes many interesting personalized web applications such as personalization for buzz query detection, for query expansion, and for query recommendation. These personalized applications can greatly enhance user interaction with web search engines by providing more customized information discovered from user input (i.e., queries), but the LQP task has rarely been investigated in the literature. Although exist abundant work on estimating global web search trends does exist, it often encounters the big challenge of data sparsity when personalization comes into play. In this article, we tackle the LQP task by proposing a series of collaborative language models (CLMs). CLMs alleviate the data sparsity issue by collaboratively collecting queries and trend information from the other locations. The traditional statistical language models assume a fixed background language model, which loses the taste of personalization. In contrast, CLMs are personalized language models with flexible background language models customized to various locations. The most sophisticated CLM enables the collaboration to adapt to specific query topics, which further advances the personalization level. An extensive set of experiments have been conducted on a large-scale web query log to demonstrate the effectiveness of the proposed models.

In this paper, we present our approach for geographic personalization of a content recommendation system. More specifically, our work focuses on recommending query topics to users. We do this by mining the search query logs to detect trending local topics. For a set of queries we compute their counts and what we call buzz scores, which is a metric for detecting trending behavior. We also compute the entropy of the geographic distribution of the queries as means of detecting their location affinity. We cluster the queries into trending topics and assign the topics to their corresponding location. Human editors then select a subset of these local topics and enter them into a recommendation system. In turn the recommendation system optimizes a pool of trending local and global topics by exploiting user feedback. We present some editorial evaluation of the technique and results of a live experiment. Inclusion of local topics in selected locations into the global pool of topics resulted in more than 6% relative increase in user engagement with the recommendation system compared to using the global topics exclusively.

Web search providers often include search services for domain-specific subcollections, called verticals, such as news, images, videos, job postings, company summaries, and artist profiles. We address the problem of vertical selection, predicting relevant verticals (if any) for queries issued to the search engine's main web search page. In contrast to prior query classification and resource selection tasks, vertical selection is associated with unique resources that can inform the classification decision. We focus on three sources of evidence: (1) the query string, from which features are derived independent of external resources, (2) logs of queries previously issued directly to the vertical, and (3) corpora representative of vertical content. We focus on 18 different verticals, which differ in terms of semantics, media type, size, and level of query traffic. We compare our method to prior work in federated search and retrieval effectiveness prediction. An in-depth error analysis reveals unique challenges across different verticals and provides insight into vertical selection for future work.

The internal competition between categories in the adaptive resonance theory (ART) neural model can be biased by replacing the original choice function by one that contains an attentional tuning parameter under external control. For the same input but different values of the attentional tuning parameter, the network can learn and recall different categories with different degrees of generality, thus permitting the coexistence of both general and specific categorizations of the same set of data. Any number of these categorizations can be learned within one and the same network by virtue of generalization and discrimination properties. A simple model in which the attentional tuning parameter and the vigilance parameter of ART are linked together is described. The self-stabilization property is shown to be preserved for an arbitrary sequence of analog inputs, and for arbitrary orderings of arbitrarily chosen vigilance levels.

A new learning law, the Direct Coding Rule, is proposed for bottom-up long term memory learning in Adaptive Resonance Theory (ART) networks. This law requires less computational precision than the traditional Weber Law Rule and modifies the search dynamics of the network to accelerate convergence. Following a brief mathematical analysis of the new learning law, an ART1 network based on this law is applied to a passive radar detection problem. The simulation results allow comparison of the new law to the Weber Law Rule, with and without weight quantization, from the speed and cost viewpoints.>