The goal of the Facebook recommendation engine is to compare and rank heterogeneous types of content in order to find the most relevant recommendations based on user preference and page context. The challenges for such a recommendation engine include several aspects: 1) the online queries being processed are at very large scale; 2) with new content types and new user-generated content constantly added to the system, the candidate object set and underlying data distribution change rapidly; 3) different types of content usually have very distinct characteristics, which makes generic feature engineering difficult; and 4) unlike a search engine that can capture intention of users based on their search queries, our recommendation engine needs to focus more on users' profile and interests, past behaviors and current actions in order to infer their cognitive states. In this presentation, we would like to introduce an effective, scalable, online machine learning framework we developed in order to address the aforementioned challenges. We also want to discuss the insights, approaches and experiences we have accumulated during our research and development process.

Consensus clustering is the problem of reconciling clustering information about the same data set coming from different sources or from different runs of the same algorithm. Cast as an optimization problem, consensus clustering is known as median partition, and has been shown to be NP-complete. A number of heuristics have been proposed as approximate solutions, some with performance guarantees. In practice, the problem is apparently easy to approximate, but guidance is necessary as to which heuristic to use depending on the number of elements and clusterings given. We have implemented a number of heuristics for the consensus clustering problem, and here we compare their performance, independent of data size, in terms of efficacy and efficiency, on both simulated and real data sets. We find that based on the underlying algorithms and their behavior in practice the heuristics can be categorized into two distinct groups, with ramification as to which one to use in a given situation, and that a hybrid solution is the best bet in general. We have also developed a refined consensus clustering heuristic for the occasions when the given clusterings may be too disparate, and their consensus may not be representative of any one of them, and we show that in practice the refined consensus clusterings can be much superior to the general consensus clustering.