In this paper, we introduce a large scale online multi-task deep learning framework for modeling multiple feed ads auction prediction tasks on an industry-scale feed ads recommendation platform. Multiple prediction tasks are combined into one single model which is continuously trained on real time new ads data. Multi-tasking ads auction models in real-time faces many real-world challenges. For example, each task may be trained on different set of training data; the labels of different tasks may have different arrival time due to label delay; different tasks will interact with each other; combining the losses of each task is non-trivial. We tackle these challenges using practical and novel techniques such as multi-stage training for handling label delay, Multi-gate Mixture-of-Experts (MMoE) to optimize model interaction and an auto-parameter learning algorithm to optimize the loss weights of different tasks. We demonstrate that our proposed techniques can lead to quality improvements and substantial resource saving compared to modeling each single task independently.

In this paper, we analyze a collaborative filter that answers the simple question: What is popular amongst your “friends”? While this basic principle seems to be prevalent in many practical implementations, there does not appear to be much theoretical analysis of its performance. In this paper, we partly fill this gap. While recent works on this topic, such as the low-rank matrix completion literature, consider the probability of error in recovering the entire rating matrix, we consider probability of an error in an individual recommendation [bit error rate (BER)]. For a mathematical model introduced by Aditya et al. in 2009 and 2011, we identify three regimes of operation for our algorithm (named Popularity Amongst Friends) in the limit as the matrix size grows to infinity. In a regime characterized by large number of samples and small degrees of freedom (defined precisely for the model in the paper), the asymptotic BER is zero; in a regime characterized by large number of samples and large degrees of freedom, the asymptotic BER is bounded away from 0 and 1/2 (and is identified exactly except for a special case); and in a regime characterized by a small number of samples, the algorithm fails. We then compare these results with the performance of the optimal recommender. We also present numerical results for the MovieLens and Netflix datasets. We discuss the empirical performance in light of our theoretical results and compare with an approach based on low-rank matrix completion.

Motivated by applications such as recommendation systems, we consider the estimation of a binary random field X obtained by unknown row and column permutations of a block constant random matrix. The estimation of X is based on observations Y, which are obtained by passing entries of X through a binary symmetric channel (BSC) (representing noisy user behavior) and an erasure channel (representing missing data). We analyze an estimation algorithm based on local popularity. We study the bit error rate (BER) in the limit as the matrix size approaches infinity and the erasure rate approaches unity at a specified rate. Our main result identifies three regimes characterized by the cluster size and erasure rate. In one regime, the algorithm has asymptotically zero BER, in another regime the BER is bounded away from 0 and 1/2, while in the remaining regime, the algorithm fails and BER approaches 1/2. Numerical results for the Movielens dataset and comparison with earlier work is also given.

Consider a multiple input multiple output (MIMO) system with pulse amplitude modulation (PAM), where each transmit antenna is allowed to choose a different transmission delay. (We refer to this system as a MIMO system with asynchronous PAM.) Such MIMO systems have been studied before for increasing diversity and avoiding unintentional beamforming. We quantify the capacity/outage capacity improvement due to asynchronism and the corresponding spectral leakage (if any) for MIMO point-to-point and MIMO broadcast channels. We show that under commonly used transmitter spectral mask constraints, the channel has unbounded degrees of freedom. Synchronous PAM exploits only a finite number of these degrees of freedom, and asynchronism helps to improve capacity. For most cases of interest, two distinct shifts attain most of the capacity improvement. These gains are robust to interference from adjacent carriers. Analysis of the spectrum of the asynchronous PAM suggests that optimal water-filling based schemes are useful for improving capacity only in directional links due to possible spectral leakage, while uniform power allocation leads to capacity improvement without any extra spectral leakage.