It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GP) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-products of some Gaussian vectors across distributed machines? Using information theoretic bounds, we obtain an optimal solution for the problem which is based on vector quantization. Two suboptimal and more practical schemes are also presented as substitutes for the vector quantization scheme. In particular, it is shown that the performance of one of the practical schemes which is called per-symbol quantization is very close to the optimal one. Schemes provided for the inner-product calculations are incorporated into our proposed distributed learning methods for GPs. Experimental results show that with spending few bits per symbol in our communication scheme, our proposed methods outperform previous zero rate distributed GP learning schemes such as Bayesian Committee Model (BCM) and Product of experts (PoE).

A central machine is interested in estimating the underlying structure of a sparse Gaussian Graphical Model (GGM) from a dataset distributed across multiple local machines. The local machines can communicate with the central machine through a wireless multiple access channel. In this paper, we are interested in designing effective strategies where reliable learning is feasible under power and bandwidth limitations. Two approaches are proposed: Signs and Uncoded methods. In the Signs method, the local machines quantize their data into binary vectors and an optimal channel coding scheme is used to reliably send the vectors to the central machine where the structure is learned from the received data. In the Uncoded method, data symbols are scaled and transmitted through the channel. The central machine uses the received noisy symbols to recover the structure. Theoretical results show that both methods can recover the structure with high probability for a large enough sample size. Experimental results indicate the superiority of the Signs method over the Uncoded method under several circumstances.

Semantic video analysis and automatic concept extraction play an important role in several applications; including content-based search engines, video indexing, and video summarization. As the Bayesian network is a powerful tool for learning complex patterns, a novel Bayesian network-based method is proposed for automatic event detection and summarization in soccer videos. The proposed method includes efficient algorithms for shot boundary detection, shot view classification, mid-level visual feature extraction, and construction of the related Bayesian network. The method contains of three main stages. In the first stage, the shot boundaries are detected. Using the hidden Markov model, the video is segmented into large and meaningful semantic units, called play-break sequences. In the next stage, several features are extracted from each of these units. Finally, in the last stage, in order to achieve high level semantic features (events and concepts), the Bayesian network is used. The basic part of the method is constructing the Bayesian network, for which the structure is estimated using the Chow-Liu tree. The joint distributions of random variables of the network are modeled by applying the Farlie-Gumbel-Morgenstern family of Copulas. The performance of the proposed method is evaluated on a dataset with about 9 h of soccer videos. The method is capable of detecting seven different events in soccer videos; namely, goal, card, goal attempt, corner, foul, offside, and nonhighlights. Experimental results show the effectiveness and robustness of the proposed method on detecting these events.