We present a multi-task learning approach to jointly estimate the means of multiple independent data sets. The proposed multi-task averaging (MTA) algorithm results in a convex combination of the single-task averages. We derive the optimal amount of regularization, and show that it can be effectively estimated. Simulations and real data experiments demonstrate that MTA both maximum likelihood and James-Stein estimators, and that our approach to estimating the amount of regularization rivals cross-validation in performance but is more computationally efficient.

We show how to compute lower bounds for the supremum Bayes error if the class-conditional distributions must satisfy moment constraints, where the supremum is with respect to the unknown class-conditional distributions. Our approach makes use of Curto and Fialkow's solutions for the truncated moment problem. The lower bound shows that the popular Gaussian assumption is not robust in this regard. We also construct an upper bound for the supremum Bayes error by constraining the decision boundary to be linear.

Although the Dirichlet distribution is widely used, the independence structure of its components limits its accuracy as a model. The proposed shadow Dirichlet distribution manipulates the support in order to model probability mass functions (pmfs) with dependencies or constraints that often arise in real world problems, such as regularized pmfs, monotonic pmfs, and pmfs with bounded variation. We describe some properties of this new class of distributions, provide maximum entropy constructions, give an expectation-maximization method for estimating the mean parameter, and illustrate with real data.

To characterize the differences between two positive functions or two distributions, a class of distortion functions has recently been defined termed the functional Bregman divergences. The class generalizes the standard Bregman divergence defined for vectors, and includes total squared difference and relative entropy. Recently a key property was discovered for the vector Bregman divergence: that the mean minimizes the average Bregman divergence for a finite set of vectors. In this paper the analog result is proven: that the mean function minimizes the average Bregman divergence for a set of positive functions that can be parameterized by a finite number of parameters. In addition, the relationship of the functional Bregman divergence to the vector Bregman divergence and pointwise Bregman divergence is stated, as well as some important properties.

A class of distortions termed functional Bregman divergences is defined, which includes squared error and relative entropy. A functional Bregman divergence acts on functions or distributions, and generalizes the standard Bregman divergence for vectors and a previous pointwise Bregman divergence that was defined for functions. A recent result showed that the mean minimizes the expected Bregman divergence. The new functional definition enables the extension of this result to the continuous case to show that the mean minimizes the expected functional Bregman divergence over a set of functions or distributions. It is shown how this theorem applies to the Bayesian estimation of distributions. Estimation of the uniform distribution from independent and identically drawn samples is presented as a case study.

In addition to the stable fixed point which should be achieved under steady-state conditions, the sampling phase-locked loop (SPLL) implemented with a loop filter has another stable attractor in which an unwanted periodic solution, called false lock, develops in the loop. After the acquisition process, the SPLL either reaches the desired fixed point or gets into false lock, depending on the initial conditions. In every implemented circuit, the development of false lock has to be prevented. Although the false lock problem was reported earlier, an exact model to describe the behavior of SPLL in false lock has not yet been published. This paper propose a phenomenological model to explain why the false lock can develop and to describe the operation of SPLL in false lock.

Double sampling phase-locked loop (2-SPLL) is used in frequency synthesis. The unique feature of single sampling PLL is that its phase detector (PD) has a very low unwanted periodic output in steady state. In 2-SPLL, a second sample and hold circuit (S&H) is connected in cascade to suppress further the unwanted periodic PD output. However, the absence of a proper baseband model and design equations prevent the exploitation of all benefits of 2-SPLL. This paper develops a nonlinear baseband model for the 2-SPLL. Then based on the linearized baseband model, analysis is performed in the frequency domain and the frequency responses required to determine the spectrum of the output signal as a function of reference and VCO noise and modulation inputs are developed. Finally, the theoretical results are verified by measurements.