In this paper, we present a mathematical and computational framework for comparing and matching distributions in reproducing kernel Hilbert spaces (RKHS). This framework, called optimal transport in RKHS, is a generalization of the optimal transport problem in input spaces to (potentially) infinite-dimensional feature spaces. We provide a computable formulation of Kantorovich's optimal transport in RKHS. In particular, we explore the case in which data distributions in RKHS are Gaussian, obtaining closed-form expressions of both the estimated Wasserstein distance and optimal transport map via kernel matrices. Based on these expressions, we generalize the Bures metric on covariance matrices to infinite-dimensional settings, providing a new metric between covariance operators. Moreover, we extend the correlation alignment problem to Hilbert spaces, giving a new strategy for matching distributions in RKHS. Empirically, we apply the derived formulas under the Gaussianity assumption to image classification and domain adaptation. In both tasks, our algorithms yield state-of-the-art performances, demonstrating the effectiveness and potential of our framework.

Sparse linear arrays, such as nested and co-prime arrays, are capable of resolving O(M2) sources using only O(M) sensors by exploiting their so-called difference coarray model. One popular approach to exploit the difference coarray model is to construct an augmented covariance matrix from the sample covariance matrix. By applying common direction-of-arrival (DOA) estimation algorithms to this augmented covariance matrix, more sources than the number of sensors can be identified. In this letter, inspired by the optimal transport theory, we develop a new approach to construct this augmented covariance matrix. We formulate a structured covariance estimation problem that minimizes the Bures-Wasserstein distance between the sample covariance matrix and the subsampled augmented covariance matrix, which can be either casted to a semi-definite programming problem, or directly solved using gradient-based methods. Our approach contributes to a new grid-less DOA estimation algorithm for sparse linear arrays. Numerical examples show that our approach achieves state-of-art estimation performance.

Domain shift, which occurs when there is a mismatch between the distributions of training (source) and testing (target) datasets, usually results in poor performance of the trained model on the target domain. Existing algorithms typically solve this issue by reducing the distribution discrepancy in the input spaces. However, for kernel-based learning machines, performance highly depends on the statistical properties of data in reproducing kernel Hilbert spaces (RKHS). Motivated by these considerations, we propose a novel strategy for matching distributions in RKHS, which is done by aligning the RKHS covariance matrices (descriptors) across domains. This strategy is a generalization of the correlation alignment problem in Euclidean spaces to (potentially) infinite-dimensional feature spaces. In this paper, we provide two alignment approaches, for both of which we obtain closed-form expressions via kernel matrices. Furthermore, our approaches are scalable to large datasets since they can naturally handle out-of-sample instances. We conduct extensive experiments (248 domain adaptation tasks) to evaluate our approaches. Experiment results show that our approaches outperform other state-of-the-art methods in both accuracy and computationally efficiency.

Graph-structured data arise in wide applications, such as computer vision, bioinformatics, and social networks. Quantifying similarities among graphs is a fundamental problem. In this paper, we develop a framework for computing graph kernels, based on return probabilities of random walks. The advantages of our proposed kernels are that they can effectively exploit various node attributes, while being scalable to large datasets. We conduct extensive graph classification experiments to evaluate our graph kernels. The experimental results show that our graph kernels significantly outperform other state-of-the-art approaches in both accuracy and computational efficiency.

Sparse linear arrays, such as co-prime and nested arrays, can identify up to O(M2) sources with only O(M) sensors by using co-array based MUSIC. We conduct analytical performance analysis of two coarray based MUSIC algorithms, namely the direct augmentation based MUSIC, and the spatial smoothing based MUSIC. In addition, we analyze the Cramér-Rao bound for sparse linear arrays, and show that for co-prime and nested arrays, it can decrease at a rate of O(M-5) as the number of sensors M goes to infinity, in contrast to O(M-3) in the ULA case. We use numerical examples to demonstrate our analytical results.

We investigate the problem of direction of arrival (DOA) estimation using sparse linear arrays, such as co-prime and nested arrays, in the case of missing data resulting from sensor failures. We introduce a signal model where sensor failures occur after taking certain number of snapshots. We formulate a structured covariance estimation problem by exploiting the special geometry of sparse linear arrays, which also provides enhanced degrees of freedom. Numerical examples show that, by utilizing the information in both complete measurements and incomplete measurements, our method achieves better estimation accuracy than the traditional method using only complete measurements.

In this paper, we provide a novel regression algorithm based on a Gaussian random field (GRF) indexed by a Riemannian manifold (M, g). We utilize both the labeled and unlabeled data sets to exploit the geometric structure of M. We use the recovered heat (H) kernel as the covariance function for the GRF (HGRF). We propose a Monte Carlo integral theorem on Riemannian manifolds and derive the corresponding convergence rate and approximation error. Based on this theorem, we correctly normalize the recovered eigenvector to make it compatible with Riemannian measure. More importantly, we prove that the HGRF is intrinsic to the original data manifold by comparing the pullback geometry and the original geoemtry of M. Essentially it is a semi-supervised learning method, which means the unlabeled data can be utilized to help identify the geometry structure of the unknown manifold M.