This paper deals with batch normalization for SPDnet, a deep learning architecture specifically designed to handle covariance matrices. Current SPDnet batch normalization relies on geometric means, which require expensive iterative computations and lacks formal backpropagation derivations. These limitations are addressed through two contributions: (i) formal backpropagation derivation for SPDnet batch normalization, and (ii) computationally efficient closed-form alternatives to the geometric mean: log-Euclidean, arithmetic and harmonic means, and the geometric mean of arithmetic and harmonic means. Simulation results on the batch normalization layer with different means demonstrate significant reductions in memory consumption and comparable training time by avoiding automatic differentiation, demonstrating the effectiveness of our approach. Numerical experiments on three real datasets show that simpler means can outperform the geometric mean with up to 5% accuracy improvements while reducing training time by a factor of 5.

In this paper, the issue of averaging data on a manifold is addressed. While the Fréchet mean resulting from Riemannian geometry appears ideal, it is unfortunately not always available and often computationally very expensive. To overcome this,$R$-barycenters have been proposed and successfully applied to Stiefel and Grassmann manifolds. However,$R$-barycenters still suffer severe limitations as they rely on iterative algorithms and complicated operators. We propose simpler, yet efficient, barycenters that we call$RL$-barycenters. We show that, in the setting relevant to most applications, our framework yields astonishingly simple barycenters: arithmetic means projected onto the manifold. We apply this approach to the Stiefel and Grassmann manifolds. On simulated data, our approach is competitive with respect to existing averaging methods, while computationally cheaper.

This paper considers the problem of obtaining a low-rank factorization of a given correlation matrix. In order to handle possible spurious correlation coefficients within the input, a robust formulation is proposed with a criterion based on the Huber loss function. Minimizing this fitting criterion under the low-rank correlation structure constraint is then addressed using the block majorization-minimization framework. Several algorithm options are explored and compared in terms of computational complexity. The merits of the proposed correlation fitting method are then validated on simulations, and for the process of dimension reduction of microarray data.

In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory based method that estimates Fréchet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods.

This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a Riemannian manifold (a smooth manifold endowed with a Riemannian metric) and follows a given prior distribution. In this setup, the chosen Riemannian metric induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure. Performance bounds for such error measure were previously obtained in the non-Bayesian case (when the unknown parameter is assumed to deterministic), and referred to as intrinsic Cramér-Rao bound. The presented result then appears either as: a) an extension of the intrinsic Cramér-Rao bound to the Bayesian estimation framework; b) a generalization of the Van-Trees inequality (Bayesian Cramér-Rao bound) that accounts for the aforementioned geometric structures. In a second part, we leverage this formalism to study the problem of covariance matrix estimation when the data follow a Gaussian distribution, and whose covariance matrix is drawn from an inverse Wishart distribution. Performance bounds for this problem are obtained for both the mean squared error (Euclidean metric) and the natural Riemannian distance for Hermitian positive definite matrices (affine invariant metric). Numerical simulation illustrate that assessing the error with the affine invariant metric is revealing of interesting properties of the maximum a posteriori and minimum mean square error estimator, which are not observed when using the Euclidean metric.

This work deals with elliptical Wishart distributions on the set of symmetric positive definite matrices. It contains two major contributions. First, the information geometry associated with elliptical Wishart distributions is derived. Second, this geometry is leveraged to propose Riemannian-optimization-based maximum likelihood estimators of any elliptical Wishart distribution. Particular attention is given to two specific distributions: the t- and Kotz Wishart ones. The performance of the proposed methods is assessed through numerical experiments on simulated data.

This paper presents a new classification framework for both first and second order statistics, i.e. mean/location and covariance matrix. In the last decade, several covariance matrix classification algorithms have been proposed. They often leverage the Riemannian geometry of symmetric positive definite matrices (SPD) with its affine invariant metric and have shown strong performance in many applications. However, their underlying statistical model assumes a zero mean hypothesis. In practice, it is often estimated and then removed in a preprocessing step. This is of course damaging for applications where the mean is a discriminative feature. Unfortunately, the distance associated to the affine invariant metric for both mean and covariance matrix remains unknown. Leveraging previous works on geodesic triangles, we propose two affine invariant divergences that use both statistics. Then, we derive an algorithm to compute the associated Riemannian centers of mass. Finally, a divergence based Nearest centroid, applied on the crop classification dataset Breizhcrops, shows the interest of the proposed framework.

We consider the problem of jointly estimating the location and scatter matrix of a Compound Gaussian distribution with unknown deterministic texture parameters. When the location is known, the Maximum Likelihood Estimator (MLE) of the scatter matrix corresponds to Tyler’s M-estimator, which can be computed using fixed point iterations. However, when the location is unknown, the joint estimation problem remains challenging since the associated standard fixed-point procedure to evaluate the solution may often diverge. In this paper, we propose a stable algorithm based on Riemannian optimization for this problem. Finally, numerical simulations show the good performance and usefulness of the proposed algorithm.

This paper aims at providing an original Riemannian geometry to derive robust covariance matrix estimators in spiked models (i.e. when the covariance matrix has a low-rank plus identity structure). The considered geometry is the one induced by the product of the Stiefel manifold and the manifold of Hermitian positive definite matrices, quotiented by the unitary group. One of the main contributions is to consider a Riemannian metric related to the Fisher information metric of elliptical distributions, leading to new representations for the tangent spaces and a new retraction. A new robust covariance matrix estimator is then obtained as the minimizer of Tyler's cost function, redefined directly on the set of low-rank plus identity matrices, and computed with the aforementioned tools. The main interest of this approach is that it appears well suited to the cases where the sample size is lower than the dimension, as illustrated by numerical experiments.

We consider the optimal performance of blind separation of Gaussian sources. In practice, this estimation problem is solved by a two-step procedure: estimation of a set of covariance matrices from the observed data and approximate joint diagonalization of this set to find the unmixing matrix. Rather than studying the theoretical performance of a specific method, we are interested in the optimal attainable performance of any estimator. To do so, we consider the so-called intrinsic Cramér-Rao bound, which exploits the geometry of the parameters of the model. Unlike previous works developing a Cramér-Rao bound in this context, our solution does not require any additional hypotheses. To obtain our bound, we define and study a new Riemannian manifold holding the parameters of interest. An original estimation error measure is defined with the help of our Riemannian distance function. The corresponding Fisher information matrix is then obtained from the Fisher information metric and orthonormal bases on the tangent spaces of the manifold. Finally, our theoretical results are validated on simulated data.

Relying on recent advances in statistical estimation of covariance distances based on random matrix theory, this article proposes an improved covariance and precision matrix estimation for a wide family of metrics. The method is shown to largely outperform the sample covariance matrix estimate and to compete with state-of-the-art methods, while at the same time being computationally simpler and faster. Applications to linear and quadratic discriminant analyses also show significant gains, therefore suggesting practical interest to statistical machine learning.

Scatter matrix and its normalized counterpart, referred to as shape matrix, are key parameters in multivariate statistical signal processing, as they generalize the concept of covariance matrix in the widely used Complex Elliptically Symmetric distributions. Following the framework of [1], intrinsic Cramér-Rao bounds are derived for the problem of scatter and shape matrices estimation with samples following a Complex Elliptically Symmetric distribution. The Fisher Information Metric and its associated Riemannian distance (namely, CES-Fisher) on the manifold of Hermitian positive definite matrices are derived. Based on these results, intrinsic Cramér-Rao bounds on the considered problems are then expressed for three different distances (Euclidean, natural Riemannian, and CES-Fisher). These contributions are therefore a generalization of Theorems 4 and 5 of [1] to a wider class of distributions and metrics for both scatter and shape matrices.