In information geometry, statistical models are considered as differentiable manifolds, where each probability distribution represents a unique point on the manifold. A Riemannian metric can be systematically obtained from a divergence function using Eguchi's theory (1992); the well-known Fisher-Rao metric is obtained from the Kullback-Leibler (KL) divergence. The geometric derivation of the classical Cramér-Rao Lower Bound (CRLB) by Amari and Nagaoka (2000) is based on this metric. In this paper, we study a Riemannian metric obtained by applying Eguchi's theory to the Basu-Harris-Hjort-Jones (BHHJ) divergence (1998) and derive a generalized Cramér-Rao bound using Amari-Nagaoka's approach. There are potential applications for this bound in robust estimation.

This paper generalizes the notion of sufficiency for estimation problems beyond maximum likelihood. In particular, we consider estimation problems based on Jones et al. and Basu et al. likelihood functions that are popular among distance-based robust inference methods. We first characterize the probability distributions that always have a fixed number of sufficient statistics (independent of sample size) with respect to these likelihood functions. These distributions are power-law extensions of the usual exponential family and contain Student distributions as a special case. We then extend the notion of minimal sufficient statistics and compute it for these power-law families. Finally, we establish a Rao-Blackwell-type theorem for finding the best estimators for a power-law family. This helps us establish Cramér-Rao-type lower bounds for power-law families.

We formulate a general framework for which Campbell’s source coding, Arikan’s guessing, Huleihel et al.’s memorryless guessing, Bunte and Lapidoth’s tasks partitioning problems are specific ones. We then use this framework to show an equivalence among these problems.

We propose a generalized notion of principle of sufficiency when the underlying inference method is not necessarily maximum likelihood. This notion is based on certain generalized likelihood functions that arise in robust inference problems. Particularly, in this paper, we consider the Basu et al. estimation [1]. We identify the specific form of the probability distributions that have a fixed number of sufficient statistics with respect to this estimation. These distributions are power-law in nature and Student distributions are a part of this family.

In [12], it was shown that a generalized maximum likelihood estimation problem on a (canonical) α-powerlaw model (M(α)-family) can be solved by solving a system of linear equations. This was due to an orthogonality relationship between the M(α)-family and a linear family with respect to the relative α-entropy (or the Iα-divergence). Relative α-entropy is a generalization of the usual relative entropy (or the Kullback-Leibler divergence). M(α)-family is a generalization of the usual exponential family. In this paper, we first generalize the M(α)-family including the multivariate, continuous case and show that the Student-t distributions fall in this family. We then extend the above stated result of [12] to the general M(α)-family. Finally we apply this result to the Student-t distribution and find generalized estimators for its parameters.

Information geometry describes a framework where probability densities can be viewed as differential geometry structures. This approach has shown that the geometry in the space of probability distributions that are parameterized by their covariance matrices is linked to the fundamental concepts of estimation theory. In particular, prior work proposes a Riemannian metric - the distance between the parameterized probability distributions - that is equivalent to the Fisher Information Matrix, and helpful in obtaining the deterministic Cramér-Rao lower bound (CRLB). Recent work in this framework has led to establishing links with several practical applications. However, classical CRLB is useful only for unbiased estimators and inaccurately predicts the mean square error in low signal-to-noise (SNR) scenarios. In this paper, we propose a general Riemannian metric that, at once, is used to obtain both Bayesian CRLB and deterministic CRLB along with their vector parameter extensions. We also extend our results to the Barankin bound, thereby enhancing their applicability to low SNR situations.

Motivated by a recent result by van Erven and Harremoës, we study a forward projection problem for the Rényi divergence on a particular α-convex set, termed α-linear family. The solution to this problem yields a parametric family of probability measures which turns out to be an extension of the exponential family, and it is termed α-exponential family. An orthogonality relationship between the α-exponential and α-linear families is first established and is then used to transform the reverse projection on an α-exponential family into a forward projection on an α-linear family. The full paper version of this work is available on the arXiv at http://arxiv.org/abs/1512.02515.

This paper studies forward and reverse projections for the Rényi divergence of order α E (0, ∞) on α-convex sets. The forward projection on such a set is motivated by some works of Tsallis et al. in statistical physics, and the reverse projection is motivated by robust statistics. In a recent work, van Erven and Harremoës proved a Pythagorean inequality for Rényi divergences on α-convex sets under the assumption that the forward projection exists. Continuing this study, a sufficient condition for the existence of a forward projection is proved for probability measures on a general alphabet. For αϵ(1, ∞), the proof relies on a new Apollonius theorem for the Hellinger divergence, and for α E (0, 1), the proof relies on the Banach- Alaoglu theorem from the functional analysis. Further projection results are then obtained in the finite alphabet setting. These include a projection theorem on a specific α-convex set, which is termed an α-linear family, generalizing a result by Csiszar to α ≠ 1. The solution to this problem yields a parametric family of probability measures, which turns out to be an extension of the exponential family, and it is termed an α-exponential family. An orthogonality relationship between the α-exponential and α-linear families is established, and it is used to turn the reverse projection on an α-exponential family into a forward projection on an α-linear family. This paper also proves a convergence result of an iterative procedure used to calculate the forward projection on an intersection of a finite number of α-linear families.