Hypothesis testing has emerged as a significant research area due to its applications in various domains. In many real-world scenarios, we can only obtain training samples of both hypotheses instead of the underlying distributions, and the decision rule shall be restricted to the form of testing sample statistics due to the computational requirement of highdimensional data. In this paper, we study the fixed-length hypothesis testing problem under such constraints. By applying the information-geometric method, we provide the corresponding asymptotic optimal error exponent with respect to the sequence lengths, and propose a valid decision rule. Moreover, we present a geometric interpretation of the trade-off between the sampling processes of training and testing.

Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, existing works typically use all available samples from sources in training, which constrains their training efficiency and may lead to suboptimal results. To address this, we propose a theoretical framework that answers the question: what is the optimal quantity of source samples needed from each source task to jointly train the target model? Specifically, we introduce a generalization error measure based on K-L divergence, and minimize it based on high-dimensional statistical analysis to determine the optimal transfer quantity for each source task. Additionally, we develop an architecture-agnostic and data-efficient algorithm OTQMS to implement our theoretical results for target model training in multi-source transfer learning. Experimental studies on diverse architectures and two real-world benchmark datasets show that our proposed algorithm significantly outperforms state-of-the-art approaches in both accuracy and data efficiency. The code is available at https://github.com/zqy0126/OTQMS.

Estimating unknown parameters in restricted parameter space is an important problem with applications in communication, statistics, and machine learning. In this paper, we adopt the conventional minimax formulation to investigate such problems. In particular, we focus on studying the second-order characterizations of the minimax risk in the asymptotic regime. We first show that the second-order convergence rate of the minimax risk depends on the local flatness of the Fisher information around its global optimum. Then, we demonstrate that the second-order terms can be computed by solving certain ordinary differential equations, where the coefficients of the second-order terms can be explicitly expressed in some cases. Finally, the estimators achieving the minimax risk are also given, which provides potential guidance for machine learning designs.

The Hirschfeld-Gebelein-Renyi (HGR) maximal correlation shows widespread applications in statistics and machine learning fields. This paper explores the HGR maximal correlation among two discrete time random processes that form Markov chains with infinite chain lengths. Under the specific form of Gaussian random variables, the optimal correlation functions are linear to the data. Therefore, this problem can be reduced to solving the largest singular value of a particular matrix. Then, we present the analytical expression of the asymptotic HGR maximal correlation, where a geometric interpretation is also provided. This study offers insights into the effective design of feature extraction in machine learning tasks.

In this paper, we investigate the parameter estimation problems, where the unknown parameter is assumed to be within certain restricted parameter space. To this end, we adopt the conventional minimax formulation and apply the weighted mean-squared error as the loss function. In particular, we focus on analyzing the minimax risk in the asymptotic regime, where the second-order convergence rate, and the ordinary differential equation for computing the second-order terms are presented. Moreover, our results are applied to some widely considered probability distribution models, where the analytical expressions of the second-order terms and the corresponding estimators are explicitly provided. Finally, some numerical simulations are also presented when the second-order term cannot be explicitly expressed, which further supports our theoretical results.

In many federated learning scenarios, the distributed nodes represent and exchange information in the form of functions or statistics of data, and the computation and communication are often restricted by the dimensionality of the functions. In this paper, we explore the collaborative distributed parameter estimation under such constraints. Specifically, we assume that each node can observe a sequence of i.i.d. sampled data and communicate some statistics of the observed data with dimensionality constraints. We characterize the Cramer-Rao lower bound (CRLB) and construct the asymptotic efficient estimator that achieves CRLB. In addition, we provide the information geometric interpretation of the CRLB as projecting the score function onto the functional subspaces spanned by the distributed nodes. Finally, we present the neural estimator to compute the optimal statistics that the nodes shall transmit to each other for continuous variables.

Multimodal learning is an increasingly important research topic. Exploiting conditional dependency across multiple modalities has been shown useful for estimating of the multimodal joint distribution, especially when the number of training samples is insufficient. However, it is difficult to theoretically characterize such conditional dependency structure. To address this issue, we establish a mathematical framework and formulate the estimation problem based on the minimax principle. Then, we propose an estimator which is close to the analytical solution of the problem under a mild assumption on the sample size. Moreover, the proposed estimator is a linear combination of the learning results from the true dependency structure and the conditional one. The combining coefficient is related to three aspects: the number of training samples, the fitness of the conditional dependency structure, and the cardinality of each modality. Finally, numerical simulations are provided to verify our theoretical results that the proposed estimator is close to the optimal solution of the formulated problem.