Hypothesis testing has emerged as a significant research area due to its applications in various domains. In many real-world scenarios, we can only obtain training samples of both hypotheses instead of the underlying distributions, and the decision rule shall be restricted to the form of testing sample statistics due to the computational requirement of highdimensional data. In this paper, we study the fixed-length hypothesis testing problem under such constraints. By applying the information-geometric method, we provide the corresponding asymptotic optimal error exponent with respect to the sequence lengths, and propose a valid decision rule. Moreover, we present a geometric interpretation of the trade-off between the sampling processes of training and testing.

Estimating unknown parameters in restricted parameter space is an important problem with applications in communication, statistics, and machine learning. In this paper, we adopt the conventional minimax formulation to investigate such problems. In particular, we focus on studying the second-order characterizations of the minimax risk in the asymptotic regime. We first show that the second-order convergence rate of the minimax risk depends on the local flatness of the Fisher information around its global optimum. Then, we demonstrate that the second-order terms can be computed by solving certain ordinary differential equations, where the coefficients of the second-order terms can be explicitly expressed in some cases. Finally, the estimators achieving the minimax risk are also given, which provides potential guidance for machine learning designs.

The Hirschfeld-Gebelein-Renyi (HGR) maximal correlation shows widespread applications in statistics and machine learning fields. This paper explores the HGR maximal correlation among two discrete time random processes that form Markov chains with infinite chain lengths. Under the specific form of Gaussian random variables, the optimal correlation functions are linear to the data. Therefore, this problem can be reduced to solving the largest singular value of a particular matrix. Then, we present the analytical expression of the asymptotic HGR maximal correlation, where a geometric interpretation is also provided. This study offers insights into the effective design of feature extraction in machine learning tasks.

In this paper, we investigate the parameter estimation problems, where the unknown parameter is assumed to be within certain restricted parameter space. To this end, we adopt the conventional minimax formulation and apply the weighted mean-squared error as the loss function. In particular, we focus on analyzing the minimax risk in the asymptotic regime, where the second-order convergence rate, and the ordinary differential equation for computing the second-order terms are presented. Moreover, our results are applied to some widely considered probability distribution models, where the analytical expressions of the second-order terms and the corresponding estimators are explicitly provided. Finally, some numerical simulations are also presented when the second-order term cannot be explicitly expressed, which further supports our theoretical results.

Dependency structures between modalities have been utilized explicitly and implicitly in multimodal learning to enhance classification performance, particularly when the training samples are insufficient. Recent efforts have con-centrated on developing mathematical frameworks utilizing conditional dependency structures, but the non-asymptotic relations between the training sample size and various structures are not sufficiently addressed. To address this issue, we propose a mathematical framework that can be utilized to characterize conditional dependency structures in analytic ways. It provides an explicit description of the sample size in learning various structures in a non-asymptotic regime. Additionally, it demonstrates how task complexity and a fitness evaluation of conditional dependence structures affect the results. Furthermore, we develop an autonomously updated coefficient algorithm auto-CODES based on the theoretical framework and conduct experiments on multimodal emotion recognition tasks using the MELD dataset. The experimental results validate our theory and show the effectiveness of the proposed algorithm.

Face swapping, the technique of transferring the identity from one face to another, merges as a field with significant practical applications. However, previous swapping methods often result in visible artifacts. To address this issue, in our paper, we propose CodeSwap, a symmetrical framework to achieve face swapping with high-fidelity and realism. Specifically, our method firstly utilizes a codebook that captures the knowledge of high quality facial features. Building on this foundation, the face swapping is then converted into the code manipulation task in a code space. To achieve this, we design a Transformer-based architecture to update each code independently, which enable more precise manipulations. Furthermore, we incorporate a mask generator to achieve seamless blending of the generated face with the background of target image. A distinctive characteristic of our method is its symmetrical approach to processing both target and source images, simultaneously extracting information from each to improve the quality of face swapping. This symmetry also simplifies the bidirectional exchange of faces in a singular operation. Through extensive experiments on ClelebA-HQ and FF++, our method is proven to not only achieve efficient identity transfer but also substantially reduce the visible artifacts.

Deep Joint Source-Channel Coding (JSCC) has gained increased attention, asserting its significance in the communication field. However, existing Deep JSCC techniques struggle to mitigate time-varying noise due to the deep neural networks being trained beforehand and fixed. To address this issue, we propose a robust deep JSCC scheme. Firstly, a multi-network parallel structure, as well as error-correcting codes, is introduced to effectively exploit label information. Secondly, a closed-form linear encoder and decoder pair is employed at the input and output ends of the channel to deal with the varying noise, which releases the neural network from dealing with a large range of varying noise levels. Thirdly, a transfer learning algorithm is utilized for estimating real-time noise statistics, which outperforms conventional estimation methods when noise statistics are time-dependent. These three components are effectively integrated as a comprehensive transmission system. Experimental results demonstrate that our optimized scheme outperforms existing approaches in the literature.

In many federated learning scenarios, the distributed nodes represent and exchange information in the form of functions or statistics of data, and the computation and communication are often restricted by the dimensionality of the functions. In this paper, we explore the collaborative distributed parameter estimation under such constraints. Specifically, we assume that each node can observe a sequence of i.i.d. sampled data and communicate some statistics of the observed data with dimensionality constraints. We characterize the Cramer-Rao lower bound (CRLB) and construct the asymptotic efficient estimator that achieves CRLB. In addition, we provide the information geometric interpretation of the CRLB as projecting the score function onto the functional subspaces spanned by the distributed nodes. Finally, we present the neural estimator to compute the optimal statistics that the nodes shall transmit to each other for continuous variables.

Distributed implementations of gradient-based algorithms have been essential for training large machine learning models on massive datasets. However, distributed learning algorithms are confronted with several challenges, including communication costs, straggler issues, and attacks from Byzantine adversaries. Existing works on attack-resilient distributed learning, e.g., the coordinate-wise median of gradients, usually neglect communication and/or straggler issues, and fail to defend against well-crafted attacks. Moreover, those methods are ineffective when more than half of workers are corrupted by a Byzantine adversary. To tackle those challenges simultaneously, we develop a robust gradient aggregation framework that is compatible with gradient compression and straggler mitigation techniques. Our proposed framework requires the parameter server to maintain an honest gradient as a reference at each iteration, thus can compute trust-score and similarity for each received gradient and tolerate arbitrary number of corrupted workers. We also provide convergence analysis of our method for non-convex optimization problems. Finally, experiments of image classification task on Fashion-MNIST dataset are conducted under various Byzantine attacks and gradient sparsification operations, and the numerical results demonstrate the effectiveness of our proposed strategy.

For signal detection tasks in wireless communications, most of the existing algorithms either ignore the co-channel interference or treat it as Gaussian noise, which may result in unsatisfactory accuracy when the interference is non-negligible and complex-distributed. When neural networks are motivated in the design of the detectors, difficulty arises in the training due to the fact that there are few accessible pilots in each packet. In this paper, we consider a data-driven detector based on multi-source transfer learning (MSTL) for signal detection in a fading channel with interference. The MSTL detector transfers channel knowledge of previous packets into the latest detection. In particular, we consider a linear combination of the pilots and historical symbols in the distribution space, and design the optimal combination coefficients based on the number of those symbols as well as distributions similarity. Numerical simulations on a Gauss-Markov flat Rayleigh fading channel with co-channel interference validate the advantages of our algorithms, compared with several existing training schemes including directly applying the fully connected deep neural network (FCDNN) and conventional linear minimum mean square error (LMMSE) detector.

In this paper, we study the information transmission problem under the distributed learning framework, where each worker node is merely permitted to transmit a m-dimensional statistic to improve learning results of the target node. Specifically, we evaluate the corresponding expected population risk (EPR) under the regime of large sample sizes. We prove that the performance can be enhanced since the transmitted statistics contribute to estimating the underlying distribution under the mean square error measured by the EPR norm matrix. Accordingly, the transmitted statistics correspond to the eigenvectors of this matrix, and the desired transmission allocates these eigenvectors among the statistics such that the EPR is minimal. Moreover, we provide the analytical solution of the desired statistics for single-node and two-node transmission, where a geometrical interpretation is given to explain the eigenvector selection. For the general case, an efficient algorithm that can output the allocation solution is developed based on the node partitions.

Current transfer learning algorithm designs mainly focus on the similarities between source and target tasks, while the impacts of the sample sizes of these tasks are often not sufficiently addressed. This paper proposes a mathematical framework for quantifying the transferability in multi-source transfer learning problems, with both the task similarities and the sample complexity of learning models taken into account. In particular, we consider the setup where the models learned from different tasks are linearly combined for learning the target task, and use the optimal combining coefficients to measure the transferability. Then, we demonstrate the analytical expression of this transferability measure, characterized by the sample sizes, model complexity, and the similarities between source and target tasks, which provides fundamental insights of the knowledge transferring mechanism and the guidance for algorithm designs. Furthermore, we apply our analyses for practical learning tasks, and establish a quantifiable transferability measure by exploiting a parameterized model. In addition, we develop an alternating iterative algorithm to implement our theoretical results for training deep neural networks in multi-source transfer learning tasks. Finally, experiments on image classification tasks show that our approach outperforms existing transfer learning algorithms in multi-source and few-shot scenarios.