Gaussian processes (GPs) stand as crucial tools in machine learning and signal processing, with their effectiveness hinging on kernel design and hyperparameter optimization. This article presents a novel GP linear multiple kernel (LMK) and a generic sparsity-aware distributed learning framework to optimize the hyperparameters. The newly proposed grid spectral mixture product (GSMP) kernel is tailored for multidimensional data, effectively reducing the number of hyperparameters while maintaining good approximation capability. We further demonstrate that the associated hyperparameter optimization of this kernel yields sparse solutions. To exploit the inherent sparsity of the solutions, we introduce the sparse linear multiple kernel learning (SLIM-KL) framework. The framework incorporates a quantized alternating direction method of multipliers (ADMMs) scheme for collaborative learning among multiple agents, where the local optimization problem is solved using a distributed successive convex approximation (DSCA) algorithm. SLIM-KL effectively manages large-scale hyperparameter optimization for the proposed kernel, simultaneously ensuring data privacy and minimizing communication costs. The theoretical analysis establishes convergence guarantees for the learning framework, while experiments on diverse datasets demonstrate the superior prediction performance and efficiency of our proposed methods.

Dictionary learning (DL) is a pivotal task in machine learning and signal processing, involving extracting representative features from a given dataset. However, conventional DL models are known to be highly sensitive to outliers. To circumvent this issue, we introduce a new and robust DL model based on unbalanced optimal transport (UOT). Compared to DL models based on conventional robust distances and the Wasserstein distance, our model not only captures and leverages the structural information within the data but also demonstrates strong resilience to outliers. By employing the structure of the proposed robust DL model, we develop a novel hybrid block coordinate descent (BCD) algorithm. The proposed algorithm maintains computational tractability by exploiting special block structures of the subproblems. In addition, we establish the convergence of our algorithm without the Lipschitz smooth condition. Through extensive experimentation, we validate our theoretical results and demonstrate the effectiveness of the proposed method on synthetic data, MNIST data, Olivetti faces dataset, and hyperspectral images (HSIs) datasets.

This article considers the Wasserstein barycentric coordinates problem for Gaussian distributions which is the inverse problem of the Wasserstein barycenter problem. These coordinates take into account the underlying geometry of the measure space of Gaussian distributions and are thus meaningful for applications such as diffusion analysis and distributed information fusion. When the probability supports are discrete and identical, the theory of Wasserstein barycentric coordinates is well developed. However, for general probability distributions, the computation of Wasserstein barycentric coordinates is intractable since the technical hurdles involve solving a non-convex and non-concave optimization problem. For Gaussian distributions, we derive the closed-form expression of the derivatives for the objective function and propose a projected gradient descent method to solve the problem. Finally, we illustrate its application in diffusion tensor image (DTI) denoising including simulated DTI with different noise levels and DTI of the human brain.

This paper addresses the localization challenge in cooperative multi-agent wireless sensor networks, specifically focusing on range-based localization. To enhance robustness against outliers in range measurements, we employ the Huber function, leading to the formulation of a robust yet nonconvex optimization problem with coupled agent variables. Confronted with this nonconvex optimization challenge, particularly in largescale networks, we reformulate the problem using Lagrange duality and conjugate theory. This restructuring yields subproblems characterized by smooth strong convexity for dual variables and a simplified form for primal variables, thereby facilitating an efficient solution. Building upon this reformulation, we introduce a novel distributed primal-dual algorithm that employs coordinate descent and proximal minimization techniques within an iterative framework. This approach furnishes closed-form solutions for both primal and dual variables. Theoretically, our method ensures not only the convergence of the sequence of objective function values but also, by leveraging the KurdykaŁojasiewicz property, we establish the guaranteed global convergence of the location estimates sequence to a critical point of the original objective function. Notably, our proposed approach exhibits lower computational complexity, communication cost, and storage space compared to existing methods. Numerical experiments underscore the superiority of the proposed method in terms of robustness and localization accuracy when compared to the other methods in the literature.

As a novel distributed learning paradigm, federated learning (FL) faces serious challenges in dealing with massive clients with heterogeneous data distribution and computation and communication resources. Various client-variance-reduction schemes and client sampling strategies have been respectively introduced to improve the robustness of FL. Among others, primal-dual algorithms such as the alternating direction of method multipliers (ADMM) have been found being resilient to data distribution and outperform most of the primal-only FL algorithms. However, the reason behind remains a mystery still. In this paper, we firstly reveal the fact that the federated ADMM is essentially a client-variance-reduced algorithm. While this explains the inherent robustness of federated ADMM, the vanilla version of it lacks the ability to be adaptive to the degree of client heterogeneity. Besides, the global model at the server under client sampling is biased which slows down the practical convergence. To go beyond ADMM, we propose a novel primal-dual FL algorithm, termed FedVRA, that allows one to adaptively control the variance-reduction level and biasness of the global model. In addition, FedVRA unifies several representative FL algorithms in the sense that they are either special instances of FedVRA or are close to it. Extensions of FedVRA to semi/un-supervised learning are also presented. Experiments based on (semi-)supervised image classification tasks demonstrate superiority of FedVRA over the existing schemes in learning scenarios with massive heterogeneous clients and client sampling.

This work investigates unsupervised domain adaptation (UDA)-based semantic segmentation of very high-resolution (VHR) remote sensing (RS) images from different domains. Most existing UDA methods resort to generative adversarial networks (GANs) to cope with the domain shift problem caused by the discrepancies across different domains. However, these GAN-based UDA methods directly align two domains in the appearance, latent, or output space based on convolutional neural networks (CNNs), making them ineffective in exploiting long-range dependencies across the high-level feature maps derived from different domains. Unfortunately, such high-level features play an essential role in characterizing RS images with complex content. To circumvent this obstacle, a mutually boosted attention transformer (MBATrans) is proposed to capture cross-domain dependencies of semantic feature representations in this work. Compared with conventional UDA methods, MBATrans can significantly reduce domain discrepancies by capturing transferable features using global attention. More specifically, MBATrans utilizes a novel mutually boosted attention (MBA) module to align cross-domain feature maps while enhancing domain-general features. Furthermore, a novel GAN-based network with improved discriminative capability is devised by integrating an additional discriminator to learn domain-specific features. Extensive experiments on two large-scale VHR RS datasets, namely, International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen, confirm the superior performance of the proposed MBATrans-augmented GAN (MBATA-GAN) architecture. The source code in this work is available athttps://github.com/sstary/SSRS.

Network topology inference is a fundamental problem in many applications of network science, such as locating the source of fake news and brain connectivity network detection. Many real-world situations suffer from a critical problem in which only a limited number of observed nodes are available. In this work, the problem of network topology inference under the framework of partial observability is considered. Based on the vector autoregression model, we propose a novel unbiased estimator for symmetric network topology with Gaussian noise and the Laplacian combination rule. Theoretically, we prove that this estimator converges in probability to the network combination matrix. Furthermore, by utilizing the Gaussian mixture model algorithm, an effective algorithm called the network inference Gauss algorithm is developed to infer the network structure. Finally, compared with state-of-the-art methods, numerical experiments demonstrate that better performance is obtained in the case of small sample sizes when using the proposed algorithm.