Uncertainty estimation plays a key role in many practical areas such as simulation and parameter optimization. Gaussian process (GP) is one popular model that provides naturally well-calibrated uncertainty estimates. However, it is challenging to learn a global GP posterior under the federated learning (FL) framework. In FL, clients’ private data should not be shared, but merging local kernels directly leads to privacy leakage. Previous works that consider federated GPs avoid it and focus on the personalized setting. This sacrifices information from other clients that can be exploited to benefit generalization. We present Federated Bayesian Neural Regression (FedBNR) that learns a global federated GP while respecting clients’ privacy. We incorporate deep kernel learning and random features by defining a unifying random kernel (URK). URK enables a principled approach of learning a global posterior as if all client data is centralized. Experiments conducted on real world regression datasets show statistically significant improvements.

Direct alignment methods typically train large language models (LLMs) by contrasting the likelihoods of preferred and dispreferred responses. While effective at capturing relative preferences, these methods are widely observed to suppress the absolute likelihoods of example responses. As a result, aligned models can deviate from expected patterns, exhibiting reward‑hacking effect even without an explicit reward model. This fundamental limitation of contrastive alignment, termed likelihood underdetermination, motivates us to revisit direct preference optimization (DPO)—the seminal direct alignment method. Interestingly, we show that the DPO loss admits a principled decomposition. The reformulated loss not only extends naturally to a broader range of feedback types, but also unveils the root cause of likelihood underdetermination. Specifically, we identify that standard DPO implicitly oversimplifies a regularizer in the reformulated loss; restoring this full term effectively resolves the underdetermination. Building on these insights, we introduce PRoximalized PReference Optimization (PRO), a unified alignment method that accommodates diverse feedback types while eliminating likelihood underdetermination through an efficient approximation of the full regularizer. Empirical evaluations demonstrate the consistent superiority of PRO over existing methods across pairwise, binary and scalar feedback.

Federated Learning (FL) involves training a model over a dataset distributed among clients, with the constraint that each client’s dataset is localized and possibly heterogeneous. In FL, small and noisy datasets are common, highlighting the need for well-calibrated models that represent the uncertainty of predictions. The closest FL techniques to achieving such goals are the Bayesian FL methods which collect parameter samples from local posteriors, and aggregate them to approximate the global posterior. To improve scalability for larger models, one common Bayesian approach is to approximate the global predictive posterior by multiplying local predictive posteriors. In this work, we demonstrate that this method gives systematically overconfident predictions, and we remedy this by proposing β-Predictive Bayes, a Bayesian FL algorithm that interpolates between a mixture and product of the predictive posteriors, using a tunable parameter β. This parameter is tuned to improve the global ensemble’s calibration, before it is distilled to a single model. Our method is evaluated on a variety of regression and classification datasets to demonstrate its superiority in calibration to other baselines, even as data heterogeneity increases. Code available at https://github.com/hasanmohsin/betaPredBayesFL. Our paper's full version is at https://arxiv.org/abs/2312.09817.

Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.

Model-based offline reinforcement learning (RL) aims to find highly rewarding policy, by leveraging a previously collected static dataset and a dynamics model. While the dynamics model learned through reuse of the static dataset, its generalization ability hopefully promotes policy learning if properly utilized. To that end, several works propose to quantify the uncertainty of predicted dynamics, and explicitly apply it to penalize reward. However, as the dynamics and the reward are intrinsically different factors in context of MDP, characterizing the impact of dynamics uncertainty through reward penalty may incur unexpected tradeoff between model utilization and risk avoidance. In this work, we instead maintain a belief distribution over dynamics, and evaluate/optimize policy through biased sampling from the belief. The sampling procedure, biased towards pessimism, is derived based on an alternating Markov game formulation of offline RL. We formally show that the biased sampling naturally induces an updated dynamics belief with policy-dependent reweighting factor, termed Pessimism-Modulated Dynamics Belief. To improve policy, we devise an iterative regularized policy optimization algorithm for the game, with guarantee of monotonous improvement under certain condition. To make practical, we further devise an offline RL algorithm to approximately find the solution. Empirical results show that the proposed approach achieves state-of-the-art performance on a wide range of benchmark tasks.

In this paper, we strive to improve the throughput of heterogeneous cellular networks by exploiting the pre-cached files at user end to manage interference. We consider a transmission scheme, where network-coded multicast is employed to help cancel multi-user interference, and reconstructed interference cancellation (RIC) is used to help eliminate inter-cell interference. Because RIC is opportunistic, base station (BS) muting is used to coordinate the residual strong interference. Since user association affects residual interference and is coupled with BS muting while both are operated in a very different timescale from content caching, we jointly optimize user association and BS muting for a given caching policy to maximize the number of users simultaneously served by the transmission scheme. By transforming the formulated problem into a maximal independent set problem with constructed conflict graph, the global optimal solution is found with graph theory methods. By exploiting the topology feature of heterogeneous networks, we proceed to propose two low-complexity algorithms, respectively, implemented in a centralized and distributed manner, which are viable for large-scale networks. Simulation results show that the optimized transmission scheme achieves a remarkable performance gain over the existing schemes.

In this paper, we strive to improve the throughput of wireless networks by exploiting the pre-cached files at user end to manage interference. We employ network-coded multicast (NCM) to help cancel multi-user interference, and reconstructed interference cancellation (RIC) to help cancel inter-cell interference. Because inter-cell interference may not be thoroughly canceled by RIC, base station (BS) muting is used to coordinate the residual strong interference. We jointly optimize user association and BS muting to maximize the number of users simultaneously served by a system using NCM and RIC with any given caching policy. Owing to the complex relation among BS muting, user association, NCM, and RIC, the formulated problem is NP-hard. By transforming the problem into the well-known maximal independent set problem with constructed conflict graph, the global optimal solution can be found efficiently with graph theory methods. Simulation results show that the proposed transmission strategy achieves a remarkable performance gain.

Proactive caching in base station (BS) is a promising way to leverage the human-behavior related information to boost the system throughput and improve user experience with low cost. Yet existing caching policies are "blind" to users, i.e., the users are unaware about whether the files to be requested are locally cached at the BS around them, whose performance will degrade in mobile systems due to the randomness of user behavior. For example, when a user stays in a cell where the BS caches the file to be requested later, the user may not initiate the request. When the user sends the request, it may have entered another cell where the BS does not cache the requested file. In this paper, we introduce a simple idea of informing the users about what the BS has cached to make local caching efficient, which can be regarded as a form of recommendation. Since the probabilities that the users request the files are unknown, we resort to Q-learning to perceive the request probability and the statistics of random arrival and departure of mobile users. Then, a cache replacement policy is optimized. Simulation results demonstrate that the proposed policy with recommendation can provide higher long-term system reward than existing policies without recommendation.