Federated Learning (FL) enables collaborative learning across distributed clients while preserving data privacy. However, FL faces significant challenges when dealing with heterogeneous data distributions, which can lead to suboptimal global models that fail to generalize across diverse clients. In this work, we propose a novel framework designed to tackle these challenges by introducing a dual-adapter approach. The method utilizes a larger local adapter for client-specific personalization and a smaller global adapter to facilitate efficient knowledge sharing across clients. Additionally, we incorporate a pruning mechanism to reduce communication overhead by selectively removing less impactful parameters from the local adapter. Through extensive experiments on a range of vision and language tasks, our method demonstrates superior performance compared to existing approaches. It achieves higher test accuracy, lower performance variance among clients, and improved worst-case performance, all while significantly reducing communication and computation costs. Overall, the proposed method addresses the critical trade-off between model personalization and generalization, offering a scalable solution for real-world FL applications.

In federated learning for multimodal models, balancing global generalization and local personalization remains challenging due to heterogeneous client data distributions. To address this, we propose a novel federated learning framework employing dual Low-Rank Adaptation (LoRA) adapters within a frozen Contrastive Language-Image Pre-training (CLIP) backbone. Each client maintains both global and local adapters, dynamically orchestrated via a lightweight gating network that adaptively fuses the adapters based on input-specific features. Unlike traditional parameter averaging, our server aggregates client knowledge through federated distillation on a compact reference dataset, effectively mitigating parameter conflicts across clients. Experiments demonstrate that our adaptive fusion strategy significantly improves personalized representation quality, outperforming standard LoRA-based federated approaches, especially in scenarios with diverse local data distributions.

Safety is a critical component of autonomous systems and remains a challenge for learning-based policies to be utilized in the real world. In particular, policies learned using reinforcement learning often fail to generalize to novel environments due to unsafe behavior. In this paper, we propose Sim-to-Lab-to-Real to bridge the reality gap with a probabilistically guaranteed safety-aware policy distribution. To improve safety, we apply a dual policy setup where a performance policy is trained using the cumulative task reward and a backup (safety) policy is trained by solving the Safety Bellman Equation based on Hamilton-Jacobi (HJ) reachability analysis. In Sim-to-Lab transfer, we apply a supervisory control scheme to shield unsafe actions during exploration; in Lab-to-Real transfer, we leverage the Probably Approximately Correct (PAC)-Bayes framework to provide lower bounds on the expected performance and safety of policies in unseen environments. Additionally, inheriting from the HJ reachability analysis, the bound accounts for the expectation over the worst-case safety in each environment. We empirically study the proposed framework for ego-vision navigation in two types of indoor environments with varying degrees of photorealism. We also demonstrate strong generalization performance through hardware experiments in real indoor spaces with a quadrupedal robot. See https://sites.google.com/princeton.edu/sim-to-lab-to-real for supplementary material.

We study the class of reach-avoid dynamic games in which multiple agents interact noncooperatively, and each wishes to satisfy a distinct target criterion while avoiding a failure criterion. Reach-avoid games are commonly used to express safety-critical optimal control problems found in mobile robot motion planning. Here, we focus on finding time-consistent solutions, in which future motion plans remain optimal even when a robot diverges from the plan early on due to, e.g., intrinsic dynamic uncertainty or extrinsic environment disturbances. Our main contribution is a computationally-efficient algorithm for multi-agent reach-avoid games which renders time-consistent solutions for all players. We demonstrate our approach in two- and three-player simulated driving scenarios, in which our method provides safe control strategies for all agents.

This paper aims to develop a wearable tactile sensor for measuring skin vibrations using a polyvinylidene fluoride (PVDF) film, which is a polymer piezo material. The sensor is worn on the finger pad where is remote from contact fingertip and detects skin-propagated vibrations when fingertip touches an object. The proposed sensor allows users to touch with bare fingers and to conduct active touch. A transfer function from vibrations applied on the fingertip to the sensor output is expressed by using a finger model, a sensor model, and an electric model of the PVDF film. On the basis of the transfer function, frequency response of the sensor is measured and estimation of vibrations is tested. Furthermore, the sensor output is investigated for three materials with different textures. Results show the validity and availability of the sensor.