We propose Plausible Token Amplification (PTA) to improve the accuracy of Differentially Private In-Context Learning (DP-ICL) using DP synthetic demonstrations. While Tang et al. empirically improved the accuracy of DP-ICL by limiting vocabulary space during DP synthetic demonstration generation, its theoretical basis remains unexplored. By interpreting ICL as implicit Bayesian inference on a concept underlying demonstrations, we not only provide theoretical evidence supporting Tang et al.’s empirical method but also introduce PTA, a refined method for modifying next-token probability distribution. Through the modification, PTA highlights tokens that distinctly represent the ground-truth concept underlying the original demonstrations. As a result, generated DP synthetic demonstrations guide the Large Language Model to successfully infer the ground-truth concept, which improves the accuracy of DP-ICL. Experimental evaluations on both synthetic and real-world text-classification datasets validated the effectiveness of PTA.

Federated learning (FL) is a promising technology for achieving privacy-preserving distributed learning. Although most existing studies on FL have provided a single global model for all clients, the resulting single model cannot handle heterogeneous local environments. Therefore, we propose a novel FL framework called fog-enabled federated learning with mutual knowledge distillation (F2MKD), in which the server and each client are ascribed to distinctive models. The remarkable feature of F2MKD is the data collection at the fog servers while ensuring data confidentiality among the fog servers. This study paves the new way for client-friendly, collaborative distributed learning. The code is available at https://github.com/d3-ai/F2MKD.

Federated learning (FL), a machine learning technique that preserves privacy by aggregating models from each device without exchanging personal data, has gained significant interest. This paper aims to establish an efficient and practical asynchronous FL method for decentralized systems. In asynchronous decentralized FL, the order of learning and aggregation is arbitrary. First, we explore the impact of this ordering on FL. Then, we propose to utilize the version information regarding model updates. Our strategy is to aggregate only the updated models after the previous round to improve the quality of the device’s model by avoiding the reaggregation of older models. Moreover, we also design a new practical framework for asynchronous decentralized FL by extending Flower framework. Our framework realizes effective communication ability by leveraging gRPC communication and thus can be applied to practical systems without the central server. Our evaluation shows the effectiveness of our methods that aggregate only the updated models from other devices. In addition, we show the impact of ordering on learning and aggregation according to situations.

This paper describes the development of a human-like multi-modal soft finger and its ability to sense the texture of objects. This fingertip has two silicon rubber layers of different hardness; strain gauges and PVDF films are randomly distributed as tactile sensors. Owing to the dynamics of the silicon between sensors, the fingertip is supposed to have several sensor modalities. Preliminary experiments show that the fingertip can detect the difference between the textures of objects (paper and wood).