Autoregressive (AR) models represented by Transformer has attained the highest performance benchmarks in neural machine translation, which benefits from the powerful learning ability of the model with the attention mechanism. It differs from the way human translators translate a sentence, where prior knowledge plays a important role. Inspired by this, a knowledge distillation based translation prompt information fusion method is proposed to improve the AR model. It introduces two modules to improve the AR model: translation prompt information fusion and knowledge distillation. The main steps can be summarized as follows: Firstly, Training a non-autoregressive (NAR) model based on the bilingual corpus. Then, the translation prompt information generated by NAR are integrated into the AR model from two aspects. On the on hand, incorporating the translation produced by the NAR model into the decoder of the AR model. On the other hand, utilizing the output distribution of the NAR model to guide the output distribution of the AR model. Experimental results across several translation tasks with low-resource and rich-resource indicate the effectiveness of the proposed method.

Vertical federated learning is a distributed machine learning framework that participants share the joint users' ID space but differ in feature space. Logistic regression is one of the companies most widely used classification algorithms for it's simplicity, computationally efficient and interpretability. Many works focus on vertical federated logistic regression to train high-quality models collaboratively. Most existing works need a third-party coordinator to assist with intermediate data computing. However, finding an authoritative third party in the real world is nearly impossible. Some works do not need a trusted third-party coordinator but risk leaking original data from plaintext intermediate data. To solve these problems, we propose an efficient and secure vertical federated logistic regression framework without a third-party coordinator (ESVFLR) using homomorphic encryption. Our method can prevent leaking privacy information in the both training and inference phase. We evaluate the performance of ESVFLR on public datasets, and the experimental results show that it has a low computation and communication cost while maintaining a high level of accuracy.

To address the limited server coverage of traditional over-the-air federated learning (OA-FL), we propose a new OA-FL framework for MIMO-based cloud radio access network (Cloud-RAN), called MIMO Cloud-RAN OA-FL (MIMOCROF). The proposed MIMOCROF consists of three stages in each training round. The first stage of edge aggregation allows each access point (AP) to collect local updates from edge devices and construct an edge update using MIMO multiple access. In the second stage of global aggregation, the cloud server (CS) aggregates edge updates received from the APs to form a global update through a fronthaul network. In the third stage of model updating and broadcasting, the CS sends the updated global model parameters to the APs, and the latter then broadcast the parameters to their served devices. To effectively exploit inter-AP correlation, we model the global aggregation stage as a lossy distributed source coding (L-DSC) problem. Based on the rate-distortion theory, we further analyze the performance of the MIMOCROF framework. We formulate a communication-learning optimization problem to improve the system performance by considering the inter-AP correlation. To solve this problem, we develop an algorithm by using alternating optimization (AO) and majorization-minimization (MM). Furthermore, we propose a practical L-DSC that exploits inter-AP correlation. Numerical results show that the proposed practical L-DSC effectively utilizes inter-AP correlation and is superior to other baseline schemes in performance.

To address limited server coverage in traditional over-the-air federated learning (OA-FL), we introduce the framework of multiple input multiple output (MIMO) cloud radio access network (Cloud-RAN) OA-FL (MIMOCROF). This framework involves a two-step model aggregation method in each training round. Firstly, each base station (BS) aggregates the local updates from its served devices, resulting in an edge update. Secondly, the cloud server (CS) aggregates the edge updates from the BSs. By modeling the second step as a lossy distributed source coding (L-DSC) process, we analyze the performance of MIMOCROF from the perspective of rate-distortion theory, resulting in a unified communication-learning design approach. In the proposed design, we jointly optimize rate resources and beamforming vectors, thereby leveraging the correlation inherent in FL to achieve performance gains. Numerical results show that, by solving the optimization problem, MIMOCROF performs comparably to the error-free bound and significantly outperforms other benchmark schemes.

Deep reinforcement learning outperforms humans in a variety of complex decision-making tasks. However, training an effective policy requires millions of interactions between the agent and the target environment, making the learning process time consuming and resource intensive. Inspired by neural network compression, we propose a lightweight policy learning framework, FastAct, to accelerate policy training by speeding up Actor inference. Specifically, FastAct accelerates the Actor by dynamically using multiple compression techniques to quickly generate experience data for policy training. Compared to the traditional framework, the FastAct adds two modules, Compressor and Scheduler. The Compressor integrates multiple compression algorithms through the chain-of-responsibility design pattern and aims to maximize the inference speed of the Actors. To ensure the quality of the experience data, we develop the Scheduler to dynamically adjust the compression algorithm according to the statistical information of the Actor. In addition, FastAct applies the off-policy value estimator V-trace to correct the value function of the target policy. We implemented FastAct based on IMPALA and conducted extensive experiments on Atari. Compared with IMPALA, FastAct speeds up the Actor by 2.0x to 4.1x. And FastAct reduces training time by 29.8% to 40.3% while maintaining a similar result.

To achieve communication-efficient federated multi-task learning (FMTL), we propose an over-the-air FMTL (OA-FMTL) framework, where multiple learning tasks deployed on edge devices share a non-orthogonal fading channel under the coordination of an edge server (ES). To overcome the inter-task interference inherent in the non-orthogonal transmission among tasks, we design a novel transmission method called model sparsification and random compression (MSRC) as well as a reception method called modified turbo compressed sensing (M-Turbo-CS). More specifically, at each edge device, the local model updates of all tasks are first sparsified andrandomlycompressed with different random compression matrices for different tasks, before being superimposed and sent over the uplink channel. Then the ES constructes the model aggregations of all the tasks from the channel observation data through a modified version of the turbo compressed sensing (Turbo-CS) algorithm called M-Turbo-CS. We analyze the performance of the proposed OA-FMTL framework with MSRC and M-Turbo-CS. Based on the analysis, we formulate a communication-learning optimization problem to improve the system performance by adjusting the power allocation among the tasks at the edge devices. Numerical simulations show that our proposed OA-FMTL efficiently suppresses the inter-task interference to achieve a learning performance comparable to the inter-task interference free bound at a significantly reduced communication overhead. It is also shown that the proposed inter-task power allocation optimization algorithm further reduces the overall communication overhead by appropriately adjusting the power allocation among the tasks.

In this letter, we introduce over-the-air computation into the communication design of federated multi-task learning (FMTL), and propose an over-the-air federated multi-task learning (OA-FMTL) framework, where multiple learning tasks deployed on edge devices share a non-orthogonal fading channel under the coordination of an edge server (ES). Specifically, the model updates for all the tasks are transmitted and superimposed concurrently over a non-orthogonal uplink fading channel, and the model aggregations of all the tasks are reconstructed at the ES through a modified version of the turbo compressed sensing algorithm (Turbo-CS) that overcomes inter-task interference. Both convergence analysis and numerical results show that the OA-FMTL framework can significantly improve the system efficiency in terms of reducing the number of channel uses without causing substantial learning performance degradation.