Decentralized learning is widely employed for collaboratively training models using distributed data over wireless networks. Existing decentralized learning methods primarily focus on training single-modal networks. For the decentralized multi-modal learning (DMML), the modality heterogeneity and the non-independent and non-identically distributed (non-IID) data across devices make it difficult for the training model to capture the correlated features across different modalities. Moreover, modality competition can result in training imbalance among different modalities, which can significantly impact the performance of DMML. To improve the training performance in the presence of non-IID data and modality heterogeneity, we propose a novel DMML with knowledge distillation (DMMLKD) framework, which decomposes the extracted feature into the modality-common and the modality-specific components. In the proposed DMML-KD, a generator is applied to learn the global conditional distribution of the modality-common features, thereby guiding the modality-common features of different devices towards the same distribution. Meanwhile, we propose to decrease the number of local iterations for the modalities with fast training speed in DMML-KD to address the imbalanced training. We design a balance metric based on the parameter variation to evaluate the training speed of different modalities in DMML-KD. Using this metric, we optimize the number of local iterations for different modalities on each device under the constraint of remaining energy on devices. Experimental results demonstrate that the proposed DMML-KD with training balance can effectively improve the training performance of DMML.

As an edge intelligence algorithm for multi-device collaborative training, federated learning (FL) can protect data privacy but increase the computing load of wireless devices. In contrast, split learning (SL) can reduce the computing load of devices by model splitting and assignment. To take advantage of FL and SL, we propose a hybrid federated split learning (HFSL) framework for wireless networks in this paper, which combines the multi-worker collaborative training of FL and the flexible splitting of SL. To reduce the computational idleness in model splitting, we design a parallel computing scheme for model splitting without label sharing and conduct a theoretical analysis of the impact of the delayed gradient on the convergence. Aiming to obtain the trade-off between the training time and energy consumption, we model the joint optimization problem of splitting decisions, the bandwidth, and computing resources as a multi-objective problem. As such, we propose a predictive generative adversarial network (GAN)-powered multi-objective optimization algorithm to obtain the Pareto front of the problem, which utilizes the discriminator to guide the training of the generator to predict promising solutions. Experimental results demonstrate that the proposed algorithm outperforms the considered baselines in finding Pareto optimal solutions, and the solutions obtained from the proposed HFSL framework can dominate the solution of FL.

With the booming development of Internet of things (IoT) devices and machine learning (ML) technique, edge machine learning is emerging to process the enormous sampled data for realizing intelligent applications at the network edge. With limited edge resources, a well-structured neural network and numerous training data are the two main factors that affect the performance of edge machine learning. In this paper, we cooperatively optimize the data collection and the neural architecture to minimize the energy consumption of devices and the error on a specific task. We derive the Rademacher complexity bounds theoretically to evaluate the generalization error of the neural architectures in the search space and then formulate the optimization problem accordingly. Then we develop a scheme to solve the problem that dynamically performs the data collection based on policy gradient reinforcement learning and the parameter-sharing neural architecture search (NAS) algorithm. By this way, the transmission power of each device can be adjusted based on the data quality assessed by the NAS result in each round to effectively collect data. And with the growing high-quality data, the NAS algorithm can gradually find the optimal architecture for the task. Experimental results show that the neural architectures found by the proposed algorithm outperform the existing architectures while saving energy in the device.

Mobile services are becoming more diverse, making them have different demands on communications, computing, and caching (3C) resources in mobile systems. Unlike the traditional work that considers only one type of service, this paper designs a unified framework to characterize the different kinds of services, and jointly optimizes the 3C resources of the base station (BS) and mobile devices to provide differentiated quality of service (QoS) for diverse services. In the proposed framework, we model the task required by the mobile device to be generated at the BS, the mobile device, or both of them, which means the requested tasks are served through different paths, consuming different bandwidth, computing and caching resources. Since diverse services have different QoS, we formulate a multi-objective programming (MOP) to optimize the allocation of the 3C resources for minimizing the total delay while maximizing the number of executed tasks requested by the mobile devices. We transform the MOP problem as a multi-objective Markov decision process (MO-MDP) and design a multi-objective proximal policy optimization (MO-PPO) algorithm to solve the MO-MDP. The proposed MO-PPO first trains two sub-policies separately for the two objectives, and then combines them to search for Pareto dominating solutions. By alternately perform the separate training and the combination, we can finally obtain a set of Pareto optimal solutions and the corresponding Pareto front. Simulation results show that the proposed MO-PPO outperforms traditional methods in finding a higher-quality set of Pareto optimal solutions and can more appropriately allocate 3C resources to different types of services.

Federated learning (FL) is a decentralized algorithm that can train a globally shared model without the requirement to send the raw data to a centralized server by user equipments (UEs). Consider the UEs with non-independently and identically distributed (non-IID) data, heterogeneous computational capabilities and wireless channel conditions, FL becomes unproductive over a wireless network. In this paper, we jointly optimize the user scheduling policy and resource allocation to achieve a tradeoff among the fairness of user scheduling, the accuracy of FL, training time, and energy consumption of UEs. The optimization problem is formulated as a Markov Decision Process considering the potential impact of current scheduling on subsequent training and available resources. To solve the problem, a policy network is trained based on an actor-critic deep reinforcement learning framework. Simulation results show that the proposed user scheduling and resource allocation policy reduces the time and energy cost of the training process while improving the freshness of local update and performance on the 20% worst UEs compared with random user selection and resource allocation policy.