Federated learning (FL) offers a decentralized approach to machine learning. In FL, models are trained by the data from multiple devices or clients without necessarily centralizing this data, thus preserving privacy and reducing the need for data transfer. Despite its potential, FL faces inherent obstacles, most notably the challenge of achieving a fast convergence rate, especially with large, non-identically distributed client datasets. Also, weight pruning, an effective approach to reduce the number of weight parameters in a deep neural network, is hard to be employed on FL because it involves additional challenges to the convergence between different clients. To deal with the above issues, we propose a unified framework for the convergence and weight pruning in FL. We leverage the inherent structure of the Alternating Direction Method of Multipliers (ADMM) to partition the primary loss function for individual clients and apply specific dual variables to hasten the global model’s convergence. Our method, when tested on MNIST and SVHN datasets, consistently outperforms the established approaches on the convergence rate and model accuracy under the same weight pruning rate. For example, when the ResNet-18 model is pruned by 100 ×, our method achieves 0.65% to 3.09% accuracy improvement for the SVHN dataset under federated learning with non-identically distributed (non-IID) data distribution compared with the established approaches.

This paper introduces a representative-based approach for distributed learning that transforms multiple raw data points into a virtual representation. Unlike traditional distributed learning methods such as Federated Learning, which do not offer human interpretability, our method makes complex machine learning processes accessible and comprehensible. It achieves this by condensing extensive datasets into digestible formats, thus fostering intuitive human-machine interactions. Additionally, this approach maintains privacy and communication efficiency, and it matches the training performance of models using raw data. Simulation results show that our approach is competitive with or outperforms traditional Federated Learning in accuracy and convergence, especially in scenarios with complex models and a higher number of clients. This framework marks a step forward in integrating human intuition with machine intelligence, which potentially enhances human-machine learning interfaces and collaborative efforts.

In this work, we consider a binary hypothesis testing problem involving human decision-makers. Due to the nature of human behavior, human decision-makers observe the phenomenon of interest sequentially up to a random length of time. The humans use a belief model to accumulate the log-likelihood ratios until they cease observing the phenomenon. The belief model is used to characterize the perception of the human decision-maker towards observations at different instants of time, i.e., some decision-makers may assign greater importance to observations that were observed earlier, rather than later and vice-versa. We further consider the performance of a group of humans using a global decision-maker that fuses human decisions using the Chair-Varshney rule. When the number of observations that were used by the humans to arrive at their respective decisions are available to the fusion center (FC), the weights in the Chair-Varshney rule are modified to include this information in the decision fusion rule. Numerical and simulation results are presented to corroborate and validate theoretical results.