Deep convolutional neural networks are hindered from empowering resource-constrained devices due to their demanding computational and storage resources. Structured pruning effectively removes the redundant components from neural networks and obtains compact models. Previous pruning methods usually evaluate the importance of filters from a layer-wise perspective, which is deprived of global guidance. We propose an adaptive pruning algorithm based on relevance scores to evaluate the contribution of each channel by calculating its relevance score from the back-propagation of the neural network's output. Our method identifies and removes channels with low contribution from a global perspective. Unlike previous methods that manually set the pruning rate for each pruning iteration, our method adaptively adjusts the pruning rate. In addition, our method performs satisfactorily with limited data for one-shot pruning in the absence of fine-tuning. The ability to obtain compact models through one-shot pruning with limited data is ideally suited for edge computing scenarios. We validate the effectiveness of our method with multiple combinations of convolutional neural networks and datasets. Our approach outperforms existing pruning methods in scenarios with limited data.

Lightweight models with high accuracy is critical for edge intelligence. Although the Knowledge Distillation (KD) has been successfully applied to reduce the accuracy loss of quantized neural networks, especially for resource-constrained edge devices, the process of pre-training complex high-precision teacher networks in KD however, will bring huge training overhead. Recently proposed online distillation frameworks offer a good solution for teacher-free distillation, but the regularization effect and simple average aggregation of KD further weaken the representation capability of quantized models that have been reconstructed. In this work, we propose Precision-Mixed and Weight-Average Ensemble (PMWAE) consisting of multiple group members and a group leader. PMWAE provides additional knowledge by changing the bit-precision of the activation and generates aggregated weights for each member in group by attention-based mechanism. The ensemble knowledge is further passed to the group leader to obtain the final model. Extensive experiments on the CIFAR-10/100 and ImageNet-1K datasets show that our method outperforms the existing state-of-the-art methods, both on standard convolutions and depth-wise separable convolutions.

Federated learning (FL) lately has shown much promise in improving the shared model and preserving data privacy. However, these existing methods are only of limited utility in the Internet of Things (IoT) scenarios, as they either heavily depend on high-quality labeled data or only perform well under idealized conditions, which typically cannot be found in practical applications. In this article, we propose a novel federated unsupervised learning method for image classification without the use of any ground truth annotations. In IoT scenarios, a big challenge is that decentralized data among multiple clients is normally nonindependent and identically distributed (non-IID), leading to performance degradation. To address this issue, we further propose a dynamic update mechanism that can decide how to update the local model based on weights divergence. Extensive experiments show that our method outperforms all baseline methods by large margins, including +6.67% on CIFAR-10, +5.15% on STL-10, and +8.44% on SVHN in terms of classification accuracy. In particular, we obtain promising results on Mini-ImageNet and COVID-19 data sets and outperform several federated unsupervised learning methods under non-IID settings.

Federated learning(FL) is a privacy-preserving dis-tributed learning paradigm in which clients cooperate with each other to train a global model. It is becoming progressively prevalent with the rapid development of edge devices. A critical challenge in federated learning is the data heterogeneity among clients, resulting in the global model generated by standard federated learning being unable to be adapted to all clients. To tackle this problem, we propose the CFedPer for personalized FL, which generates a personalized model for each cluster after clustering to address the deficiency of standard federated learning. Our algorithm is organized into two optimization phases. The pre-start phase clusters clients by our proposed similarity-based clustering model using distribution vector and similarity matrix. In the in-training phase, we represent the neural network as the base layer and personalization layer and propose a novel optimization objective with a regularization term for the personalization layer to achieve a balance between per-sonalization and generalization, preventing over-personalization. Extensive experiments on various datasets and data distributions indicate that the performance of our algorithm is superior to the existing algorithms in terms of average local accuracy and variance among clients.

In the Industrial Internet of Things (IIoT), model and computing power sharing among devices can improve resource utilization and work efficiency. However, data privacy and security issues hinder the sharing process. Besides, in the process of model sharing, due to the customization of industrial equipment functions and the high separation of model and task types between devices, it is difficult to share model and optimize models among devices with different task requirements. In this article, we propose an adaptively federated multitask learning (AFL) for IIoT devices efficiently model sharing. Inspired by the parameter sharing mechanism, AFL builds a sparse sharing structure by designing an iterative pruning network and generating subnets for each task. Moreover, for better share relevant information, we further propose tailored task mask layers for effectively training specialized subnets, and an adaptive loss function to dynamically adjust the priority between tasks. Extensive experiments show that AFL can successfully fit hundreds of tasks from different devices into one model, which preserves both high accuracy and system scalability, and outperforms other related approaches that naively combine federated learning with multitask learning.

Blockchain as a new decentralized chain structure can be used in edge computing to solve the security issue caused by edge nodes in model training. However, large amounts of data exchanges in the process of model training of edge computing reduce the performance of blockchain, and meanwhile, the block needed to be saved in the edge node challenges storage capacity of the edge node. Therefore, in the paper we propose a safe and efficient model training mechanism based on onchain and offchain collaboration. In the mechanism, edge nodes train models locally, store the model parameters in offchain and only return identifiers for model aggregation. By the method, the storage pressure of the edge node is reduced and the efficiency of executing consensus algorithms are increased. Moreover, in the mechanism we design a reputation evaluation model based on confidence factors to avoid the uploading of random and wrong data of edge nodes. Evaluation results show that our schemes can reduce the average delay and resources consumption, increase transaction throughput and maintain security compared with a state-of-the-art scheme.

Federated learning (FL) is a distributed framework for machine learning (ML) model training. Training agents upload local model parameters rather than original training data, and the central server performs parameter aggregation. FL can protect user data privacy and break the information island when training the ML model. Federated Average (FedAvg) is an aggregation method commonly used in the FL training task. The central server calculates the mean value of the local model parameters to obtain the new global parameters. FedAvg assumes that all the training agents are honest, which means the central server lacks terminal agents' knowability. When there are attackers in the training agents, the global model's performance may be deeply affected, and the training task cannot be completed normally. To solve this problem, we propose a Federated Learning method with aggregation based on the Importance of training agent (FedIM), in which the central server performs pre-evaluation on the agent parameters before aggregation, calculates the weights of parameters according to the historical behavior records of training terminals and performs federated aggregation to improve the anti-poisoning ability of learning task. Experiments show that our method can effectively improve the global model's anti-poisoning ability and accelerate the training speed compared with the FedAvg method when malicious agents are involved.

The blockchain system allows various trans-actions and information storage to be executed in a decentralized manner, while smart contracts require multiple nodes to be executed in the local sandbox environment according to preset settings to ensure the consistency of each node, which makes smart contracts unable to proactively obtain data from the outside world. Decentralized oracle can realize the acquisition of off-chain data with a low speed under the premise of ensuring the decentralization of the blockchain. Some oracles use on-chain data storage and maintenance to speed up data acquisition, but this will face higher costs of data storage and maintenance, so current oracles cannot simultaneously ensure privacy and security while taking into account execution cost and processing speed. In this article, we propose Select-Storage, a new oracle design pattern to achieve low operating cost and high processing speed without compromising security. Through experimental analysis, and comparison with other design patterns in processing time and on-chain and off-chain call costs, we have proved the superiority of the Select-Storage design pattern.

The traditional centralized data processing model represented by cloud computing cannot meet the data processing requirements that are gradually tending to the edge. Therefore, a new distributed computing model coordinated by the end devices, edges and cloud has become the main development direction. However, artificial intelligence algorithms that are widely used in cloud-only approach are difficult to embed in resource-constrained distributed frameworks. To address this issue, we propose Triple-partition Network, a neural network model augment with three exit points. The structure of three exit points allows to segment the traditional neural network and deploying them on the end devices, edges, and cloud. By setting up suitable exit points through the Entropy Topsis comprehensive evaluation model, part of the data can exit the network in advance to improve the efficiency of computing services. In this experiment, the classic neural networks (Alexnet, Resnet) are used to study the Triple-partition Network on a state-of-art platform and show that trained Triple-partition Network can greatly reduce the end-to-end latency by over 3x while achieving high accuracy.

Task offloading has been a hot topic in the field of edge computing. Resources fairness of edge computing servers which is the destination of task offloading directly impacts life of server and the process quality of task. In this paper, we propose a subtask-virtual machine mapping model (subtask-VM mapping model) to complete task offloading from the terminals to the servers. Considering the reasonable allocation of server resources, we also propose stack-based cache mechanism (SCM) to ensure the fairness of server resources allocation. We transform the problem of mapping model solution into the problem of optimal matching in the bipartite graph, and verify the performance of our algorithm by contrast experiment. In particular, the fair performance of our algorithm for server-side is over 84%.