Deploying Convolutional Neural Network (CNN) models on ubiquitous Internet of Things (IoT) devices in a cloud-assisted manner to provide users with a variety of high-quality services has become mainstream. Most existing studies speed up model cloud training/on-device inference by reducing the number of convolution (Conv) parameters and floating-point operations (FLOPs). However, they usually employ two or more lightweight operations (e.g., depthwise Conv,$1\times 1$cheap Conv) to replace a Conv, which can still affect the model's speedup even with fewer parameters and FLOPs. To this end, we propose the Grouped NonLinear transformation generation method (GroupNL), leveraging data-agnostic, hyperparameters-fixed, and lightweight Nonlinear Transformation Functions (NLFs) to generate diversified feature maps on demand via grouping, thereby reducing resource consumption while improving the robustness of CNNs. First, in a GroupNL Conv layer, a small set of feature maps, i.e., seed feature maps, are generated based on the seed Conv operation. Then, we split seed feature maps into several groups, each with a set of different NLFs, to generate the required number of diversified feature maps with tensor manipulation operators and nonlinear processing in a lightweight manner without additional Conv operations. We further introduce a sparse GroupNL Conv to speed up by reasonably designing the seed Conv groups between the number of input channels and seed feature maps. Experiments conducted on benchmarks and on-device resource measurements demonstrate that the GroupNL Conv is an impressive alternative to Conv layers in baseline models. Specifically, on Icons-50 dataset, the accuracy of GroupNL-ResNet-18 is 2.86% higher than ResNet-18; on ImageNet-C dataset, the accuracy of GroupNL-EfficientNet-ES achieves about 1.1% higher than EfficientNet-ES. In addition, we verified the efficiency of GroupNL-based models in terms of cloud training and on-device inference.

Using low-rank adaptation to fine-tune pre-trained neural network models has attracted widespread attention due to its advantages of low resource requirements, high precision, and no additional inference delay. However, most existing methods are designed for large language models based on the Transformer structure and lack adaptation to convolutional neural networks (CNN) widely used on Internet of Things (IoT) devices. In addition, IoT devices are usually deployed outdoors and collect large amounts of data that are affected by environmental conditions. Providing a highly robust CNN model is a prerequisite for providing high-quality services. To this end, this paper proposes a highly robust seed-filter adaptation method (SeFA) for pre-trained CNNs. SeFA introduces an adaptation branch with the same structure as the backbone network. In the adaptation branch, some filters are first designated seed filters and grouped. Then, additional filters are generated from the grouped seed filters and nonlinear transformation functions (NLFs) with different hyperparameters. The seed filters' parameters are updated during model training, and the NLF hyperparameters are randomly initialized and frozen. Both grouping seed filters and configuring NLFs with non-learnable hyperparameters can improve the model's robustness. This is because grouping seed filters can generate diverse filters on demand without increasing the model's complexity, and the NLFs' rules can regularize the model. It is worth mentioning that the number of fine-tuning parameters of the pre-trained model that can adapt to downstream tasks can be flexibly controlled by specifying the number of seed filters. The key idea of this paper is to propose SeFA with flexible, controllable, learnable parameters, high robustness, and adaptability to pretrained CNN models, thereby facilitating fine-tuning on resource-constrained IoT devices and providing highly robust visual services. Experimental results on the CIFAR-10, CIFAR-10-C, CIFAR-100, CIFAR-100-C, and Icons-50 datasets demonstrate that the proposed SeFA outperforms other state-of-the-art methods. Specifically, on the ResNet-152 model and the CIFAR-10-C dataset, the accuracy of our SeFA is about 7% higher than that of the full fine-tuning method. In addition, we verify the efficiency of SeFA-based models for fine-tuning and inference on the device.

Cloud-assisted resource-constrained and widely used outdoor IoT devices deploying highly robust convolutional neural network (CNN) models to provide high-quality visual services have attracted widespread attention from industry and academia. Most existing methods suffer from two limitations: (i) large amount of parameter transmission, and (ii) low robustness in handling data affected by the environment. To this end, this paper proposes a transmission-friendly and robust CNN design method called NonLinear transformation generation Filter with NonLearnable hyperparameters, namely NL2Filter. In NL2Filter, some filters are first designated as seed filters, whose parameters are learnable, that is, updated as the model is trained. Other filters in this layer are generated based on the seed filters and nonlinear transformation function (NLF). The hyperparameters of the NLF in NL2Filter are randomly initialized and remain unchanged, so they can be saved and reproduced using random seed. After the cloud server trains the NL2Filter-CNN model, it only needs to send a small number of learnable parameters and random seed to reproduce the CNN model trained by the cloud server. Compared with sending the complete CNN model, sending only a small number of learnable parameters and random seed can significantly reduce the number of model parameters sent by the cloud server. On the other hand, NL2Filter groups seed filters and then uses NLFs with different hyperparameters to generate diverse filters on demand for each group, thereby improving the model's ability to capture feature diversity without increasing the complexity of the model, thereby improving the model's robustness in processing data affected by the environment. Experimental results on CIFAR-10, CIFAR-10-C, and Icons-50 datasets demonstrate that the proposed NL2Filter outperforms other state-of-the-art methods. Specifically, using the ResNet-101 architecture, on the CIFAR-10-C, NL2Filter's accuracy is about 3.9% higher than that of MonoCNN; On the Icons-50, NL2Filter's accuracy is about 2.7% higher than that of MonoCNN and about 2.3% higher than that of the standard ResNet-101.

Using low-rank adaptation to fine-tuning pretrained neural network models has attracted widespread attention due to its advantages of low resource requirements, high precision, and no additional inference delay. However, most existing methods are designed for large language models based on Transformer structure and lack adaptation to convolutional neural networks (CNN) widely used on Internet of Things (IoT) devices. In addition, IoT devices are usually deployed outdoors and collect a large amount of data affected by the environment. Providing a highly robust model is a prerequisite for providing high-quality services. To this end, this paper proposes a highly robust seed-filter adaptation method (SeFA) for pre-trained CNNs. SeFA introduces an adaptation branch with the same structure as the backbone network. In the adaptation branch, some filters are first designated seed filters and grouped. Then, other filters are generated based on the grouped seed filters and nonlinear transformation functions (NLFs) with different hyperparameters. The parameters of the seed filters are updated with model training, and the hyperparameters of the NLFs are randomly initialized and frozen. Both grouping seed filters and configuring NLFs with nonlearnable hyperparameters can improve the robustness of the model. This is because grouping seed filters can generate diverse filters on demand without increasing the model's complexity, and the NLFs' rules can regularize the model. The key idea of this paper is to propose SeFA with flexible controllable learnable parameters, high robustness and adaptability to pretrained CNN models, to facilitate fine-tuning of pre-trained CNN models on resource-constrained IoT devices to provide highly robust visual services. Experimental results on the CIFAR-10, CIFAR-10-C, CIFAR-100, CIFAR-100-C, and Icons50 datasets demonstrate that the proposed SeFA outperforms other state-of-the-art methods. Specifically, based on the ResNet152, on the CIFAR-10-C dataset, the accuracy of our SeFA is about$+7 {\%}$higher than that of the full fine-tuning method.

Deploying convolutional neural networks (CNNs) on resource-constrained Internet of Things (IoT) devices facilitates convenient intelligent services, which has attracted extensive attention. However, the dynamic resource availability poses significant challenges for resource-intensive CNNs. Recent advancements in efficient CNNs have two limitations: i) elevated resource consumption due to extensive datasets and post-training calibration; ii) lacking flexibility facing dynamic resources due to static model structure. To this end, we propose a RESourCe-Aware oncE-for-alL (ReScale) framework for efficient CNN deployment on IoT devices. Specifically, we propose a hierarchical filter generation mechanism to generate different amounts of filters dynamically, mapping a few learnable filters to abundant filters for discriminative feature extraction. Besides, we set a coefficient$\gamma$to modulate the filter generation according to the available resources of IoT devices. With this framework, we only need to train CNN models once to handle different resource availability of IoT devices. Experimental results show that our proposed ReScale framework can generate more efficient models with lower resource consumption while maintaining high accuracy. Through the coefficient$\gamma$, our method enables continuous model generation, ensuring robust adaptation to dynamic resource constraints.

Using pruning techniques to prune redundant weights in large language models (LLMs) for model size reduction, enabling deployment on devices to deliver high-quality services, has garnered significant attention from industry and academia. However, most existing pruning methods suffer from two major problems: (1) they rely on operations with high computational complexity, such as Hessian matrix calculation, and (2) high pruning rate leads to a significant decrease in model accuracy. To this end, this paper proposes a low resource requirement and low accuracy loss post-training pruning approach, namely the compensated post-training pruning method (CPP) for on-device LLM services. First, CPP employs singular value decomposition on weight matrices, sorting the decomposed singular values in descending order. Based on the pruning ratio, it retains the principal eigenvector corresponding to the larger singular values. To maintain consistent output feature distributions, CPP applies orthogonal transformations to input data and weight matrices, leveraging the principle of matrix orthogonality invariance. Compared with pruning approaches based on the Hessian matrix, CPP does not need to iteratively calculate the second derivative, thereby avoiding a high computational overhead. Second, the CPP incorporates bias compensation to use valuable information in pruned weights to improve the accuracy of the model. It constructs mapping relationships between input features and pruned weights through tensor decomposition techniques to generate bias compensation terms. These terms fine-tune the output of each layer, reducing pruning-induced errors from ratio-based pruning, and consequently improving model accuracy. Finally, CPP is validated through comprehensive experiments on nine benchmark datasets (e.g., WikiText-2 and PIQA) using large language models (i.e., LLaMA and OPT). Specifically, compared with the FLAP, CPP demonstrates 6.68% reduction in average pruning time. At pruning ratios of 10% and 20%, CPP achieves 2. 14% and 1.45% reductions in perplexity, respectively, along with 1.22% to 3.43% improvement in zeroshot inference accuracy.

Federated Learning (FL) faces challenges due to straggler clients that impede timely parameter uploads, potentially leading to suboptimal global model performance. Existing approaches using synchronous and asynchronous communication suffer from long waiting times or convergence issues. We propose Fed-OGD, a novel asynchronous FL method addressing the straggler problem through gradient orthogonalization. Our approach innovatively frames the straggler issue using catastrophic forgetting theory, viewing stragglers as instances of the global model “forgetting” to aggregate their parameters. Fed-OGD introduces an Orthogonal Gradient Descent (OGD) technique that caches straggler gradients and orthogonalizes the difference between these and current active client gradients. By projecting active gradients onto straggler orthogonal bases and subtracting the resulting components, we obtain orthogonalized gradients guiding the model towards optimality. We provide theoretical convergence guarantees and demonstrate Fed-OGD’s effectiveness through extensive experiments. Our method achieves state-of-the-art performance across multiple datasets among SOTA FL baselines, with notable improvements in non-IID (non-Independent and identically distributed) scenarios: there are few main categories with many samples while other categories hold few samples in a client. Fed-OGD achieves that 16.66% increase in accuracy on CIFAR-10, and significant gains on CIFAR-100 (5.37%), Tiny-ImageNet (38.51%), and AG_NEWS (16.30%).

Deploying quantized deep neural network (DNN) models with resource adaptation capabilities on ubiquitous Internet of Things (IoT) devices to provide high-quality AI services can leverage the benefits of compression and meet multi-scenario resource requirements. However, existing dynamic/mixed precision quantization requires retraining or special hardware, whereas post-training quantization (PTQ) has two limitations for resource adaptation: (i) The state-of-the-art PTQ methods only provide one fixed bitwidth model, which makes it challenging to adapt to the dynamic resources of IoT devices; (ii) Deploying multiple PTQ models with diverse bitwidths consumes large storage resources and switching overheads. To this end, this paper introduces a resource-friendly post-training integer-nesting quantization, i.e., NestQuant, for on-device quantized model switching on IoT devices. The proposed NestQuant incorporates the integer weight decomposition, which bit-wise splits quantized weights into higher-bit and lower-bit weights of integer data types. It also contains a decomposed weights nesting mechanism to optimize the higher-bit weights by adaptive rounding and nest them into the original quantized weights. In deployment, we can send and store only one NestQuant model and switch between the full-bit/part-bit model by paging in/out lower-bit weights to adapt to resource changes and reduce consumption. Experimental results on the ImageNet-1K pretrained DNNs demonstrated that the NestQuant model can achieve high performance in top-1 accuracy, and reduce in terms of data transmission, storage consumption, and switching overheads. In particular, the ResNet-101 with INT8 nesting INT6 can achieve 78.1% and 77.9% accuracy for full-bit and part-bit models, respectively, and reduce switching overheads by approximately 78.1% compared with diverse bitwidths PTQ models. Code:https://github.com/jianhayes/NESTQUANT.

For the creation of wireless network applications, node locations are frequently necessary. However, communication effectiveness, measurement accuracy, and localization stability will be low in irregular multi-hop networks when locating nodes using conventional algorithms. To this end, a novel cooperative localization algorithm using expected minimum segments (LEMS, for short) is proposed in this paper. LEMS begins by measuring the distance between paired nodes, which is completed along with network initialization. Then, each unlocated node constructs its own sub-network, including it, based on the error characteristics among anchor nodes. Finally, each unlocated node searches for its estimated location in its sub-region based on the objective function generated by the chaotic mapping. Simulation results demonstrate that the proposed algorithm significantly outperforms the state-of-the-art regarding efficiency, accuracy, and stability for various irregular networks. Specifically, our proposed algorithm achieves a median improvement in localization accuracy of 0.62 to 29.57 times and a reduction in the range of localization errors of 0.06 to 16.8 times.

Accurate geographical information of nodes is crucial for network applications. However, many existing positioning algorithms face challenges in achieving efficient, accurate, and robust performance when applied to irregular networks with holes or obstacles. Therefore, we introduce a new algorithm, named Network Division Positioning (NDP), to tackle this issue. In NDP, we use a similarity function to derive the distance between neighboring nodes and explore routing paths concurrently, facilitating efficient distance measurement. Next, we analyze measurement errors between landmark nodes to define a threshold that filters out incorrect distances, ensuring measuring and positioning accuracy. To enhance robustness, we first identify collinearity issues by examining the positional relationship between unpositioned nodes and their nearest landmark. Subsequently, we addressed the poor positioning results and built the subnetwork utilizing the nearest landmark node and its associated measurement distance, seeking the most accurate and robust estimated position within this subnetwork. The simulation results demonstrate that NDP outperforms state-of-the-art algorithms in terms of efficiency, accuracy, and robustness when dealing with various irregular networks. Specifically, NDP enhances positioning accuracy by at least 40.82% in terms of the median.

Deploying the convolutional neural network (CNN) model on Internet of Things (IoT) devices to provide diverse recognition services has received increasing attention. Due to the limited storage, computing, and other resources of IoT devices, it has become mainstream to first train the CNN model on the edge/cloud server and then send the trained CNN to the IoT device. However, most existing related methods suffer from two limitations, (i) low performance due to service interference or insufficient mutual assistance, and (ii) large memory resources and switching resource overhead. To this end, this article proposes a resource-efficient multiple recognition services framework for IoT devices. The proposed framework is based on the edge server-assisted IoT device training of the CNN model, and the framework includes a deeper weight adaptation (DeepWAdapt) algorithm to mitigate service interference. The DeepWAdapt algorithm consists of a set of learnable masks, and by inserting these masks into the appropriate layers of the CNN model, it mitigates mutual interference between services caused by training a single CNN model for multiple services. Each service has a specific set of masks. These learnable masks work like keys for each service, selecting appropriate and specific features for each service from a shared feature set. Experimental results demonstrate that the DeepWAdapt outperforms other state-of-the-art methods on image-level classification services and pixel-level dense prediction services. Specifically, when executing 40 services based on ResNet18, the proposed DeepWAdapt achieves 66.82% F1-score on the CelebA dataset, which is +2.61% F1-score than the previous state-of-the-art result. In addition, compared with the routing method, our proposed DeepWAdapt also reduces network transmission traffic by approximately 35%.

Cloud-assisted Internet of Things (IoT) device deployment of deep neural networks (DNNs) promotes On-device deep learning to provide users with ubiquitous high-quality services by solving the contradiction between insufficient IoT device resources and intensive demand for high-performance DNN resources. However, most existing methods optimize DNNs by considering one or two terms of transmission, computation, and storage resources, but do not consider all three terms at the same time in cloud-assisted IoT device deployment and updating DNNs. To this end, we propose a non-learnable module-based ResNet and a cloud-assisted on-device deep learning framework, ReFrame, based on the consideration of three indicators: model transmission parameters, computation resources, and storage resources. In the proposed method, we first specify that some parameters in DNNs are non-learnable and randomly initialized, so that, these parameters can be saved and reproduced with a few random seeds. By doing so, the cloud only transmits random seeds and learnable parameters to reduce the number of parameter transmissions. Second, we reduce the computation resource consumption of the model by introducing computation-friendly operators, such as pooling, to replace vanilla convolutions. Finally, since random seeds are used to save non-learnable model parameters, on IoT devices we only need to store random seeds and learnable parameters to reproduce the well-trained model. Compared with saving the complete model, our method greatly reduces IoT device storage resource consumption. Experimental results on image classification, object detection, and semantic segmentation tasks demonstrate the effectiveness of the proposed method. Specifically, on the CIFAR-10, our proposed method reduces approximately 89% of FLOPs and 90% of transmitted data in the prototype system compared to ResNet-18.

Deploying a single deep neural network (DNN) model on ubiquitous Internet of Things (IoT) devices to provide multiple vision services (e.g., semantic segmentation service, facial attribute recognition service) has attracted significant interest from industry and academia. However, the most relevant studies have two limitations: (i) the multiple services DNN commonly has low performance due to the gradient interference between different vision services; (ii) the multiple services DNN has low inference speed on IoT devices. To this end, this paper introduces a multiple vision services acceleration framework for IoT devices. The proposed MCBNet contains a novel multi-controllable branching structure to control different services explicitly for improving multiple services performance and a service balance layer with the per-channel trainable masks in specific branches for mitigating gradient interference. In deployment, the proposed framework converts the multi-branch topology MCBNet model into the plain topology Rep-MCBNet model by re-parameterization technique for accelerating services inference speed. Experimental results on the CelebA and Cityscapes datasets show that our proposed framework outperforms the state-of-the-art approaches in terms of performance and inference speed. In particular, we deploy the Rep-MCBNet on the laptop with Intel mobile processors demonstrating that the proposed framework achieves 1.9× faster processing frames per second than ETR-NLP, with 1.5× fewer parameters and 1.3× fewer floating-point operations.

Accelerometer signals play a critical role in many fields, for example navigation and vehicle safety. However, uncontrollable factors such as defective equipment and harsh environments make the signals recorded by sensors contain a large amount of noise, which poses a great challenge. Most existing methods are based on traditional signal processing and often face problems of incomplete noise reduction or signal distortion after denoising. In this paper, we propose a data-driven denoising method for accelerometer signals (ADNet) based on the Wave_U_Net network architecture, incorporating Multi-Head Attention mechanism and Spatial Attention mechanism. This enhances the network’s feature extraction capability and accelerates its convergence speed. Meanwhile, the attention mechanism focuses the model’s attention on the clean signal’s feature information, improving the network’s fitting capability to the signal distribution. Therefore, ADNet can achieve more thorough denoising while reducing signal distortion after denoising. Finally, we conduct experiments on Walking speed dataset and field experiment dataset to verify the effectiveness of ADNet. The experimental results show that ADNet outperforms other baseline models in terms of Mean Square Error, Mean Absolute Error, Root Mean Square Error, and Signal-to-Noise Ratio.

Extracting features from image data on Internet of Things (IoT) devices to reduce the amount of data that needs to be uploaded to cloud/edge servers has received increasing attention. However, most of the existing related approaches suffer from two major limitations, (i) low performance and high network traffic, and (ii) a lot of storage resource consumption. To this end, we propose a resource-efficient feature extraction framework for image processing in IoT devices. The proposed framework consists of the edge-assisted extractor generation method and the NestE method. The extractor generated by the edge-assisted extractor generation method can extract the features required by the application, which can not only avoid the IoT device uploading useless feature data but also improve application performance. The proposed NestE generates a nonredundant subextractor by splitting the extractor into multiple subextractors, removing redundant subextractors, and nesting small-capacity subextractors in large-capacity subextractors in a parameter-sharing manner. Compared with deploying multiple independent subextractors on IoT devices, deploying the nonredundant multifunctional extractor can save considerable storage resources and switching overhead. Extensive experimental results show that the proposed framework reduces the storage footprint by approximately 90.7% and switching overhead by approximately 92.4% compared with deploying independent subextractors when using the classical principal component analysis algorithm.

Due to the contradiction between limited bandwidth and huge transmission parameters, federated Learning (FL) has been an ongoing challenge to reduce the model parameters that need to be transmitted to server in clients for fast transmission. Existing works that attempt to reduce the amount of transmitted parameters have limitations: 1) the reduced number of parameters is not significant; 2) the performance of the global model is limited. In this paper, we propose a novel method called Fed-KGF that significantly reduces the amount of model parameters while improving the global model performance. Our goal is to reduce those transmitted parameters by reducing the number of convolution kernels. Specifically, we construct an incomplete model with a few representative convolution kernels, and propose Kernel Generation Function (KGF) to generate other convolution kernels to render the incomplete model to be a complete one. We discard those generated kernels after training local models, and solely transmit those representative kernels during training, thereby significantly reducing the transmitted parameters. Furthermore, there is a client-drift in the traditional FL because of the averaging method, which hurts the global model performance. We innovatively select one or few modules from all client models in a permutation way, and only aggregate the uploaded modules rather than averaging all modules to reduce client-drift, thus improving the global model performance and further reducing the transmitted parameters. Experimental results on both non-Independent and Identically Distributed (non-IID) and IID scenarios for image classification and object detection tasks demonstrate that our Fed-KGF outperforms SOTA FL models.

Node positioning is an essential function of wireless networks and serves as the foundation for many applications. In the existing works, the cooperative positioning approaches have been extensively studied and are shown to be effective for scenarios with energy and cost constraints. However, these approaches may not perform well in concave networks with holes or obstacles. To address this issue, this paper proposes adistributed cooperative positioning model with adaptive hop-range selection(DCP-AHS for short) for concave networks. DCP-AHS first uses a low-complexity and fast convergent distance estimation method based on the local neighbor nodes. It then uses an adaptive hop-range selection method based on the residual analysis between pairs of anchors. Within the hop range, an unknown node uses multi-lateration with the optimal weight function to determine its estimated position. Finally, a weighted Bounding-Box method with the virtual anchor is employed to avoid significant position estimation errors caused by the collinearity issues. Simulation results demonstrated that the proposed DCP-AHS significantly outperformed the existing algorithms regarding efficiency, accuracy, and stability in various concave networks. Specifically, our proposed model achieved a median improvement of 16.62% to 81.65% in positioning accuracy compared to the comparison algorithms.

With the advancement of intelligent and networked technology, the Industrial Internet of Things (IIoT) faces an escalating threat from cyberattacks, especially by Advanced Persistent Threat (APT) attacks. These novel and complex attacks, characterized by their dynamic nature and life-long duration, pose significant challenges to existing security protection methods. The challenges are twofold, i.e., sparse reward problem in the long-lasting attack, and partial observation of attack actions. To this end, we propose a Security-as-a-Service based reinforcement learning method, namely Attention Augmented Dueling Deep Q-learning Network (AD2QN), to make real-time defense strategies for the hot standby IIoT. First, we build the attack-defend confrontation model as black boxes interact with the IIoT environment to play a long-lasting partially observable zero-sum stochastic game on the server. Then, to dynamically generate optimal defense strategies as the service, AD2QN is proposed employing information completion and prediction to more informed action selection. Furthermore, AD2QN utilizes an iteratively updated reward network to deal with the sparse reward problem. Extensive simulation results shown that the defense strategies generated by our method have a higher defense success rate and a stable defense performance with the average success rate of 0.7384, while the average success rate of baseline methods was 0.7375, in the best case.

Multi-task learning (MTL) seeks to learn a single model to accomplish multiple tasks by leveraging shared information among the tasks. Existing MTL models, however, have been known to suffer from negative interference among tasks. Efforts to mitigate task interference have focused on either loss/gradient balancing or implicit parameter partitioning with partial overlaps among the tasks. In this paper, we propose ETR-NLP to mitigate task interference through a synergistic combination of non-learnable primitives (NLPs) and explicit task routing (ETR). Our key idea is to employ non-learnable primitives to extract a diverse set of task-agnostic features and recombine them into a shared branch common to all tasks and explicit task-specific branches reserved for each task. The non-learnable primitives and the explicit decoupling of learnable parameters into shared and task-specific ones afford the flexibility needed for minimizing task interference. We evaluate the efficacy of ETR-NLP networks for both image-level classification and pixel-level dense prediction MTL problems. Experimental results indicate that ETR-NLP significantly outperforms state-of-the-art baselines with fewer learnable parameters and similar FLOPs across all datasets. Code is available at this URL.

Deploying high-performance convolutional neural network (CNN) models on low-earth orbit (LEO) satellites for rapid remote sensing image processing has attracted significant interest from industry and academia. However, the limited resources available on LEO satellites contrast with the demands of resource-intensive CNN models, necessitating the adoption of ground-station server assistance for training and updating these models. Existing approaches often require large floating-point operations (FLOPs) and substantial model parameter transmissions, presenting considerable challenges. To address these issues, this paper introduces a ground-station server-assisted framework. With the proposed framework, each layer of the CNN model contains only one learnable feature map (called the seed feature map) from which other feature maps are generated based on specific rules. The hyperparameters of these rules are randomly generated instead of being trained, thus enabling the generation of multiple feature maps from the seed feature map and significantly reducing FLOPs. Furthermore, since the random hyperparameters can be saved using a few random seeds, the ground station server assistance can be facilitated in updating the CNN model deployed on the LEO satellite. Experimental results on the ISPRS Vaihingen, ISPRS Potsdam, UAVid, and LoveDA datasets for semantic segmentation services demonstrate that the proposed framework outperforms existing state-of-the-art approaches. In particular, the SineFM-based model achieves a higher mIoU than the UNetFormer on the UAVid dataset, with 3.3 × fewer parameters and 2.2 × fewer FLOPs.

Deploying high-performance convolutional neural networks (CNNs) on ubiquitous Internet of Things (IoT) devices to provide convenient services has attracted increasing attention. However, most existing studies have two limitations, (i) low performance due to lack of effective learning strategies; (ii) difficulty to measure resource consumption due to lack of real deployment. To this end, this paper proposes a novel localized knowledge distillation (LKD) method to train the resource-efficient CNN and implements a cloud-assisted system to evaluate the on-device performance. The proposed LKD follows the layer-wise heterogeneous information distribution in the CNN and distills the knowledge from features that contains the most crucial knowledge to the resource-efficient CNN. Thus, the knowledge guided to learn by limited parameters is reduced to the crucial part, which is more reliable for the resource-efficient CNN. The cloud-assisted system consists cloud server and IoT devices, which allows the training and deployment of the resource-efficient CNN, and the measurement of the corresponding resource consumption. Experimental results show that the proposed LKD could improve the performance on standard benchmarks close to the counterpart CNN, and the robustness on corrupted data by approximately 11.3% for the resource-efficient CNN. The measurements on the cloud-assisted system also demonstrate the resource efficiency for transmission and on-device running.

It has become mainstream for mobile devices to offload latency-sensitive applications to edge servers for execution to meet low-latency requirements. However, the existing related studies lack the consideration of application hit ratio, which makes them unable to meet the increasingly complex offloading of multi-applications including multi-tasks. To this end, this paper proposes a Multi-task offloading and Service placement optimization (MSO) method with the goal of maximizing the application hit ratio to provide high-quality service. The proposed MSO is constructed with Improved Multi-Agent Q-Learning (IMAQL) and load-balancing algorithms. IMAQL aims to learn an optimal service placement policy by using Q-learning techniques. Next, the load-balancing algorithm is designed to offload tasks according to the service placement policy. To verify the effectiveness of the MSO method, we conduct extensive experiments on a publicly available dataset. The experimental results show that the proposed method can improve the application hit ratio by appropriately 2.6% to 9% compared with other methods.

With the rapid development of the Internet of Things (IoT) and emerging Mobile Edge Computing (MEC) technologies, various IoT image recognition services are revolutionizing our lives by providing diverse cognitive assistance. However, most existing related approaches are difficult to meet the diversified needs of users because they believe that the MEC platform is a single layer. In addition, due to the mutual interference between the data, it is not easy for them to extract the discriminative features (DFs) necessary to analyze the input data. To this end, this article proposes an IoT image recognition services framework for different needs in the MEC environment, which consists of Hierarchical Discriminative Feature Extraction (HDFE) and Sub-extractor Deployment (Sub-ED) algorithms. We first propose HDFE, which can avoid mutual interference between data by separately optimizing the data structure, thereby generating an extractor that extracts effective DFs. Then there is Sub-ED, which divides the extractor into a series of sub-extractors and deploys them on appropriate MEC platforms. By doing so, the IoT device can connect to the corresponding MEC platform according to its service types, and use the sub-extract to extract DFs. Then, the MEC platform uploads the extracted feature data to the cloud server for further processing, e.g., feature matching. Finally, the cloud server sends the processed result back to the IoT device. Experimental results show that compared with the state-of-the-art approaches, the proposed framework improves recognition accuracy by about 6% and reduces network traffic by up to 94%.

Task offloading offloads latency-sensitive and computation-intensive applications from resource-constrained terminal devices to relatively resource-rich edge servers to meet users’ demands for latency and energy consumption, which has attracted extensive attention from academia and industry. However, most of the existing researches only considers offloading dependent tasks within a single application or multiple independent applications, while ignoring the dependencies between applications. To this end, this paper proposes an offloading strategy for distributed dependent applications under the condition of limited computing and cache resources. The goal of the proposed strategy is to minimize the weighted sum of latency and energy to complete all applications while solving the offloading and resource allocation problems of dependent applications. However, the dual dependencies between applications and tasks within the application complicate offloading tasks. To accommodate this issue, we represent the dual dependencies as a directed acyclic graph. Then, we design the offloading strategy as follows: First, we transform the formulated non-convex problem into convex optimization subproblems. Second, we iteratively calculate the task priority and obtain the optimal offloading decision of the task according to the priority. Finally, we perform validation on real datasets. Compared with several state-of-the-art methods, our proposed strategy can significantly reduce the weighted sum of latency and energy.

Deploying deep convolutional neural network (CNN) models on ubiquitous Internet of Things (IoT) devices has attracted much attention from industry and academia since it greatly facilitates our lives by providing various rapid-response services. Due to the limited resources of IoT devices, cloud-assisted training of CNN models has become the mainstream. However, most existing related works suffer froma large amount of model parameter transmission and weak model robustness. To this end, this paper proposes a cloud-assisted CNN training framework with low model parameter transmission and strong model robustness. In the proposed framework, we first introduce MonoCNN, which contains only a few learnable filters, and other filters are nonlearnable. These nonlearnable filter parameters are generated according to certain rules, i.e., the filter generation function (FGF), and can be saved and reproduced by a few random seeds. Thus, the cloud server only needs to send these learnable filters and a few seeds to the IoT device. Compared to transmitting all model parameters, sending several learnable filter parameters and seeds can significantly reduce parameter transmission. Then, we investigate multiple FGFs and enable the IoT device to use the FGF to generate multiple filters and combine them into MonoCNN. Thus, MonoCNN is affected not only by the training data but also by the FGF. The rules of the FGF play a role in regularizing the MonoCNN, thereby improving its robustness. Experimental results show that compared to state-of-the-art methods, our proposed framework can reduce a large amount of model parameter transfer between the cloud server and the IoT device while improving the performance by approximately 2.2% when dealing with corrupted data.

Deploying high-performance vision transformer (ViT) models on ubiquitous Internet of Things (IoT) devices to provide high-quality vision services will revolutionize the way we live, work, and interact with the world. Due to the contradiction between the limited resources of IoT devices and resource-intensive ViT models, the use of cloud servers to assist ViT model training has become mainstream. However, due to the larger number of parameters and floating-point operations (FLOPs) of the existing ViT models, the model parameters transmitted by cloud servers are large and difficult to run on resource-constrained IoT devices. To this end, this article proposes a transmission-friendly ViT model, TFormer, for deployment on resource-constrained IoT devices with the assistance of a cloud server. The high performance and small number of model parameters and FLOPs of TFormer are attributed to the proposed hybrid layer and the proposed partially connected feed-forward network (PCS-FFN). The hybrid layer consists of nonlearnable modules and a pointwise convolution, which can obtain multitype and multiscale features with only a few parameters and FLOPs to improve the TFormer performance. The PCS-FFN adopts group convolution to reduce the number of parameters. The key idea of this article is to propose TFormer with few model parameters and FLOPs to facilitate applications running on resource-constrained IoT devices to benefit from the high performance of the ViT models. Experimental results on the ImageNet-1K, MS COCO, and ADE20K datasets for image classification, object detection, and semantic segmentation tasks demonstrate that the proposed model outperforms other state-of-the-art models. Specifically, TFormer-S achieves 5% higher accuracy on ImageNet-1K than ResNet18 with 1.4× fewer parameters and FLOPs.

The explosion of Internet of Things (IoT) devices will generate massive amounts of data. Due to the limited resources of IoT devices, they usually upload the collected data to edge/cloud servers for processing. To reduce the amount of data uploaded to the edge/cloud server, extracting the features of data on IoT devices has attracted increasing attention. However, most existing related works suffer from two major limitations: 1) difficulty meeting user needs: the extractors they generate are difficult to extract effective features from data on IoT devices and 2) consume a lot of storage resources: they deploy multiple extractors to adapt to the dynamically changing resources of IoT devices. To this end, we propose a nonredundant discriminative feature extraction (DFE) framework, which consists of similarity-based DFE (SDFE) and 2RNestE algorithms. SDFE is first proposed to generate an extractor E that can extract effective discriminative features by rationally exploring the structural information of the data set on the edge/cloud server. Then, 2RNestE is proposed, which takes E as input, and outputs a nonredundant multifunctional extractor by removing redundant subextractors and nesting the remaining nonredundant subextractors together. Finally, the edge/cloud server sends the generated extractor to the IoT device. Experimental results show that the proposed framework reduces memory footprint by about 82.6% and switching overhead by about 84.6% compared with state-of-the-art works.

Mobile applications can leverage high-quality deep learning models such as convolutional neural networks and deep neural networks to provide high-performance cognitive services. Prior work on deep learning models-based mobile applications in a cloud-edge computing environment focuses on performing lightweight data pre-processing tasks on edge servers for cloud-hosted cognitive servers. These approaches have two major limitations. First, it is uneasy for the mobile applications to assure satisfactory user experience in terms of network communication delay, because the intermediary edge servers are used only to pre-process data (e.g., images and videos) and the cloud servers are used to complete the tasks. Second, these approaches assume the pre-trained deep learning models deployed on cloud servers are static, and will not attempt to automatically upgrade in a context-aware manner. In this article, we propose a cloud-edge collaboration framework that facilitates delivering cognitive services with long-lasting, fast response, and high accuracy properties. We fist deploy a shallow model (i.e., EdgeCNN) on the edge server and a deep model (i.e., CloudCNN) on the cloud server. EdgeCNN can provide durable and rapid response cognitive services, because edge servers not only provide computing resources for mobile applications, but also close to users. Then, we enable CloudCNN to assist in training EdgeCNN to improve the performance of the latter. Thus, EdgeCNN also provides high-accuracy cognitive services. Furthermore, because users may continue to upload data to edge servers in real-world scenarios, we propose to use the ongoing assistance of CloudCNN to further improve the accuracy of the shallow model. Experimental results show that EdgeCNN can reduce the average response time of cognitive services by up to 55.08 percent and improve accuracy by up to 26.70 percent.

Mobile image recognition services, which provide people with image recognition services through the cameras of mobile devices, are revolutionizing our lives. However, most existing cloud/edge-based approaches suffer from two major limitations, (i) Low recognition accuracy and high network bandwidth pressure, and (ii) Not easy to extract features based on currently available resources of mobile devices. In this paper, we propose a resource-aware feature extraction framework for mobile image recognition services. The proposed framework consists of discriminative feature extraction (DFE) and NestDFE algorithms. The DFE algorithm can generate an extractor${{\mathbf E}}$to extract discriminative features from the image data set on the edge server and images on mobile devices. Thus, the proposed framework can achieve higher recognition accuracy and require mobile devices to upload less feature data to the edge server. The NestDFE algorithm generates a single multi-capacity extractor that acts as a series of sub-extractors and enables mobile devices to dynamically select sub-extractors. Experimental results show that the proposed framework improves recognition accuracy by about 23 percent and reduces network traffic by about 76 percent compared with existing approaches.

Mobile Edge Computing (MEC) can facilitate various important image retrieval applications for mobile users by offloading partial computation tasks from resource-limited mobile devices to edge servers. However, existing related works suffer from two major limitations. (i) High network bandwidth cost: they need to extract numerous features from the image and upload these feature data to the cloud server. (ii) Lowretrieval accuracy: they separate the feature extraction processes from the image data set in the cloud server, thus unable to provide effective features for accurate image retrieval. In this paper, we propose a cloud-guided feature extraction approach for mobile image retrieval. In the proposed approach, the cloud server first leverages the relationships among labeled images in the data set to learn a projection matrix P. Then, it uses the matrix P to extract discriminative features from the image data set and form a low-dimensional feature data set. Following that, the cloud server sends the matrix P to the edge server and uses it to multiply the image χ. The result PTχ, i.e., image features, is uploaded to the cloud server to find the label of the image with the most similar multiplying result. The label is regarded as the retrieval result and returned to the mobile user. In the cloud-guided feature extraction approach, the matrix P can extract a small number of effective image features, which not only reduces network traffic but also improves retrieval accuracy. We have implemented a prototype system to validate the proposed approach and evaluate its performance by conducting extensive experiments using a real MEC environment and data set. The experimental results show that the proposed approach reduces the network traffic by nearly 93 percent and improves the retrieval accuracy by nearly 6.9 percent compared with the state-of-the-art image retrieval approaches in MEC.

Cognitive services have revolutionized the way we live, work and interact with the world. In recent years, deep neural networks have become the mainstream approach in cognitive service, and mobile edge computing facilitates a variety of cognitive services for users by offloading computation tasks from resource-limited mobile devices to relatively wealthy edge servers. Combining the two to provide users with a higher quality of cognitive service is an issue worth researching. However, many related studies are not easy to provide fast responses because in these systems, edge servers are only used to pre-process data, and the cloud server is used to perform tasks. In this paper, we aim to study deploying deep neural network models on edge servers to provide fast services. However, a single edge server collects only a small amount of data, which results in low inference accuracy. To address this problem, we propose a cloud and edge collaboration framework. The key idea of the proposed framework is to use a cloud model to assist in training an edge model to improve the latter's inference accuracy and enable the latter to provide fast response and high-performance cognitive service. Experimental results demonstrate the effectiveness of our proposed framework.

The idea that with the help of proper dimensionality reduction, trying to make the samples with the same label be compact and the ones with the different labels be separate after projection, is introduced into classification problems with high-dimensional data. Based on the analysis of the drawbacks of Discriminant Neighborhood Embedding (DNE) and Locality-Based Discriminant Neighborhood Embedding (LDNE), being the two relatively successful Locally Discriminant Analysis methods proposed in recent years, this paper proposes a method called Similarity-balanced Discriminant Neighborhood Embedding (SBDNE). When constructing the adjacent graph, SBDNE fully takes into account the geometric construction of manifold and the problem of imbalance between the intra-class points and the inter-class points. By endowing these two kinds of samples with different similarities and selecting the near neighbors according to the similarity matrix, not only the structure in the original space can be preserved more efficiently, but also the choice of discriminative information increases. The method proposed here has a better recognition with comparisons to some classical methods, which fully shows that SBDNE method has the capacity to efficiently solve the classification problem.

Discriminant neighborhood embedding (DNE) algorithm is one of supervised linear dimensionality reduction methods. Its nonlinear version kernel discriminant neighborhood embedding (KDNE) is expected to behave well on classification tasks. However, since KDNE constructs an adjacent graph in the original space, the adjacency graph could not represent the adjacent information in the kernel mapping space. By introducing hidden space, this paper proposes a novel nonlinear method for DNE, called hidden space discriminant neighborhood embedding (HDNE). This algorithm first maps the data in the original space into a high dimensional hidden space by a set of nonlinear hidden functions, and then builds an adjacent graph incorporating neighborhood information of the dataset in the hidden space. Finally, DNE is used to find a transformation matrix which would map the data in the hidden space to a low-dimensional subspace. The proposed method is applied to ORL face and MNIST handwritten digit databases. Experimental results show that the proposed method is efficiency for classification tasks.