Today, convolutional neural network (CNN) pruning techniques often rely on manually crafted importance criteria and pruning structures. Due to their heuristic nature, these methods may lack generality, and their performance is not guaranteed. In this paper, we propose a theoretical framework to address this challenge by leveraging the concept of $\gamma$γ-weak submodularity, based on a new efficient importance function. By deriving an upper bound on the absolute error in the layer subsequent to the pruned layer, we formulate the importance function as a $\gamma$γ-weakly submodular function. This formulation enables the development of an easy-to-implement, low-complexity, and data-free oblivious algorithm for selecting filters to be removed from a convolutional layer. Extensive experiments show that our method outperforms state-of-the-art benchmark networks across various datasets, with a computational cost comparable to the simplest pruning techniques, such as $l_{2}$l2-norm pruning. Notably, the proposed method achieves an accuracy of 76.52%, compared to 75.15% for the overall best baseline, with a 25.5% reduction in network parameters. According to our proposed resource-efficiency metric for pruning methods, the ACLI approach demonstrates orders-of-magnitude higher efficiency than the other baselines, while maintaining competitive accuracy.

Convolutional neural network (CNN) pruning has traditionally relied on heuristically designed importance criteria, often leading to limited generalizability and inconsistent performance. In this article, we propose a novel framework centered around filter replacement (FR), introducing pruning as a process of replacing selected filters with zero filters. Through a rigorous analysis, we derive an upper bound on the absolute error in the output of the subsequent layer and use this bound to define an efficient importance function. This importance function exhibits $\gamma $ -weakly submodular properties, enabling the development of a simple, low-complexity, and data-free oblivious algorithm for selecting filters to prune. In addition, we extend the FR framework to include nonzero filter alternatives, leveraging a best-approximation technique to construct optimal replacements for the pruned filters. Extensive experiments on benchmark networks and datasets validate the effectiveness of our method. The proposed approach achieves state-of-the-art results, with a complexity comparable to basic techniques such as $l_{2}$ -norm pruning. Notably, our pruning method achieves 76.52% accuracy (ACC) in ResNet-50 on the ImageNet dataset, surpassing the baseline of 75.15%, while reducing network parameters by 25.5%. Our proposed resource efficiency (RE) metric assesses that the layer interdependence-aware pruning (LIAP) method is up to $10^{11}$ times more efficient than existing techniques, setting a new standard for resource-aware CNN pruning.

In today's machine learning (ML), the need for vast amounts of training data has become a significant challenge. Transfer learning (TL) offers a promising solution by leveraging knowledge across different domains/tasks, effectively addressing data scarcity. However, TL encounters computational and communication challenges in resource-constrained scenarios, and negative transfer (NT) can arise from specific data distributions. This article presents a novel focus on maximizing the accuracy of instance-based TL in multisource resource-constrained environments while mitigating NT, a key concern in TL. Previous studies have overlooked the impact of resource consumption in addressing the NT problem. To address these challenges, we introduce an optimization model named multisource resource-constrained optimized TL (MSOPTL), which employs a convex combination of empirical sources and target errors while considering feasibility and resource constraints. Moreover, we enhance one of the generalization error upper bounds in domain adaptation setting by demonstrating the potential to substitute the divergence with the Kullback-Leibler (KL) divergence. We utilize this enhanced error upper bound as one of the feasibility constraints of MSOPTL. Our suggested model can be applied as a versatile framework for various ML methods. Our approach is extensively validated in a neural network (NN)-based classification problem, demonstrating the efficiency of MSOPTL in achieving the desired trade-offs between TL's benefits and associated costs. This advancement holds tremendous potential for enhancing edge artificial intelligence (AI) applications in resource-constrained environments.

Today’s machine learning applications in the healthcare sector often require a vast amount of training data and lengthy training, which are not always available. Transfer learning (TL) offers a promising solution by leveraging knowledge across different domains/tasks, effectively addressing data scarcity. However, TL encounters computational and communication challenges in resource-constrained scenarios, and negative transfer (NT) can arise, posing significant challenges for sensitive tasks such as stroke prediction. This paper presents a novel method to maximize the accuracy of TL in resource-constrained environments while mitigating NT, a key concern in TL. Previous studies have overlooked simultaneously the resource consumption and the NT problem in the network-based TL era. To address this shortcoming, we formulate an optimization model named RCTL, which employs the empirical fine-tuning error of the target task as the objective while considering NT overcoming condition and resource limitations as the feasibility constraints. Our approach is extensively validated in a stroke prediction scenario, demonstrating the efficiency of RCTL in achieving the desired trade-offs between TL’s benefits and associated costs. The obtained results show the potential of enhancing edge AI applications in resource-constrained healthcare infrastructure.

Deep Reinforcement Learning (DRL) solutions to 5G problems often face with communication unreliability issues due to imbalanced state-space distributions and the scarcity of rare samples. Generative Adversarial Network (GAN) is promising to improve DRL reliability. However, employing GANs in resource-constrained edge environments is very challenging due to their heavy resource consumption. Previous general resource allocation models for training neural networks do not consider GAN quality requirements such as the minimum number of training samples. We propose an architecture for sharing edge and cloud resources among multiple GANs, then formulate an optimization model, named OGAN, to maximize DRL reliability with respect to resource constraints for training GANs and fine-tuning DRLs. OGAN allocates resources for training several GANs and DRLs concurrently based on an upper bound error. Difference convex programming is then used to solve this mixed-integer non-linear model. Our experimental results show that OGAN improves the overall system reliability and performance by 23 % and 22 %, respectively, compared to baselines.

Artificial neural networks (NNs) are widely used in many modern applications, including signal processing and communication systems. Conventionally, NNs are trained by different forms of stochastic gradient descent (GD) algorithm. However, since NN optimization cost functions are non-convex, training NNs with GD has fundamental common shortcoming of all non-convex problems. To address this issue, in this paper, we take advantage of the difference of convex (DC) programming as an innovative approach for smooth/non-smooth non-convex optimizations. We model the training of NN as a DC problem and propose DC programming as an alternative optimization technique to find NN parameters. Furthermore, we obtains the convex components of the DC function. In particular, we efficiently compute convex components of regression and binary classification cost functions by means of convex analysis tools. We verify our proposed model by comparing its result with the conventional gradient descent optimizer. Simulation results confirm that the superiority of the proposed DC programming approach over GD.

Transfer learning (TL) reduces the training overheads by transferring knowledge across domains/tasks. However, the advantages of TL come with computation and communication costs. Therefore, the decision to transfer knowledge between learners should be optimized while at the same time avoiding negative transfer (NT), i.e. when the source information does not improve but rather degrades the learning performance in the target. In this paper, we propose a new notion namely, regret of learner (RoL) as a quantitative measure for the learner's performance, computation costs and communication resources of TL. Then, we use a convex combination of the empirical source and target errors with respect to the feasibility and resource constraints to design an optimization model called OPTL that deploys a TL model in a resource-constrained environment to avoid NT. This model can be employed as a general framework for different ML methods and various communication scenarios and use cases by changing the unification parameters. To validate our approach, we use OPTL for optimized TL in a deep learning (DL)-based classification problem. Extensive experiments confirm the efficiency of our proposed method.

Recently, transfer learning (TL) has emerged as a powerful machine learning method in distributed environments. Transferring the knowledge between distributed agents helps reduce both learning time and computing costs. However, in a communication system, the advantage of TL comes with communication costs. To make an optimal decision of transfer between two agents, we try to answer three key questions: i) which information should be transferred from a source to a target?, ii) how this transferred information will be adapted to the target? and iii) when should TL be triggered to optimize the costs?. To this end, we introduce a new concept of similarity based on the Best Approximation Theory and a general transfer rule. Then, we propose a model to evaluate the feasibility and optimality of TL. We verify our proposed model in the context of the wireless channel selection problem using contextual multi-armed bandits. Experimental results show optimal TL decisions can be made, and Extra Action is an efficient technique for TL in channel selection.

A key challenge facing any channel selection technique is the dynamic nature of wireless channels. To address this issue, reinforcement learning techniques have widely been used, e.g., contextual multi-armed bandit (CMAB) theory. In fact, prior works solved the problem at each individual node. However, they did not consider the cooperative learning techniques, e.g., transfer learning. In communication systems, the advantage of transfer learning comes with computation and communication costs. Therefore, the decision of transferring the knowledge between agents should be optimized. In this paper, we develop a model to evaluate the feasibility and optimality of transfer learning for CMAB-based channel selection in communication systems. To this end, we introduce a utility model for evaluating these economical aspects. Leveraging Best Approximation Theory, we propose a new similarity concept and a transfer rule applied in the context of channel selection. Experimental results show that Extra Action is an efficient technique for transfer learning in a channel selection regime. More importantly, our proposed utility and optimization model is shown to be a powerful framework for deciding when transfer learning is feasible, and when it is optimal.