Selfish nodes will provide less computing resources or steal data, which degrade the performance of vehicular edge computing (VEC). In this paper, we design a secure offloading mechanism for VEC against selfish nodes, which formulates a reputation evaluation function based on the task latency requirements and the data protection level. Further, we design a safety-guide-based multi-agent reinforcement learning algorithm for vehicles to select the BS and sub-channel of the underlying tasks. Specially, the state including the BS reputations, channel gain, task latency and local computational capability is used as the fundamental to optimize the secure offloading policy. Our proposed algorithm designs a safety guide to measure the risk of the choosing policy based on a designed penalty, including the evaluation of the transmission collisions and task latency requirements. To make a balance between the optimization exploitation and safe exploration, this algorithm uses the penalty as one of the metrics to update the policy selection network. Experimental results verify that the proposed scheme outperforms the benchmark with less task latency and higher system energy efficiency.

Deep learning-based steganography techniques mainly rely on large-scale datasets with sufficient training and testing images, but the concealment performance may degrade in the few-shot-based wireless scenarios, such as vehicular networks. In this paper, we propose a few-shot image steganography framework with deep reinforcement learning (RL), which applies an adaptive generative adversarial network (GAN) to embed secret messages in cover images. This is the first work combining hierarchical deep RL with image steganography. We design an improved three-level hierarchical structure for deep RL, which determines the encoder in the GAN to improve the image quality and similarity, the number of cover images to form a few-shot training dataset, and the secret embedding depth to improve concealment performance and reduce overhead. Experiments are performed on both grayscale and RGB images from BOSSbase and Div2K datasets, demonstrating that the proposed framework surpasses SteganoGAN and HiNet benchmarks with better concealment performance and higher ability against steganalysis.

Deploying high-complexity deep neural network (DNN) on mobile devices presents significant challenges, stemming from the conflict between their computationally intensive demands and the constrained computational resources. Device-to-device ($D$2$D$) assisted mobile edge computing (MEC) exploits sharing resources among devices to perform DNN inference collaboratively for higher efficiency. However, most existing collaborative inference schemes ignore the structure of DNN models, thus suffering from high latency under dynamic network conditions and computational loads. In this paper, we propose a learning-based low-latency collaborative inference scheme for multi-branch models, which schedules the computations of each DNN layer to optimize the alignment between the DNN structure's characteristics and the D2D network environments. Specifically, the computational task of each branch is assigned to nearby mobile devices or the edge server for efficient collaboration. In addition, we design proximal policy optimization with task-specific architecture to choose the computational scheduling policy, which contains a two-stream NN to extract features from the D2D network and the tasks at each layer, as well as a multi-head output to obtain the collaborative devices for all tasks in the layer. Simulation results show that the proposed scheme reduces the inference latency compared with benchmarks.

Vehicular edge computing (VEC) helps improve the task computational performance of vehicles on roads but has difficulty in defending against eavesdropping and selfish attacks simultaneously. In this paper, we design a reputation-based smart contract with blockchain and propose a multi-agent reinforcement learning (RL) based secure offloading scheme for VEC against both eavesdropping and selfish attacks. This scheme has a three-level hierarchical structure for each vehicle and uses the reputations obtained from the blockchain as the basis to optimize the edge node selection, offloading ratio, and power allocation, which aims to reduce the task computational latency, the vehicle energy consumption and eavesdropping rate. By using a punishment function based on the constraints, this scheme avoids exploring dangerous policies that can cause task failure or severe data leakage. A multi-agent deep RL-based secure offloading scheme is proposed for vehicles with sufficient resources, which evaluates the long-term risk rather than the punishment function to further improve the secure offloading performance. The regret bound is analyzedand the cumulative reward upper bound is provided. Simulation results verify the effectiveness of our schemes as compared with the benchmark.

Distributed multi-exit neural networks (MeNNs) enable mobile devices to handle complex tasks such as image classification, but their performance is highly dependent on transmission quality and is therefore vulnerable to side-channel attacks. In this paper, we design a side-channel attack model and propose an efficient inference framework based on the distributed MeNN to resist the designed attack. First, we design an intelligent side-channel attack model, in which the attacker can eavesdrop on the communication channel and use deep reinforcement learning (RL) to predict the early exit decision of each sample. Next, we develop a defense method that employs a hierarchical and multi-agent RL to determine whether to infer locally or offload to a chosen early exit on the server, and to adjust the transmit power accordingly. We further propose a critic-guided safety mechanism that steers local agents away from risky policies that would cause inference failures or severe data leakage. We prove that our framework enforces a strict instantaneous security constraint and asymptotically achieves the optimum by deriving a regret bound. Extensive experiments on several datasets (including CIFAR‑10, CIFAR‑100, STL‑10, EMNIST, FMNIST, and Stanford Cars) show that our method reduces inference latency, improves classification accuracy, and significantly enhances robustness against side-channel attacks, as compared with two benchmarks SCAN and PCE.

Due to the different privacy and local model quality requirements for each participant, federated learning (FL) is vulnerable to membership inference attacks. To solve this issue, we propose a risk-aware reinforcement learning (RL)-based personalized differentially private FL framework. This framework uses local model accuracy and privacy loss as the constraints to satisfy the user’s personalized requirements. By designing a multi-agent RL, this framework optimizes perturbation policy including perturbation mechanisms and parameters (such as privacy budget and probabilistic relaxation). The goal of each participant is to improve global accuracy and reduce privacy loss, attack success rate, and short-term risk value. Firstly, the framework designs a two-level hierarchical policy selection module to choose the perturbation policy to accelerate learning speed. Secondly, our proposed framework designs a punishment function to evaluate short-term risk and an R-network to estimate long-term risk, which guarantees safe exploration. Thirdly, this framework formulates an improved Boltzmann policy distribution to increase the impact of risk, thus avoiding risky policies that may cause severe privacy leakage or local task failure. We also analyze the convergence performance and provide privacy analysis for both Gaussian and Laplace mechanisms. Experimental results based on the MNIST dataset demonstrate the effectiveness of our framework compared with benchmarks.

In low-altitude networks, unmanned aerial vehicles (UAVs) can offer services such as logistics, intelligence surveillance, and environmental monitoring, aided by base stations (BSs) with substantial computational resources. However, BSs must defend against malicious UAVs that may overload resources or launch denial-of-service attacks. In this paper, we formulate a blockchain-enabled access control model, which uses the UAV identities (IDs) and trajectories, positive and negative interactions with the BS to evaluate the reputations of UAVs. In the blockchain, the elected miner generates blocks containing UAV IDs, coordinates, interactions, and reputation values. To defend against malicious UAVs, this paper formulates a trusted computational resource allocation optimization problem, solved by safe reinforcement learning (RL) with a three-level hierarchical structure. Specifically, this method uses the designed structure to optimize the BS access control, resource allocations, and block size. In particular, we design an E-network to evaluate the long-term risk resulting from the chosen policy, which is used to refine the policy distribution for safe exploration. A modified reward function accounts for immediate risks, preventing short-term dangerous explorations that could lead to illegal access or computational failures. We prove the Lyapunov asymptotic stability of the proposed system and derive the reward upper bound. Simulation results show that our scheme can converge to the upper bound, outperform the benchmark, and validate the effectiveness via ablation experiments.

The rapid growth in the number of vehicles and types of services such as video transmission improves the quality-of-service (QoS) requirements and increases the difficulty in resisting eavesdropping attacks on the Internet of Vehicles (IoV). Existing video transmission schemes that either ignore the impact of eavesdropping attacks or have the full knowledge of the attack model have performance degradation in highly dynamic IoV systems. In this paper, we propose a reinforcement learning-based secure video transmission scheme for IoV systems, which jointly optimizes the access control policy for each vehicle (i.e., the selection of access nodes such as the base stations or unmanned aerial vehicles) and the corresponding transmit power level against active eavesdropping. This scheme uses the QoS and eavesdropping rate as the criteria to evaluate the long-term risk of each state-action pair, which is estimated by a designed deep Q-network to avoid the risky access control policies that cause severe data leakage or video transmission failure. Simulation results show that our scheme reduces the energy consumption, transmission latency, and eavesdropping rate compared with the benchmark.

Federated learning helps protect data privacy for Internet of vehicles (Io V) by selecting a number of participated nodes but suffers from performance degradation such as low model training accuracy in the highly dynamic and large-scale Io V systems under selfish attacks. In this paper, we propose a risk-aware reinforcement learning based federated learning framework against selfish attacks for Io V,which jointly optimizes the training policy (i.e., the selection of participated vehicles and the corresponding local training data size) based on the state including the global model training accuracy, local model quality, training latency, data rate, and participation rate. By designing a punishment function to evaluate the immediate risk of each choosing training policy, this scheme avoids risky policies that result in extremely low training accuracy and high training latency to satisfy the requirements of local tasks such as the quality of service requirements. An evaluated neural network involved fully connected layers is designed to fast extract the global and local training features and thus accelerate the convergence speed. Experimental results based on both the MNIST and CIFAR-10 datasets verify that our scheme outperforms the benchmarks with higher training accuracy and less training latency.

Matrix computation is a crucial mathematical tool in scientific fields such as Artificial Intelligence and Cryptographic computation. However, it is difficult for resource-limited devices to execute large-scale matrix computations independently. Outsourcing matrix computation (OMC) is a promising solution that engages a cloud server to process complicated matrix computations for resource-limited devices. However, existing OMC schemes lack public verifiability, and thus resource-limited devices cannot verdict the correctness of the computing results. In this paper, for the first time, we propose a smart contract-based OMC scheme that publicly verifies the outsourcing matrix computation results. In our scheme, a smart contract running over the blockchain serves as a decentralized trusted third party to ensure the correctness of the matrix computation results. To overcome the Verifier's Dilemma in the blockchain, we present a blockchain-compatible matrix verification method that decreases the time complexity from$O(n^{3})$to$O(n^{2})$by utilizing a blinding method with the check digit and padding matrices. We make the verification become the form of comparing whether two results are identical rather than naive re-computing. Finally, we perform experiments on Ethereum and ARM Cortex-M4 and give in-depth analysis and performance evaluation, demonstrating our scheme's practicability and effectiveness.

Federated learning (FL) that improves data privacy reduces the computational overhead for Internet of Vehicles (IoV) systems but has difficulty in defending against selfish attacks due to the restricted quality of service requirements and the high mobility of vehicles. In this paper, we design a risk-aware hierarchical reinforcement learning-based FL framework for IoV to resist selfish attacks. By designing a two-level hierarchical policy selection module that consists of two deep neural networks, this framework divides the training policy into two sub-policies, i.e., the selection of FL participants and the corresponding local training data size, which are chosen based on the previous training performance and vehicle participation performance. This framework designs a risk-aware safety guide to avoid dangerous states such as local task failure resulting from risky training policies. Specifically, the guide uses a warning signal to evaluate the short-term risk of each state-action pair, applies an R-network to estimate the long-term risks for modifying the chosen training policy, and designs a punishment function for the modified training policy to revise the immediate reward to further enhance the safe exploration. We analyze the convergence performance and computational complexity of our scheme. Experimental results on MNIST, CIFAR-10, and Stanford Cars datasets verify the effectiveness of our scheme, including the global model accuracy, training latency, detection success rate, and convergence speed compared with the benchmarks FedAvg, MFL, DQNPS, and SHRL.

With the rapid growth in the number of high-mobility vehicles and booming enhanced applications with restricted latency requirements, downlink communication in Internet of Vehicles (IoV) systems has become increasingly vulnerable to active eavesdropping attacks. This paper proposes a federated learning-enabled secure communication framework for IoV against active eavesdropping, in which the roadside units (RSUs) apply reinforcement learning (RL) model to optimize their downlink transmit power levels, and the server helps update the RL models of the RSUs. First, we design a multi-agent deep RL algorithm for each RSU, which designs a punishment and a blacklist mechanism to mitigate risky explorations related to severe data leakage or communication outages. Second, this framework designs a risk-aware RL for the server, which uses a two-level hierarchical structure to choose the number of participated RSUs and the corresponding local training data size for higher optimization speed. This framework considers both the reward and risk in the selection of policies to reduce the probability of exploring the risky training policies that cause defense failure of the RSUs against active eavesdropping. Third, we analyze the convergence performance, computational complexity, and reward upper bound, which reveals how the power constraint, radio bandwidth and data size affect the secure communication performance. Simulation and experimental results validate the effectiveness of our schemes, such as the reductions of the eavesdropping rate, training latency, and the loss of local models compared to the benchmarks.

The trade-off between privacy and accuracy presents a challenge for current federated learning (FL) frameworks, hindering their progress from theory to application. The main issues with existing FL frameworks stem from a lack of interpretability and targeted privacy protections. To cope with these, we proposed Disentangled Federated Learning for Privacy (DFLP) which employes disentanglement, one of interpretability techniques, in private FL frameworks. Since sensitive properties are client-specific in nature, our main idea is to turn this feature into a tool that strikes the balance between data privacy and FL model performance, enabling the sensitive attributes to be private. DFLP disentangles the client-specific and class-invariant attributes to mask the sensitive attributes precisely. To our knowledge, this is the first work that successfully integrates disentanglement and the nature of sensitive attributes to achieve privacy protection while ensuring high FL model performance. Extensive experiments validate that disentanglement is an effective method for accuracy-aware privacy protection in FL frameworks.

In recent years, Ethereum has become a hotspot for criminal activities such as phishing scams that seriously compromise Ethereum transaction security. However, existing methods cannot accurately model Ethereum transaction data and make full use of the temporal structure information and basic account features. In this paper, we propose an Ethereum phishing detection framework based on temporal motif features. By designing a sampling method, we convert labeled Ethereum addresses into multi-directed transaction subgraphs with time and amount to avoid losing structure and attribute information. To learn representations for subgraphs, we define and extract the temporal motif features and general transaction features. Extensive experiments on Support Vector Machine, Random Forest, Logistic Regression, and XGBoost demonstrate that our method significantly outperforms all baselines and provides an effective phishing scams detection for Ethereum.

Federated learning (FL) emerges as a privacy-preserving paradigm to effectively integrate edge computing for the implementation of deep learning-based vehicular applications. Nevertheless, the incentive mechanism for the vehicles to participate with varied learning tasks, has not been well explored yet. In this paper, for the training control among vehicles, a multi-task FL framework is investigated, where multiple edge servers collectively coordinate numerous vehicular models from different learning tasks. Aiming at motivating the vehicles to actively participate in training while improving the system utility of multiple learning tasks, a problem of multi-task contract design is formulated, and an iterative reward allocation algorithm is proposed based on the property of the convergence performance and the contract constraints. Extensive experiments validate that the proposed algorithm can achieve the balance between the system utility and the cluster utility, with higher test accuracy.

Most safe reinforcement learning (RL) algorithms depend on the accurate reward that is rarely available in wireless security applications and suffer from severe performance degradation for the learning agents that have to choose the policy from a large action set. In this paper, we propose a safe RL algorithm, which uses a policy priority-based hierarchical structure to divide each policy into sub-policies with different selection priorities and thus compresses the action set. By applying inter-agent transfer learning to initialize the learning parameters, this algorithm accelerates the initial exploration of the optimal policy. Based on a security criterion that evaluates the risk value, the sub-policy distribution formulation avoids the dangerous sub-policies that cause learning failure such as severe network security problems in wireless security applications, e.g., Internet services interruption. We also propose a deep safe RL and design four deep neural networks in each sub-policy selection to further improve the learning efficiency for the learning agents that support four convolutional neural networks (CNNs): The Q-network evaluates the long-term expected reward of each sub-policy under the current state, and the E-network evaluates the long-term risk value. The target Q and E-networks update the learning parameters of the corresponding CNN to improve the policy exploration stability. As a case study, our proposed safe RL algorithms are implemented in the anti-jamming communication of unmanned aerial vehicles (UAVs) to select the frequency channel and transmit power to the ground node. Experimental results show that our proposed schemes significantly improve the UAV communication performance, save the UAV energy and increase the reward compared with the benchmark against jamming.

This study investigates how base stations (BSs) apply network coding to protect the downlink data and how drones relay the coded packets to resist active eavesdropping that performs jamming to induce the BS to raise the transmit power and thus steal more data. We present a drone-aided network coding framework for secure downlink transmission, which incorporates a random linear network coding algorithm to encode the BS messages against active eavesdropping. This framework designs a model-based reinforcement learning to choose the BS network coding and transmission policy based on the jamming power sent by the active eavesdropper, the previous transmission performance, and the BS channel states without the prior knowledge of the drone-eavesdropper channel states. The learning parameters such as the Q-values are updated by the real experiences in the downlink transmission process besides the simulated experiences that are generated from the virtual model in the designed Dyna architecture. Simulation results show that our proposed scheme outperforms the benchmarks in terms of the intercept probability, the transmission performance, and the BS energy consumption.

User profile perturbation protects privacy in the release of user profiles to receive recommendation services, in which the privacy budget as a privacy parameter can be controlled to effect a tradeoff between the recommendation quality and privacy protection against inference attacks. In this article, we propose a deep reinforcement learning (RL)-based user profile perturbation scheme for recommendation systems. This scheme applies differential privacy to protect user privacy and uses deep RL to choose the privacy budget against inference attackers. Based on an evaluated neural network (NN) and a target NN, this scheme enables a user device to optimize the privacy budget over time based on the sensitivity level of the clicked item, the similarities among the recommended items, and the estimated privacy loss. We provide an upper bound on the privacy protection performance of this scheme in the recommendation game and evaluate its computational complexity. Simulation results for a movie recommendation system show that this scheme increases the user privacy protection level for a given recommendation quality compared with benchmark schemes.

In controller area networks (CANs), electronic control units (ECUs) such as telematics ECUs and on-board diagnostic ports must protect the message exchange from spoofing attacks. In this paper, we propose a CAN bus authentication framework that exploits physical layer features of the messages, including message arrival intervals and signal voltages, and applies reinforcement learning to choose the authentication mode and parameter. By applying the Dyna architecture and using a double estimator, this scheme improves the utility in terms of authentication accuracy without changing the CAN bus protocol or the ECU components and requiring knowledge of the spoofing model. We also propose a deep learning version to further improve the authentication efficiency for the CAN bus. The learning scheme applies a hierarchical structure to reduce the exploration time, and uses two deep neural networks to compress the high-dimensional state space and to fully exploit the physical authentication experiences. We provide the computational complexity and the performance analysis. Experimental results verify the theoretical analysis and show that our proposed schemes significantly improve the authentication accuracy as compared with benchmark schemes.

Multiple-input and multiple-output (MIMO) radars are vulnerable to deceptive jamming launched by false target generators that send jamming signals with the goal of pretending that the radar echo signals are reflected by faked targets. In this paper, we present a reinforcement learning based energy efficient power control scheme to detect deceptive jamming for frequency diverse array MIMO radars. This scheme enables a radar to choose the transmit power over the antennas without relying on the known deceptive jamming model. Instead, based on the emergency level, the battery level, the echo signal quality, the antenna phase differences, the received jamming power and the previous detection error rate, this scheme improves the detection accuracy and the energy efficiency, and uses a Dyna architecture to train the learning parameters with the simulated jamming detection experiences for faster optimization in the dynamic game against deceptive jamming. Simulation results show that this scheme effectively improves the deceptive jamming detection accuracy and saves the radar energy. Index Terms-MIMO, radar, reinforcement learning, deceptive jamming.

Unmanned aerial vehicle (UAV) networks are vulnerable to jamming attacks because of the high mobility, limited battery and scarce spectrum resources of UAVs. In this paper, we propose a reinforcement learning based UAV relay scheme to improve the anti-jamming capability and save energy consumption of the UAV network. Based on the real-time channel conditions and the historical relay experiences, the proposed scheme enables UAVs to improve the policy of relay power and strategies without knowing the UAV network and channel model. Simulation results show that the proposed UAV relay scheme reduces the bit error rate of the messages and reduces the energy consumption of the UAV network compared with the state-of-the-art benchmark.

Mobile edge computing systems help improve the performance of computational-intensive applications on mobile devices and have to resist jamming attacks and heavy interference. In this paper, we present a reinforcement learning based mobile offloading scheme for edge computing against jamming attacks and interference, which uses safe reinforcement learning to avoid choosing the risky offloading policy that fails to meet the computational latency requirements of the tasks. This scheme enables the mobile device to choose the edge device, the transmit power and the offloading rate to improve its utility including the sharing gain, the computational latency, the energy consumption and the signal-to-interference-plus-noise ratio of the offloading signals without knowing the task generation model, the edge computing model, and the jamming/interference model. We also design a deep reinforcement learning based mobile offloading for edge computing that uses an actor network to choose the offloading policy and a critic network to update the actor network weights to improve the computational performance. We discuss the computational complexity and provide the performance bound that consists of the computational latency and the energy consumption based on the Nash equilibrium of the mobile offloading game. Simulation results show that this scheme can reduce the computational latency and save energy consumption.

Controller area networks (CANs) are vulnerable to spoofing attacks such as frame falsifying attacks, as electronic control units (ECUs) send and receive messages without any authentication and encryption. In this paper, we propose a physical authentication scheme that exploits the voltage features of the ECU signals on the CAN bus and applies reinforcement learning to choose the authentication mode such as the protection level and test threshold. This scheme enables a monitor node to optimize the authentication mode via trial-and-error without knowing the CAN bus signal model and spoofing model. Experimental results show that the proposed authentication scheme can significantly improve the authentication accuracy and response compared with a benchmark scheme.

With dynamic renewable energy generation and power demand, microgrids (MGs) exchange energy with each other to reduce their dependence on power plants. In this article, we present a reinforcement learning (RL)-based MG energy trading scheme to choose the electric energy trading policy according to the predicted future renewable energy generation, the estimated future power demand, and the MG battery level. This scheme designs a deep RL-based energy trading algorithm to address the supply-demand mismatch problem for a smart grid with a large number of MGs without relying on the renewable energy generation and power demand models of other MGs. A performance bound on the MG utility and dependence on the power plant is provided. Simulation results based on a smart grid with three MGs using wind speed data from Hong Kong Observation and electricity prices from ISO New England show that this scheme significantly reduces the average power plant schedule and thus increases the MG utility in comparison with a benchmark methodology.

Edge computing for mobile devices in vehicular ad hoc networks (VANETs) has to address rogue edge attacks, in which a rogue edge node claims to be the serving edge in the vehicle to steal user secrets and help launch other attacks such as man-in-the-middle attacks. Rogue edge detection in VANETs is more challenging than the spoofing detection in indoor wireless networks due to the high mobility of onboard units (OBUs) and the large-scale network infrastructure with roadside units (RSUs). In this paper, we propose a physical (PHY)- layer rogue edge detection scheme for VANETs according to the shared ambient radio signals observed during the same moving trace of the mobile device and the serving edge in the same vehicle. In this scheme, the edge node under test has to send the physical properties of the ambient radio signals, including the received signal strength indicator (RSSI) of the ambient signals with the corresponding source media access control (MAC) address during a given time slot. The mobile device can choose to compare the received ambient signal properties and its own record or apply the RSSI of the received signals to detect rogue edge attacks, and determines test threshold in the detection. We adopt a reinforcement learning technique to enable the mobile device to achieve the optimal detection policy in the dynamic VANET without being aware of the VANET model and the attack model. Simulation results show that the Q-learning based detection scheme can significantly reduce the detection error rate and increase the utility compared with existing schemes.

Vehicular ad-hoc networks (VANETs) are vulnerable to jamming attacks, and frequency hopping-based anti- jamming techniques are not always applicable in VANETs due to the high mobility of the onboard units (OBUs) especially under a large scale network topology. In this paper, we use unmanned aerial vehicles (UAVs) to deal with VANET jamming, especially smart jamming that changes the jamming policy based on the ongoing communication status of the VANET. More specifically, the UAV relays the data of OBUs to another roadside unit (RSU) with a better transmission condition if the serving RSU is located in a heavily jammed area. The interactions between the UAV and the jammer are formulated as an anti-jamming UAV relay game, in which the UAV decides whether or not to relay the data of the OBU to another RSU that is far away from the jammer, and the latter chooses the jamming power. The Nash equilibria (NE) of the game are derived to reveal how the best UAV relay strategy depends on the transmission cost and the radio channel model. A hotbooting policy hill climbing (PHC)-based UAV relay strategy is proposed to address jamming in the dynamic UAV-aided VANET game without the knowledge of network model and jamming model. Simulation results show that the proposed relay strategy can efficiently reduce the bit error rate (BER) of OBU data and thus increase the utility of VANET in comparison with a Q-learning based scheme.