Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing whitebox adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and statespecific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the adversarial victim-dynamics Markov decision process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose a selective state-aware reinforcement adversarial attack method, named STAR, to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks

Deep Reinforcement Learning (DRL) suffers from uncertainties and inaccuracies in the observation signal in real-world applications. Adversarial attack is an effective method for evaluating the robustness of DRL agents. However, existing attack methods targeting individual sampled actions have limited impacts on the overall policy distribution, particularly in continuous action spaces. To address these limitations, we propose the Distribution-Aware Projected Gradient Descent attack (DAPGD). DAPGD uses distribution similarity as the gradient perturbation input to attack the policy network, which leverages the entire policy distribution rather than relying on individual samples. We utilize the Bhattacharyya distance in DAPGD to measure policy similarity, enabling sensitive detection of subtle but critical differences between probability distributions. Our experiment results demonstrate that DAPGD achieves SOTA results compared to the baselines in three robot navigation tasks, achieving an average 22.03% higher reward drop compared to the best baseline.

Deep reinforcement learning (DRL) has emerged as a promising approach for robotic control, but its real-world deployment remains challenging due to its vulnerability to environmental perturbations. Existing white-box adversarial attack methods, adapted from supervised learning, fail to effectively target DRL agents as they overlook temporal dynamics and indiscriminately perturb all state dimensions, limiting their impact on long-term rewards. To address these challenges, we propose the Adaptive Gradient-Masked Reinforcement (AGMR) Attack, a white-box attack method that combines DRL with a gradient-based soft masking mechanism to dynamically identify critical state dimensions and optimize adversarial policies. AGMR selectively allocates perturbations to the most impactful state features and incorporates a dynamic adjustment mechanism to balance exploration and exploitation during training. Extensive experiments demonstrate that AGMR outperforms state-of-the-art adversarial attack methods in degrading the performance of the victim agent and enhances the victim agent’s robustness through adversarial defense mechanisms.

Developing an effective evaluation metric is crucial for accurately and swiftly measuring LiDAR perception performance. One major issue is the lack of metrics that can simultaneously generate fast and accurate evaluations based on either object detection or point cloud data. In this study, we propose a novel LiDAR perception entropy metric based on the probability of vehicle grid occupancy. This metric reflects the influence of point cloud distribution on vehicle detection performance. Based on this, we develop a LiDAR deployment optimization model, which is solved using a differential evolution-based particle swarm optimization algorithm. A comparative experiment demonstrated that the proposed PE-VGOP offers a correlation of more than 0.98 with the vehicle detection results in evaluating LiDAR perception performance. Furthermore, compared to base deployments, field experiments indicate that the proposed optimization model can significantly enhance the perception performance of various types of LiDARs, including RS-16, RS-32, and RS-80. Notably, it achieves a 25% increase in detection Recall for the RS-32 LiDAR. Additionally, sensitivity analysis under varying traffic densities further verifies the robustness of the proposed model. This study provides a practical and generalizable framework for enhancing roadside LiDAR deployment in diverse traffic environments.

Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.

Traffic flow prediction is an essential part of intelligent transportation systems, which enhance traffic mobility and safety while also increasing traffic management efficiency. However, due to the complex factors, such as the nonlinear and dynamic nature of traffic flow data, effectively capturing the spatial and temporal characteristics of the data becomes a major challenge. In addition, accurate predictions are essential to guide traffic travel. Thus, we present a traffic flow prediction network in this paper designed to effectively capture spatial, long-term, short-term, and periodic characteristics. To this end, we use a convolutional neural network and an attention mechanism-based multi-layer LSTM-SNP model to develop a hybrid model, referred to as CAMLP. The model extracts spatial and short-term features effectively and focuses on key information through the attention mechanism, thereby improving prediction accuracy. Secondly, in order to capture long-term and periodic features, we utilize the BI-LSTM-SNP model and attention mechanism to process long-term data and periodic data. The extracted long-term, periodic, and short-term features are then combined for prediction. The design of the model fully considers the dynamic principles of time-varying traffic flow and the physical correlation between different road sections so that the model can more accurately reflect the dynamic changes of traffic flow. Finally, the proposed model is tested on three public datasets, and the experimental results show that the proposed model achieved the best performance among the compared models, thus confirming its effectiveness.

Traffic object detection based on computer vision (CV) can usually be deployed on the embedded computing platform of autonomous vehicles or unmanned aerial vehicles (UAVs), to provide critical information about traffic scenes for autonomous driving or traffic management. However, due to limited computing resources, there is a need for small, lightweight, and reliable object detectors. As an emerging technology, spatial pyramid pooling methods have great potential in improving the detection performance of real-time object detectors. Most of the existing works focus on the development of more complex spatial pyramid pooling methods for higher accuracy, but real-time performance is also important in the everchanging traffic scene. Thus, to balance the tradeoff between real-time detection and accuracy, we design a solution for real-time traffic object detection: a novel real-time object detector, named ESPPNet. Specifically, we propose an efficient plug-and-play spatial pyramid pooling method (ESPP). The method consists of a progressive spatial pyramid pool structure (PSPP) and a multi-scale feature enhancement module (MFEM). We first use PSPP to capture multi-scale feature maps with richer nonlinear features. Then, MFEM is used to establish effective long-range dependencies for multi-scale features. Experimental results on the VisDrone and SODA10M public datasets demonstrate that our method can achieve better real-time performance, less resource utilization, and higher accuracy, compared with other state-of-the-art methods.

Real-time object detection is an essential part of various Internet of Things (IoT) applications. Unmanned aerial vehicles (UAVs) employ visual sensors to capture high-definition images to detect objects of interest. However, current research on UAV detectors mainly focuses on developing more sophisticated network architectures, with little attention paid to the limitations of UAV computing resources. In this work, we present an end-to-end split-and-shuffle detector, named SCSDet. Unlike the mainstream detector designs that heavily rely on bottleneck structures, our method is based on inexpensive split-and-shuffle operations. It encourages the detector to avoid unnecessary transformation layers for channel down-sampling, thereby minimizing memory and computation cost. This is rarely studied in detector architecture design. Specifically, we first design a lightweight backbone structure (SCSNet) based on split-and-shuffle, which allows frequent interaction between different gradient information to capture more useful non-linear features for small-scale objects at a low cost. Next, we construct an efficient receptive field module (ERFM) to generate richer multi-receptive field expressions for the initial feature space. It significantly alleviates the adverse effects of single receptive field size on the capability of the detectors to detect small-scale objects. Finally, we propose a grouped local attention convolution (GLAConv), which utilizes local sliding windows with different coverage rates to adaptively learn channel and spatial attention. This makes the detector to focus on the foreground. Experimental results show that our method achieves high accuracy with low complexity in UAV object detection.

Digital twin (DT)-empowered platoon can improve platoon management efficiency and driving safety. However, the resource allocation scheme of low-latency platoon DT (PDT) and the interactions with platoon control strategy are important issues in the study of PDTs. In this article, we study the resource allocation in the PDT network and the interaction mechanism between PDT and platoon control for an unmanned aerial vehicle (UAV)-hosted PDT. We introduce the Age of Information (AoI) metrics to characterize the freshness of the DTs. To explore the impact of the PDT resource allocation scheme on the platoon control strategy, we propose a joint optimization model for power resource allocation and platoon control. Specifically, the allocation of power resources affects the PDT’s AoI, and the high-latency PDT in turn affects the platoon control strategy. Our objective is minimize the weighted sum of the system’s average energy consumption and the PDT’s average peak AoI. To solve the problem, we first reformulate the power resource allocation problem over a period of time as a Markov decision process (MDP) model, and then propose the Dirichlet deep deterministic policy gradient (DDPG)-based power allocation (D3PGPA) method based on Dirichlet distribution and DDPG algorithm. The method can not only effectively explores the state space while satisfying the constraints of limited resources but also improve the stability of the algorithm. Numerical results show that the D3PGPA method can host a PDT with low AoI and improve the stability of the platoon. Besides, our proposed method performs stably and outperforms other benchmark methods.

With the development of Internet of Vehicles (IoV) technology and the application of artificial intelligence-based algorithms, platoon driving based on connected autonomous vehicles (CAVs) has become one of the effective solutions to reduce environmental pollution and improve traffic safety. However, the connectivity, autonomy, and passenger comfort in platooning vehicles cannot be realized without the support of advanced communication technologies and auxiliary computing. In this work, we research the problem of computation offloading and resource allocation within a platoon network. Considering the comprehensive effects of vehicle mobility, co-channel interference, and multivehicle cooperation, we propose a system optimization model for joint computation offloading and power allocation (COPA). Our objective is to minimize the weighted sum of the system average energy consumption and task data processing delay. In the dynamic platoon network, we design a multiagent deep deterministic policy gradient (DDPG)-based joint COPA scheme, which can learn the temporal correlation of environment states and make more accurate power allocation actions. Moreover, we conduct extensive computer simulations to demonstrate the robustness and effectiveness of the DDPG-based COPA scheme. Numerical results demonstrate that the proposed scheme has a better performance compared with other benchmark schemes.

The rapid development of self-driving cars and breakthroughs in key technologies have made the truck platoon possible. In addition to reducing truck fuel consumption and air pollution by reducing air resistance, effective platoon strategies can also maximize highway throughput while improving driving safety. However, the truck platoon strategy’s current resource allocation model is still in the preliminary research stage. Therefore, inspired by the successful experience of deep reinforcement learning (DRL) in solving resource allocation problems, this article proposes a dynamic resource allocation model for the truck platoon based on the semi-Markov decision process (SMDP) and DRL, which is used to maximize system revenue when considering the resource cost and income balance of the transportation system. Precisely, the proposed method first models the process of controlling the dynamic in and out of the truck platoon as SMDP. The action value in a specific state obtained by the planning algorithm is used as a DRL sample for model training. Finally, the SMDP is optimized through the trained model to obtain a truck platoon resource that approximates the optimal strategy distribution plan. The experimental results show that compared with the traditional greedy algorithm, value iteration, and${Q}$-learning scheme concerning solving the dynamic resource allocation model of the truck platoon, the Deep${Q}$-Network (DQN) used in this article can reduce the probability of request processing delay while causing the system to obtain higher rewards.

Currently, available contactless continuous authentication (CA) techniques depend on physiological biometrics to identify individuals at long intervals through complex feature extraction, resulting in poor accuracy, high computation costs, and security vacuums during lengthy intervals. To address these issues, we propose WiPT, a WiFi-based contactless CA system that utilizes contextual features and behavioral biometrics to optimize contactless CA technology. Specifically, we designed a low-computation two-step user state detection (TUSD) mechanism that continuously monitors user states in real-time. It locks the system when the registered user leaves and allows user authentication only when the departing user returns. Therefore, it eliminates pointless periodic re-authentication and results in considerably shorter monitoring intervals while significantly reducing computation. Subsequently, benefiting from contextual features, WiPT can identify individuals using more detectable behavioral biometrics. We built a one-class classification model based on the Convolutional Autoencoder to automatically extract rich representations of WiFi signals associated with postural transition movements, resulting in lower authentication delay, higher accuracy, and anti-interference. WiPT was implemented by the widely available 802.11n devices and has been extensively evaluated with typical sit-to-stand postural transitions. WiPT achieves an average accuracy of 96.63% in authentication and 99.78% in defense across 30 subjects with an authentication delay of 5.59 milliseconds and a monitoring interval of 2 seconds. They are 4.87% and 5.03% more accurate and dozens of times less time-consuming than existing WiFi-based CA solutions.

As an important application scenario of the industrial Internet of things, the Internet of Vehicles can significantly improve road safety, improve traffic management efficiency, and improve people's travel experience. Due to the high dynamics of the Internet of vehicles environment, the traditional resource optimization technologies cannot meet the requirements of the Internet of vehicles for dynamic communication, computing and storage resources optimization management, and artificial intelligence algorithms can adaptively obtain dynamic resource allocation schemes through self-learning. Therefore, adopting artificial intelligence techniques to optimize the dynamic resource of the Internet of Vehicles is the research focus of this article. In this article, we first model the Internet of Vehicles resource allocation problem as a semi-Markov decision process that introduces a resource reservation strategy and a secondary resource allocation mechanism. Then, the reinforcement learning algorithm is used to solve the model. Thereafter, it theoretically analyzes the joint optimization of computing and communication resources, models it as a hierarchical architecture, and uses hierarchical reinforcement learning to obtain the optimal system resource allocation plan. Finally, the results of simulation experiments show that the dynamic resource allocation scheme of the Internet of vehicles based on the reinforcement learning in this article greatly improve resource utilization and user quality of experience with guaranteeing system quality of service compared with the traditional greedy algorithm.

Heterogeneous computing powered by remote clouds and local fogs is a promising technology to improve the performance of user terminals in the Internet of Things. In this paper, two semi-Markov decision process (SMDP)-based coordinated virtual machine (VM) allocation methods are proposed to balance the tradeoff between the high cost of providing services by the remote cloud and the limited computing capacity of the local fog. We first present a model-based planning method in which it is necessary to train the state transition probabilities and the expected time intervals between adjacent decision epochs. To facilitate training them, the SMDP is degraded into a continuous-time Markov decision process (CTMDP) in which the service requests and ongoing service completions follow a continuous-time Markov chain. The relative value iterative algorithm for the CTMDP is used to find an asymptotically optimal VM allocation policy. In addition, we also propose a model-free reinforcement learning (RL) method, where an optimal coordinated VM allocation policy is approximated by learning from the states and rewards of feedback. The simulation results show that the performance of the model-free RL method can converge to a level similar to that of the model-based planning method and outperform the greedy VM allocation method.

Energy harvesting (EH) becomes more desirable to save the world's energy consumption. This paper investigates energy resource allocation problem for EH networks. We propose a resource allocation framework based on a Semi-Markov Decision Process (SMDP). The objective of the framework is to provide a solution for a throughput maximization problem in EH networks by maximizing the total long-term expected reward of the EH system. The system reward is derived by considering both the income and the cost of the EH wireless communications. The numerical results illustrate a significant expected reward performance gain over a Greedy scheme. Moreover, simulations illustrate that the proposed approach is efficient and provides important guidelines for network deployment and resource management in a green radio network with EH technology.

Resource virtualization is considered as a potential technique to cope with the mushroomed multimedia services. With resource virtualization, physical resources are characterized as virtual resource shared by various Service Providers (SPs). In this paper, we focus on the virtual resource allocation problem for multimedia service in 5G network. The system rewards is maximized with the considerations of the system virtual resource consumption and the incomes generated from mobile users. To address this issue, a Multimedia Service Admission Model (MSAM) based on Semi-Markov Decision Process (SMDP) is proposed to characterize the system rewards. Then the steady-state probability and service request blocking probability are derived. Numerical results show that theoretic results are consistent with the simulation results.

This study investigates the framework of associating multiple‐input–multiple‐output (MIMO) beamforming with secure code to achieve unconditional secure communications in the wireless passive eavesdropping environment. The schemes are based on a two‐step method under Wyner's wiretap channel model. First, with MIMO transmit beamforming, one can utilise the spatial degree of freedom to cripple eavesdroppers’ interceptions even when he does not know the eavesdropper's channel state information. Consequently, by taking the threshold characteristics of the secure code, the legitimate receivers will continue to extend an average bit error rate advantage over eavesdroppers when they share similar conditions (background noise power and channel gains). By this way, the proposed system could achieve almost zero information obtained by the eavesdroppers while still keeping rather lower error transmissions for the main channel. A profound theoretical analysis for the MIMO advantage channel and the exact closed‐form expressions of secrecy outage probability for the secure code joint system are presented. The authors launch extensive experiments to verify the proposed security systems and demonstrate its feasibility and implement ability.

A novel sparse domain-based information hiding framework is proposed in this paper to attach the watermarking signal to the most significant sparse components of the host signal over the pre-defined overcomplete dictionary. The adaptive sparse domain can be utilized to embed watermarking logo with better security and robustness. This can be realized owing to the fact that, not only the sparse domain can be customized from the given samples, but also the sparse transform coefficients of the original watermarking signal can be embedded, which provides inherent privacy. This paper provides two kinds of methods that embed watermark directly and embed the sparse representation coefficients of watermarking logo, and analyzes the condition of uniqueness of the sparse solution. Experimental results demonstrate the superiority of the proposed sparse domain digital watermarking technique over the traditional frequency domain or spatial domain schemes.

The multimedia communication technologies have been widely used in the anti-money laundering (AML) field to improve the efficiency and security of the business transactions. To reduce the cost of massive multimedia processing and communications, it is desirable to allow multiple financial industries (FIs) to share the AML resources, including human, computation, communication, and storage resources, and cooperate with each other to complete the transaction tasks. In this paper, the optimal AML resource management among peer FIs towards the maximal AML rewards is studied. Specifically, an AML resource allocation model (AMLRAM) based on semi-Markov decision process(SMDP) is proposed, where the system state is represented by a tuple, i.e., the number of High-Risk Operation (HRO), the number of Low or Moderate Risk Operation (L/MRO), and the current event type (i.e., the arrival of HRO or L/MRO suspicious transaction report which needs to be further checked, and the departure of HRO or L/MRO suspicious transaction report which has been checked and releases the occupied AML resource) in the AML field. The maximal long-term rewards of the system is derived, and the optimal AML resource allocation decision among peer FIs is made to achieve the maximal system rewards. Extensive simulations validate our analysis.

Optical orthogonal codes (OOCs) are widely used as spreading codes in optical fiber networks. In this paper, a bound on the code size of 2-D OOCs with both at most one-pulse per wavelength (AM-OPPW) and at most one-pulse per time slot (AM-OPPTS) is derived. Accordingly, two constructions of optimal 2-D OOCs with both AM-OPPW and AM-OPPTS are proposed via the generalized cyclotomic classes. Furthermore, optimal 2-D OOC with AM-OPPW can be also constructed by adding more codewords into the 2-D OOCs with both AM-OPPW and AM-OPPTS.

With Internet environment is getting optimized and users preferring mobile communications, Cloud Service Providers (CSP) aim to provide services to users depending on their geographic locations with higher service availability and faster access speed. Mobile cloud computing falls into this category, where mobile users can move around and request cloud services at any given geographic locations. To build such a geographic-based mobile cloud services, an effective mobile cloud resource allocation and service request scheduling scheme is highly desired. To this end, the presented service request scheduling scheme takes a comprehensive approach by considering system parameters from both CSP and mobile users such as computation, energy, connectivity, service payment, mobile users' satisfaction, etc. Finally, the performance evaluation of the proposed scheduling scheme is evaluated through simulations where the results show that the presented scheme achieves better system overall gain compared to traditional over-provisioning approaches.