Hybrid action space, which combines discrete choices and continuous parameters, is prevalent in domains such as robot control and game AI. However, efficiently modeling and optimizing hybrid discrete-continuous action space remains a fundamental challenge, mainly due to limited policy expressiveness and poor scalability in high-dimensional settings. To address this challenge, we view the hybrid action space problem as a fully cooperative game and propose a Cooperative Hybrid Diffusion Policies (CHDP) framework to solve it. CHDP employs two cooperative agents that leverage a discrete and a continuous diffusion policy, respectively. The continuous policy is conditioned on the discrete action's representation, explicitly modeling the dependency between them. This cooperative design allows the diffusion policies to leverage their expressiveness to capture complex distributions in their respective action spaces. To mitigate the update conflicts arising from simultaneous policy updates in this cooperative setting, we employ a sequential update scheme that fosters co-adaptation. Moreover, to improve scalability when learning in high-dimensional discrete action space, we construct a codebook that embeds the action space into a low-dimensional latent space. This mapping enables the discrete policy to learn in a compact, structured space. Finally, we design a Q-function-based guidance mechanism to align the codebook's embeddings with the discrete policy's representation during training. On challenging hybrid action benchmarks, CHDP outperforms state-of-the-art method by up to 19.3% in success rate.

Sharding protocols in cryptocurrencies are dedicated to improving system throughput and reducing cross-shard transaction latency. However, they face the challenge of state explosion due to their stateful design. An intuitive solution is state sharding. Existing state sharding protocols aim to reduce the incidence of cross-shard transactions and balance transaction load. Nevertheless, it inevitably introduces the complex and sequential processing of cross-shard transactions and neglects the security of state data. To address these challenges, we propose SecChain, a secure stateless sharded blockchain via a novel state commitment in this paper. SecChain fully exploits the historical characteristics of transactions and designs the parallel cross-shard transaction execution based on asynchronous prepaid accounts. We then introduce a novel state commitment scheme to ensure the security of state data and improve the efficiency of transaction validation. In addition, we optimize the on-chain and off-chain storage by integrating an incremental partial state trie on-chain and a full trie with proofs off-chain. We perform a comprehensive evaluation using real-world workloads and results demonstrate that SecChain's throughput is 1.75× that of BrokerChain, and its latency is reduced by 7×. Furthermore, SecChain significantly reduces both proof overhead and storage overhead.

Pseudonym certificates play a crucial role in providing authorized access in vehicular networks with fine-grained privacy demands. However, the revocation of pseudonym certificates in large-scale, resource-constrained context has posed a significant challenge. The global revocation status is susceptible to disruption by localized anomalous events. Moreover, as the scale of revocation grows, the synchronization process becomes increasingly vulnerable to attacks and incurs high overhead. To decouple the global revocation status from local revocation events, we propose a spatial-temporal pseudonym revocation and verification framework that supports batch pseudonym revocation within resilient revocation cycles. Revoked pseudonyms within the event area are locally filtered, while other pseudonyms are characterized by maintaining a proof to attest their consistent and legitimate status. To improve the efficiency of revocation status updates and mitigate centralization risks, a distributed caching and proof update strategy assisted by edge nodes is presented. Extensive simulations based on real-world large-scale datasets demonstrate that PV-STR scheme significantly reduces the vulnerability window, communication overhead, and memory consumption compared to state-of-the-art approaches.

Conventional entanglement routing approaches decide the Entanglement Paths (EPs) to establish Entanglement Connections (ECs) before trying to create Entanglement Links (ELs). By doing so, very few EL failures will result in a low network throughput. In this paper, we study how to choose the EPs to establish ECs after knowing which ELs are successfully created. This is called the Deferred EP Selection (DEPS) problem. DEPS is a generalized integer multi-commodity flow problem and we cannot solve it quickly with conventional optimization methods. To address this issue, we propose a Deep Reinforcement Learning based EP Selection (DRLEPS) approach. The salient features of DRLEPS include (i) by controlling the number of candidate EPs, DRLEPS can achieve a trade-off between time complexity and the EC establishment rate; and (ii) using candidate EPs as input, DRLEPS is robust to request variation; and (iii) by training neural networks with different topologies, a model derived by DRLEPS can be applied to various networks (even with a different number of nodes) without fine-tune. Through extensive simulations, we show that even in a network with 200 nodes, DRLEPS can solve the DEPS problem in 0.39 seconds with a Nvidia GeForce 3090 GPU. It outperforms the approach always establishing ECs through the EP with the largest success probability by up to 23.4% in EC establishment rate. It also outperforms the Integer Linear Programming (ILP) based scheme, which can achieve the maximum EC establishment rate, by up to 184.2x in network throughput.

Trajectory prediction plays a crucial role in autonomous driving systems, and exploring its vulnerability has garnered widespread attention. However, existing trajectory prediction attack methods often rely on single-point attacks to make efficient perturbations. This limits their applications in real-world scenarios due to the transient nature of single-point attacks, their susceptibility to filtration, and the uncertainty regarding the deployment environment. To address these challenges, this paper proposes a novel LiDAR-induced attack framework to impose multi-frame attacks by optimization-driven adversarial location search, achieving endurance, efficiency, and robustness. This framework strategically places objects near the adversarial vehicle to implement an attack and introduces three key innovations. First, successive state perturbations are generated using a multi-frame single-point attack strategy, effectively misleading trajectory predictions over extended time horizons. Second, we efficiently optimize adversarial objects’ locations through three specialized loss functions to achieve desired perturbations. Lastly, we improve robustness by treating the adversarial object as a point without size constraints during the location search phase and reduce dependence on both the specific attack point and the adversarial object’s properties. Extensive experiments confirm the superior performance and robustness of our framework.

Collaborative perception significantly enhances individual vehicle perception performance through the exchange of sensory information among agents. However, real-world deployment faces challenges due to bandwidth constraints and inevitable calibration errors during information exchange. To address these issues, we propose mmCooper, a novel multi-agent, multi-stage, communication-efficient, and collaboration-robust cooperative perception framework. Our framework leverages a multi-stage collaboration strategy that dynamically and adaptively balances intermediate- and late-stage information to share among agents, enhancing perceptual performance while maintaining communication efficiency. To support robust collaboration despite potential misalignments and calibration errors, our framework prevents misleading low-confidence sensing information from transmission and refines the received detection results from collaborators to improve accuracy. The extensive evaluation results on both real-world and simulated datasets demonstrate the effectiveness of the mmCooper framework and its components.

The multi-agent pathfinding problem seeks to generate low-cost paths for agents to reach their targets, playing a crucial role in advancing the development of smart warehousing. Traditional methods do not utilize neural networks, resulting in high computational costs and poor scalability. In contrast, learning-based approaches leverage reinforcement learning to handle large-scale scenarios more effectively. However, they often depend heavily on expert demonstrations, and their plain reward structures can lead to suboptimal path planning. To address these challenges, we propose a novel multi-agent reinforcement learning framework: Hierarchical Decision-making Pathfinding Based on Social Value Orientation (HIPS). Specifically, HIPS adopts a hierarchical decision-making structure in which the lower-level interaction policy generates actions to interact with the environment, guided by egoistic rewards to promote low-cost path planning. Moreover, the upper-level policy is environmentoriented, learning dynamic social value orientations that enable agents to plan paths while accounting for interactions with other agents, thereby achieving an adaptive balance between individual self-interest and long-term team benefits. Extensive experimental validation demonstrates that HIPS increases the number of finished targets by at least 6% compared to existing methods without relying on expert experience.

Federated learning (FL), as an emerging distributed machine learning paradigm, aims to achieve collaborative training of a global model without sharing clients’ local data, thereby addressing user privacy and data silo issues. However, due to the distributed architecture of FL and the invisibility of local training, it is highly susceptible to poisoning attacks. While numerous poisoning defense methods have been proposed, they often suffer from limited defense effectiveness and overly strong security assumptions. To address this problem, we propose FedTPD, a dual gradient evaluation-based defense method against poisoning attacks in FL. Specifically, we first design an adaptive representative gradient selection mechanism, which involves adaptively clustering local gradients and selecting representative gradients from different clusters. Secondly, we propose a client-assisted model evaluation mechanism, wherein a reliable client is selected as an evaluator to assess the representative gradients from various clusters. Finally, we develop a bias-corrected model aggregation mechanism that aggregates the local gradients of each cluster based on the trust scores of their representative gradients, thereby reducing the impact of poisoning attacks on the global model. Experimental results on three typical datasets demonstrate that the proposed FedTPD can effectively resist various poisoning attacks. In particular, compared with state-of-the-art methods, the global model’s maximum and average accuracies on the CIFAR-10 and CIFAR-100 datasets are improved by 2.79%–28.15% and 0.79%–31.34%, and 3.48%–22.18% and 2.56%–22.06%, respectively.

As LiDAR-based 3-D object detection gains attention, existing research on point cloud adversarial attacks has exposed vulnerabilities in 3-D neural network models, which can further impact the reliability of perception systems in autonomous driving. However, current adversarial attacks primarily focus on the point cloud classification tasks, while detection tasks are more challenging to attack because they involve the localization of multiple targets and the sparse distribution of objects. Existing methods often implement attacks by adding global perturbations to the point clouds, which results in poor attack performance and lack of imperceptibility. In this article, we investigate the robustness of 3-D object detectors against adversarial examples. We propose a novel imperceptible attack method to generate 3-D adversarial point clouds. First, we introduce the saliency map for the point clouds, assigning the unique value to each point to measure its importance to the model’s discrimination results. These salient points are then aggregated and adaptively matched to different attack areas corresponding to different targets. Next, we design an optimization-based attack algorithm to generate adversarial point clouds, which are supervised by a dual-loss function consisting of the detection loss to ensure attack effectiveness and the distance loss to limit gap with the original point cloud. Finally, we conducted experiments on two real-world datasets, the KITTI and Waymo Open datasets, to evaluate the proposed attack method. Extensive experiments demonstrate that our attack method achieves average attack success rates of 83.07% and 77.11% on two datasets against seven 3-D object detectors with minimal perturbations. Additionally, the generated adversarial point clouds exhibit strong transferability across multiple mainstream detectors.

In automated warehouses, efficient management and economic benefits hinge on the effective scheduling of automated guided vehicles (AGVs) to transport diverse packets. Emerging technologies such as artificial intelligence and automation control have greatly contributed to the development of packet transport schemes for AGVs. However, the development of the logistics industry results in a massive amount of packets with diverse deadlines, which brings new challenges for the AGV scheduling system. To address this, this paper treats each AGV as an agent and designs a novel hierarchical transformer reinforcement learning (HTRL) framework to generate efficient AGV scheduling policies. Specifically, this framework consists of one encoder and two decoders to produce the packet selection and path improvement actions. These two decoders are equipped with masked self-attention mechanisms to learn efficient packet selection and path improvement policies, facilitating AGV transport efficiency to meet the deadlines of packets. Moreover, we consider the kinetic features of AGVs and design a model predictive control (MPC)-based speed control method for AGVs to prevent frequent stop-and-wait of AGVs and enhance their transport efficiency. We build up a simulated warehouse environment containing packets with different deadlines and conduct extensive experiments. Experimental results validate that the proposed HTRL framework increases the delivered packets within expiration by up to 36.6% compared to other baselines.

Industrial cloud storage systems enhance data availability and offer intelligent services to enterprises. However, they also bring significant concerns about data integrity since the cloud space provider may not consistently retain the outsourced data. Existing cloud storage verification methods impose a significant computational burden on edge devices, so they are unsuitable for industrial cloud storage systems. Although the light-weight homomorphic authenticator somewhat alleviates the computational load on the fog node, its effectiveness remains limited and it further complicates proof generation and verification. To address these problems, we propose Ascina, a new proof of retrievability framework for industrial cloud storage systems. We employ verifiable secret sharing to delegate tag computation tasks to the computing server without disclosing the signing key. Once the fog node completes its initial configuration, no further computations are required. Additionally, we propose the improved Ascina by utilizing Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (IFFT) technology to further alleviate the computational burden on the fog node. Furthermore, we propose a batch verification algorithm that simultaneously validates the integrity of multiple files while maintaining the same security assurances as single auditing. We evaluate the performance of Ascina through experiments and compare it with state-of-the-art methods. Experimental results demonstrate that the computational overhead on the fog node in Ascina is 5×-251× lower than that of Edasvic and our proof verification time and proof size are reduced by 5×-16× and 977×-3285×, respectively. As the size and quantity of files grow, Ascina demonstrates greater efficiency in both time and space.

In vector commitments, the complex process of generating and updating proofs, along with the large-sized proofs, seriously hinders the practicality of stateless cryptocurrencies. In this work, we present NTTproofs, containing two sub-schemes, a vector commitment (VC) and a mulit-vector commitment (MC). Both sub-schemes are maintainable and aggregatable, and they also enjoy fast openings (i.e., generating all the proofs) as well as efficient proof updates. MC in NTTproofs employs the Fast Number Theoretic Transform (NTT) and sharding technique to significantly improve the time of generating all proofs by up to$0.76 \times $and$0.32 \times $, respectively, over Balanceproofs, Matproofs. Moreover, our proposed MC in NTTproofs is efficiently maintainable and requires merely 15.78 milliseconds at$n_{1}=n_{2}=2^{12}$to update all proofs. Meanwhile, NTTproofs schemes exhibit superior aggregatability, taking 0.003 seconds in VC and 0.05 seconds in MC to aggregate 1024 proofs and reducing the size of an aggregated proof to a constant size of 96 Bytes. Finally, macrobenchmarks indicate that our proposed MC in NTTproofs outperforms the other schemes, but is slightly inferior to that of Balanceproofs.

With the intelligence and automation of industrial Internet of Things, a new collaborative Cloud-Fog Automation paradigm has emerged. The emergence of federated learning (FL) has further enhanced the capabilities of Cloud-Fog Automation, making it possible to develop more secure and versatile collaborative industrial models. However, FL faces various security risks. More importantly, the security risks faced by FL when applied in Cloud-Fog Automation, along with corresponding security measures, have not yet been explored. To address this issue, we make an initial attempt to analyze the security of FL within the context of Cloud-Fog Automation, with the aim of facilitating the design of a more secure FL framework for this paradigm. Specifically, we first analyze the security risks that may be encountered at different phases, then analyze the challenges that need to be faced to resolve these risks. Subsequently, we conduct a systematic review of the state-of-the-art security solutions, and finally summarize the future research directions.

Effective multi-intersection collaboration is crucial for mitigating urban traffic congestion through reinforcement learning (RL)-based traffic signal control (TSC). Existing work mainly considers scenarios involving a single vehicle type, where cooperation is typically limited to neighboring intersections. However, in urban traffic scenarios where high priority vehicles coexist with ordinary vehicles, considering only a limited number of neighboring nodes may be insufficient to ensure the swift passage of high priority vehicles while minimizing the impact on overall traffic efficiency. Therefore, we formulate the multiple intersections’ decision-making process in urban scenarios as a Markov game and propose a novel centralized cooperative RL framework called MATLIT to solve the game. Specifically, we adopt a multi-agent transformer (MAT)-based architecture that facilitates efficient global cooperation among intersections. The attention mechanism and auto-regressive process of the MAT effectively mitigate the curse of the dimensionality problem, which guarantees MATLIT to tackle large-scale traffic scenarios. Meanwhile, the stability and sequence action generation capacity of the MAT-based architecture is further enhanced by incorporating MAT with a gated mechanism. Furthermore, considering the inherent topological constraints in urban traffic scenarios, we utilize graph attention networks (GATs) to capture graph-structured mutual influences. Additionally, in response to the urban traffic scenarios with various types of high priority vehicles that have time-varying priorities, we integrate the soft actor-critic (SAC) algorithm to enhance the exploration capabilities of our framework, allowing it to learn robust strategies in heterogeneous traffic conditions. Extensive experiments demonstrate that our proposed MATLIT framework outperforms all baselines and can reduce high priority vehicles’ waiting time by 24.57% while reducing the average waiting time of all vehicles by 18.51% in realistic urban scenarios.

To assess the vulnerability of deep neural networks in the physical world, many studies have introduced adversarial examples and applied them to computer vision tasks such as object detection in recent years. Compared to patch-based adversarial attacks, camouflage-based attacks have received more and more attention due to their ability to attack detectors from multiple viewpoints. However, existing adversarial examples often rely on glass-box models and exhibit limited transferability to closed-box models, which remains a significant challenge. To address this issue, we propose the highly transferable camouflage attack, a novel physical adversarial attack framework designed to generate robust and efficient adversarial camouflage that can mislead object detectors in diverse scenarios. Specifically, we introduce a distraction method to distribute the features of the attention map between models, and propose enhanced transfer strategies to improve adversarial transferability through augmenting the input data and the attacked models. Extensive experiments demonstrate that our highly transferable camouflage attack can effectively mislead object detectors in both digital and physical worlds, enhancing the transferability of adversarial camouflage on multiple mainstream detectors.

Distributed federated learning models are vulnerable to Byzantine failures because malicious nodes can alter the training direction of the global model, making it deviate from normal training and rendering the global model unusable. To address this issue, we propose a defense scheme based on differential privacy and a Complex Accuracy-based multi-Krum Algorithm (DP-CAKA). First, we construct poisoning attacks against local models and conduct attack experiments using three Byzantine-robust federated learning methods. These experiments significantly increase the error rate of the global model, thereby demonstrating the vulnerability of distributed federated learning systems under Byzantine failures. Subsequently, we add noise to the local gradients using differential privacy to enhance privacy during the aggregation process. In addition, we design a Complex ACC-based Multi-Krum Algorithm (CAKA) that selects a few optimal local gradients to participate in the aggregation, thereby mitigating the negative impact of differential privacy on model accuracy. Experimental results demonstrate that DP-CAKA achieves an effective trade-off between privacy and availability, improving global model accuracy by 7.4% over Krum and Trimmed, and 1.5% over Median under poisoning attacks, by 20.7% over Krum and 0.5% over Median under Gaussian-attacks, and by 11.8% over Trimmed mean and 0.19% over Median under Sign-flipping attacks.

In the landscape of automated warehousing, automated guided vehicles (AGVs) play a key role in enhancing operational accuracy and reducing labor costs. Optimizing the movement path of a group of AGVs, especially in time-sensitive environments, is crucial for maintaining efficiency and reducing costs. We describe this problem as timeliness-constrained multi-agent path finding (TC-MAPF). Although current multi-agent path finding (MAPF) approaches primarily focus on maximizing throughput, they typically overlook the critical need for adhering to stringent timeliness constraints associated with transportation tasks. Moreover, these methods depend heavily on carefully designed reward functions that are specific to particular environmental settings. This reliance constrains their flexibility and broad applicability. In this paper, we propose a novel MAPF algorithm to address the TC-MAPF problem, formulating it as a constrained Markov game. In this formulation, timeliness constraints are integrated into the reward function, ensuring that the agents prioritize meeting these critical deadlines during their operations. Additionally, we introduce a cooperative multi-agent intrinsic reward (CMAIR) framework to enhance the adaptability and generalization of the reward mechanisms across various dynamic environments. The CMAIR framework comprehensively considers the positions and actions of all agents, ensuring efficient exploration and policy optimization. We evaluate our method in various simulated warehouse scenarios and demonstrate that it significantly improves throughput, particularly under stringent time constraints compared to several existing MAPF methods.

With the capability of intelligent control and communicating with surrounding vehicles and infrastructures, connected and automated vehicles (CAVs) can drive cooperatively and have more positive effects on traffic efficiency. Cooperative and real-time path planning for CAVs stands as a pivotal solution to mitigate traffic congestion and augment travel efficiency. However, most of the existing path planning schemes predominantly concentrate on minimizing the travel times of vehicles, sidelining the broader issue of alleviating traffic congestion in urban settings. Therefore, in this paper, we propose a novel collaborative vehicle path planning scheme, leveraging the intelligent control and the communicating ability of CAVs. The primary objective is to reduce traffic congestion within the overall transportation system and improve traffic efficiency. Specifically, we focus on a general urban scenario with various types of vehicles, including CAVs, connected vehicles (CVs), and traditional human-driven vehicles (TVs), To enhance traffic efficiency in such a scenario, we design a collaborative path planning scheme to discover the efficient paths for both CAVs as well as CVs. In this scheme, we treat each CAV as an agent and formulate the multiple CAVs' path-planning problem as a Markov game. To solve the above Markov game, we design a multi-agent convolutional attention reinforcement learning (MACA) framework to generate paths with minimal travel time for CAVs. More concretely, the proposed MACA framework incorporates a convolutional neural network (CNN) layer to capture spatial correlation behind traffic conditions. Additionally, a graph attention network (GAT) layer is employed to integrate the influence of neighboring agents during the path-planning process. To further reduce traffic congestion, we extend the MACA framework into a collaborative MACA (C-MACA) scheme in vehicular networks, where CAVs are empowered to periodically broadcast their path information to surrounding CVs, providing valuable insights for their path planning. Subsequently, to prevent new congestion caused by the aggregation of CVs, we design a heuristic algorithm for CVs to make informed path decisions. We build up a simulator based on a real-world city road map and conduct extensive experiments. The experimental results demonstrate that the proposed scheme can decrease CVs' travel time by up to 10.9 % and reduce the average queue length around junctions by up to 6.5 % over several state-of-the-art approaches, without sacrificing the travel efficiency of CAVs.

In order to support diverse applications in intelligent transportation, intelligent connected vehicles (ICVs) need to send multiple types of messages, such as periodic messages and event-driven messages with different frame specifications. However, existing researches often concentrate on the transmission of single-message types, overlooking hybrid communication scenarios where multiple types of messages coexist, posing challenges in meeting the diverse transmission needs of different message types. To optimize the Quality of Service (QoS) in such scenarios, we take the perspective of ICVs and formulate their decision making as a multi-agent reinforcement learning problem. More specifically, we propose a cooperative individual rewards assisted multi-agent reinforcement learning (CIRA) framework. The transformer structure in CIRA is used to avoid mutual interference during the transmission of different vehicles. Besides, the introduction of individual rewards and the dual-layer architecture of CIRA contribute to providing ICVs with more forward-looking message dissemination scheme. Finally, we set up a simulator to create dynamic traffic scenarios reflecting different real-world conditions. We conduct extensive experiments to evaluate the proposed CIRA framework’s performance. The results show that CIRA can significantly improve the packet reception rates and ensure low communication delays in various scenarios.

Quantum Data Networks (QDNs) are vital to building Distributed Quantum Computing (DQC) systems. Though several communication protocols have been proposed for QDNs, most of them are at the network layer or below. The only transport layer protocol [1] used batch processing of requests for End-to-End (E2E) quantum data transmission. It not only limits the quantum resource utilization, more importantly, it cannot guarantee reliable E2E quantum data transmission. In this paper, we propose the first asynchronous transportation layer protocol, called AQTP, for QDNs to achieve high-speed and reliable E2E quantum data transmission. AQTP has several distinct features: (i) each quantum node locally allocates quantum resources in order to improve scalability; (ii) requests are processed in an asynchronous manner, which results in a higher quantum resource utilization; and (iii) it ensures reliable data transmission even if the teleportation operations fail. Extensive simulations show that compared with a batch processed transport layer protocol, AQTP can increase the network throughput by up to 82.97%, and reduce the Average Task Completion Time (ATCT) of DQC tasks by up to 94.69%.

A group ofconnected and autonomous vehicleswith common interests can drive in a cooperative manner, namely cooperative driving. In such a networked control system, an efficient message dissemination scheme is critical for cooperative drivings to periodically broadcast their kinetic status, i.e.,beacon. However, most existing researches are designed for a simple or specific scenario, e.g., ignoring the impacts of the complex communication environment and emerging hybrid traffic scenarios. Worse still, the inevitable message transmission interference and the limited interaction among vehicles in harsh communication environments seriously hinder cooperation among cooperative drivings and deteriorate the beaconing performance. In this paper, we formulate the decision-making process of cooperative drivings as a Markov game. Furthermore, we propose acooperative hierarchical attention reinforcement learning (CHA)framework to solve this Markov game. Specifically, the hierarchical structure of CHA leads cooperative drivings to be foresighted. Besides, we integrate each hierarchical level of CHA separately with graph attention networks to incorporate agents' mutual influences in the decision-making process. Moreover, each hierarchical level learns a cooperative reward function to motivate each agent to cooperate with others under harsh communication conditions. Finally, we set up a simulator and conduct extensive experiments to validate the effectiveness of CHA.

In real-world urban environments, hybrid and disorder traffic brings new challenges for the intelligent traffic light control system (ITLCS). Apart from coordinating traffic flows around intersections, the ITLCS is responsive to ensuring high priority vehicles pass through intersections quickly. To this end, we formulate the multiple intersections’ decision-making problem as a Semi-Markov game and propose amulti-agent attention double actor-critic (MAADAC)framework to solve this game, integrating theoptions frameworkwithgraph attention networks (GATs). Specifically, the options framework empowers agents to learn to make a long sequence of satisfactory decisions, such as keeping a reasonable phase for a short period to ensure high priority vehicles pass through intersections quickly. Besides, we adopt GATs to capture graph-structure mutual influences among agents. We set up a simulator based on real-world city road networks and conduct extensive experiments to evaluate the performance of MAADAC. The experimental results show that MAADAC can reduce high priority vehicles’ waiting time in the interval of 18.16%-38.14% versus the density of vehicles in real-world urban scenarios over several state-of-the-art approaches. Also, our framework can guarantee the passing efficiency of high priority vehicles under various traffic conditions with the change in the proportion of high priority vehicles.

Since emergency vehicles (EVs) are essential for urban emergency response, it is essential to help EVs arrive faster. Existing work has investigated route optimization or traffic signal preemption, but they are insufficient because most studies consider the two areas separately and lack a deep understanding of their relationship. For instance, traffic signal preemption can cause changes in traffic flow, affecting the optimal route for EVs. Moreover, previous work does not distinguish between EV types or consider the negative impact on ordinary vehicles (OVs). To address these issues, we propose a framework that jointly considers priority allocation, routing optimization, and traffic signal preemption (EVPRT) in conjunction with the Vehicle to Everything (V2X) environment. To this end, we design an emergency vehicle priority system (EVPS) to assign priorities to different EV types. Then, we design a dynamic route optimization method to update the optimal routes for EVs. Finally, we design a multi-agent reinforcement learning (MARL) based traffic signal preemption algorithm and use a Graph Attention Network (GAT) to extract potential features of different intersections. Furthermore, to provide communication conditions for EVPRT, we adopt a V2X-based communication technology for information interaction. The simulation findings show that our proposed method significantly decreases EV travel time and enhances the capacity of urban emergency service management.

To unleash the power of buses, we propose a bus-pooling system that keeps the notion of bus stops and terminals but discards the concept of fixed bus lines by enabling buses to choose the next stop or terminal based on orders submitted by passengers. Each bus, unlike a taxi, must consider the additional delays experienced by the passengers already on board when deciding how to adapt its route to serve new orders. This paper treats each bus as an agent and formulates the buses’ re-routing decision-making process as a Semi-Markov game. Then, we propose a novel double graph attention actor-critic (DGAAC) framework by integrating high-level and low-level actor-critics separately with graph attention networks (GATs) to solve the game. Specifically, GATs embedded in high-level and low-level critics take a large-scale graph covering a city-scale area as input and capture graph-structured mutual influences among buses. In contrast, the high-level and low-level actors equipped with GATs only take the n-hop sub-graph with local information as the input and are employed as the distributed decision module of each bus. We conduct extensive experiments on one of the largest real-world datasets in Shenzhen, China, and validate that the proposed DGAAC framework greatly outperforms all baselines.

Vehicular communication networks (VCNs) have been widely recognized as promising solutions to support safety-related applications in urban transportation systems. However, constructing and maintaining such networks is quite challenging due to the complex traffic and communication environment. Substantial studies have focused on the design of the networking schemes and message dissemination protocols. Nonetheless, most existing designs only consider the connectivity and rapid end-to-end transmission, regardless of the network coverage and duration. In this article, we propose a novel temporary warning network (TWN) for safety message dissemination in the urban traffic environment, in which both the spatial distribution and temporal duration of the networking scheme are taken into account. Specifically, TWN is constructed by the selection of relay vehicles based on the spatiotemporal correlation of vehicle trajectory so that the safety message can be quickly disseminated within the Regions of Interest (RoIs). To maintain TWN during an accident, a reselection mechanism is also proposed, which enables newly come vehicles in the RoI to receive the messages in time. Finally, we conduct extensive numerical experiments to validate the effectiveness of our method in various traffic scenarios.

Nowadays, most of the taxi drivers have become users of the relocation recommendation service offered by online ride-hailing platforms (e.g., Uber and Didi Chuxing), which could oftentimes lead drivers to places with profitable orders. At the same time, electric taxis (e-taxis) are increasingly adopted and gradually replacing gasoline taxis in today’s public transportation systems due to their environmental-friendly nature. Though effective for traditional gasoline taxis, existing relocation recommendation schemes are rather suboptimal for e-taxi drivers’ user experience. On one hand, the existing schemes take no account of taxis’ refueling decisions, as the refueling durations of gasoline taxis are usually short enough to be ignored. However, the charging duration of the e-taxis spent at charging stations can be as long as hours. Obviously, an e-taxi’s battery could be easily depleted by the continuous relocations suggested by existing schemes, and thus will have to be charged for a long time afterwards, making the e-taxi driver miss numerous order-serving opportunities. On the other hand, charging posts are typically sparsely and unevenly distributed across a city. With no consideration of charging opportunities, existing schemes could probably send an e-taxi to an area with no charging post around, even though its battery is running low. To optimize e-taxi drivers’ user experience, in this paper, we design a jointcharging and relocation recommendation system for e-taxi drivers (CARE). We take the perspective of e-taxi drivers and formulate their decision making as a multi-agent reinforcement learning problem where each e-taxi driver aims to maximize his own cumulative rewards. More specifically, we propose a novelmulti-agent mean field hierarchical reinforcement learning (MFHRL)framework. The hierarchical architecture of MFHRL helps the proposed CARE provide far-sighted charging and relocation recommendations for e-taxi drivers. Besides, we integrate each hierarchical level of MFHRL separately with the mean field approximation to incorporate e-taxis’ mutual influences in decision making. We set up a simulator with one of the largest real-world e-taxi datasets in Shenzhen, China, which contains the GPS trajectory data and transaction data of 3848 e-taxis from June 1st to June 30th, 2017, coupled with 165 charging stations including 317 fast charging posts and 1421 slow charging posts. We adopt this simulator to generate 6 dynamic urban environments, which reflect the different real-world scenarios faced by e-taxi drivers. In all of these environments, we conduct extensive experiments to validate that the proposed MFHRL framework greatly outperforms all baselines by significantly increasing the rewards obtained by e-taxi drivers. Besides, we also show that the charging policy learned by MFHRL can effectively reduce the range anxiety of e-taxi drivers, which significantly boosts e-taxi drivers’ quality of experience.

A group of connected and autonomous vehicles (CAVs) with common interests can drive in a cooperative manner, namely cooperative driving, which has been verified to significantly improve road safety, traffic efficiency, and environmental sustainability. A more general scenario with various types of cooperative driving applications such as truck platooning and vehicle clustering will coexist on roads in the foreseeable future. To support such multiple cooperative drivings, it is critical to design an efficient message dissemination scheduling for vehicles to broadcast their kinetic status, i.e., beacon periodically. Most ongoing researches suggest designing the communication protocols via traffic and communication modeling on top of dedicated short range communications (DSRC) or cellular-based vehicle-to-vehicle (C-V2V) communications as a potential remedy. However, most of the existing researches are designed for a simple or specific traffic scenario, e.g., ignoring the impacts of the complex communication environment and emerging hybrid traffic scenarios. Moreover, some studies design beaconing strategies based on the implication of channel and traffic conditions in the beacons of other vehicles. However, the delayed perception of these information may seriously deteriorate the beaconing performance. In this paper, we take the perspective of cooperative drivings and formulate their decision-making process as a Markov game. Furthermore, we propose a multi-agent hierarchical attention reinforcement learning (MAHA) framework to solve the Markov game. More concretely, the hierarchical structure of the proposed MAHA can lead cooperative drivings to be foresightful. Hence, even without immediate incentives, the well-trained agents can still take favorable actions that benefit their long-term rewards. Besides, we integrate each hierarchical level of MAHA separately with the graph attention network (GAT) to incorporate agents' mutual influences in the decision-making process. Besides, we set up a simulator and adopt this simulator to generate dynamic traffic scenarios, which reflect the different real-world scenarios faced by cooperative drivings. We conduct extensive experiments to evaluate the proposed MAHA framework's performance. The results show that MAHA can significantly improve the beacon reception rate and guarantee low communication delay in all of these scenarios.

A group of connected and autonomous vehicles (CAVs) with common interests can drive in a cooperative manner, namely cooperative driving, which has been verified to significantly improve road safety, traffic efficiency and environmental sustainability. A more general scenario that various types of cooperative driving applications such as truck platooning and vehicle clustering, will coexist on roads in the foreseeable future. To support such multiple cooperative drivings, it is critical to design an efficient message dissemination scheduling in a shared communication channel. Most ongoing research suggests using the time-division multiple access (TDMA) method on top of IEEE 802.11p as a potential remedy. However, TDMA requires time synchronization and is not flexible, especially in the multiple cooperative drivings scenario where the beacon frequency needs to be updated and the number of cooperative drivings changes to meet the time-varying traffic conditions. In this paper, we focus on the study of the message dissemination protocol for platooning, a typical and well-known cooperative driving pattern. Specifically, we proposed a hybrid message dissemination protocol which aims at guaranteeing the reliable delivery of beacon messages for a multi-platooning system. We first adopt a TDMA-based medium access method for intra-platoon communication to improve the reliability and efficiency of beacon dissemination. We then present a token-passing medium access method for inter-platoon communication, which maps platoons into a token ring to schedule their beacon transmission time. We conduct extensive numerical experiments to validate the effectiveness of our protocol.

Vehicular communication networking (VCN) has been widely recognized as a promising solution to support safety-related applications in urban transportation systems. In VCN, efficient message dissemination can let vehicles be better aware of the potential risks and traffic anomalies, which is critical to road safety and traffic efficiency. Substantial studies have focused on the design of inter-vehicle message dissemination protocols. Nonetheless, most existing designs only consider the rapid end-to-end transmission, few of which take into account the broadcast coverage. In this paper, we propose a new message dissemination scheme in the urban traffic scenario by considering both the time constraint and the spatial distribution of data dissemination. Specifically, based on the temporal and spatial correlation of vehicle trajectory, relay vehicles are selected to construct a temporary warning network (TWN) for a rapid safety message dissemination in the regions of interest (ROI). Finally, we conduct extensive numerical experiments to validate the effectiveness of our method in various traffic scenarios.

Recent advances in connected vehicle technologies enable vehicles and signal controllers to cooperate and improve the traffic management at intersections. This paper explores the opportunity for cooperative and integrated vehicle and intersection control for energy efficiency (CIVIC-E2) to contribute to a more sustainable transportation system. We propose a two-level approach that jointly optimizes the traffic signal timing and vehicles' approach speed, with the objective being to minimize total energy consumption for all vehicles passing through an isolated intersection. More specifically, at the intersection level, a dynamic programming algorithm is designed to find the optimal signal timing by explicitly considering the arrival time and energy profile of each vehicle. At the vehicle level, a model predictive control strategy is adopted to ensure that vehicles pass through the intersection in a timely fashion. Our simulation study has shown that the proposed CIVIC-E2system can significantly improve intersection performance under various traffic conditions. Compared with conventional fixed-time and actuated signal control strategies, the proposed algorithm can reduce energy consumption and queue length by up to 31% and 95%, respectively.