Collaborative perception leveraging intermediate feature fusion has emerged as a leading paradigm to significantly enhance the environmental perception capabilities of autonomous driving systems. However, existing methods typically rely on discriminative supervision guided by downstream tasks. This paradigm compels models to learn minimal, task-specific representations, which conflicts with the goal of cooperative perception to capture comprehensive information, thereby limiting generalization. To address this issue, we propose DiGS-CP, a novel two-stage generative supervised collaborative perception framework. Specifically, we introduce a diffusion-based generative task that conditions on fused object-level features to generate representations of object-level point clouds. The proposed generative supervision provides fine-grained, task-agnostic signals that encourages the fusion module to learn comprehensive representations beyond task-specific requirements. By preserving and integrating complementary information from collaborative agents, our approach overcomes the limitations of task-specific learning and enhances the generalizability of the learned features. Furthermore, our two-stage architecture requires agents to transmit only object-level features, significantly reducing communication overhead. Extensive experiments on three benchmark datasets demonstrate that DiGS-CP achieves state-of-the-art performance in 3D object detection, while maintaining low bandwidth requirements and exhibiting excellent generalization ability.

Collaborative perception can significantly enhance the perceptual capabilities of autonomous vehicles by sharing sensing information through vehicular communications. However, large-scale sharing of sensing information often results in unsustainable network loads, making it challenging to maximize perception performance with limited communication resources in complex environments. To address this challenge, we propose FullPerception, an innovative cooperative perception framework that jointly orchestrates sensing information sharing and communication resource allocation at the network level. FullPerception advocates for the sharing of semantic information (neural network features) within critical areas, i.e., blind spots. With limited communication resources, FullPerception strategically eliminates these blind spots to maximize the accumulated perception performance. We formulate this strategy as a weighted optimization problem and prove its NP-hardness. We propose a simple yet effective algorithm, Proactive Conflict-free Scheduling (PCS), which guarantees a good performance ratio by considering broader contexts. PCS is meticulously combined with recursive structure, accounting for both the overall and future contexts to determine link scheduling and resource allocation. We demonstrate that FullPerception improves perception accuracy by 20% relative to single-vehicle systems and by 10% compared to existing scheduling methods through large-scale comprehensive joint simulation experiments.

Collaborative perception in autonomous driving significantly enhances the perception capabilities of individual agents. Immutable heterogeneity, where agents have different and fixed perception networks, presents a major challenge due to the semantic gap in exchanged intermediate features without modifying the perception networks. Most existing methods bridge the semantic gap through interpreters. However, they either require training a new interpreter for each new agent type, limiting extensibility, or rely on a two-stage interpretation via an intermediate standardized semantic space, causing cumulative semantic loss. To achieve both extensibility in immutable heterogeneous scenarios and low-loss feature interpretation, we propose PolyInter, a polymorphic feature interpreter. It provides an extension point where new agents integrate by overriding only their specific prompts, which are learnable parameters that guide interpretation, while reusing PolyInter’s remaining parameters. By leveraging polymorphism, our design enables a single interpreter to accommodate diverse agents and interpret their features into the ego agent’s semantic space. Experiments on the OPV2V dataset demonstrate that PolyInter improves collaborative perception precision by up to 11.1% compared to SOTA interpreters, while comparable results can be achieved by training only 1.4% of PolyInter’s parameters when adapting to new agents. Code is available at https://github.com/yuchen-xia/PolyInter.

Heterogeneous collaborative perception enables interconnected agents to share and fuse intermediate features extracted from multimodal sensor data, thereby enhancing robustness and reducing perception blind spots. However, shared high-dimensional features often contain redundant and modality-specific information, leading to semantic inconsistencies and excessive communication overhead. To address these challenges, we propose an efficient communication framework for heterogeneous collaborative perception. This framework integrates a shared-private feature decoupling module that disentangles cross-modal features into shared and private components, sharing only shared feature to improve communication efficiency. Additionally, we design a spatial redundancy elimination module to remove background information from transmitted features, further reducing bandwidth consumption. Collectively, these modules enable compact and communication-efficient perception capabilities across heterogeneous modalities, models, and tasks. Experimental results demonstrate that the proposed framework significantly reduces communication overhead in complex heterogeneous scenarios while maintaining perception accuracy.

Collaborative perception improves task performance by expanding the perception range through information sharing among agents. Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender. The experimental results demonstrate that NegoCollab significantly outperforms existing methods in common representation-based collaboration approaches. The mechanism of obtaining common representations through negotiation provides a more reliable and flexible option for common representations in heterogeneous collaborative perception.

Multi-agent collaborative perception enhances perception performance by enabling information sharing and complementary data fusion among agents. However, this process inevitably requires a trade-off between perception performance, computing resources, and communication bandwidth to ensure overall collaboration efficiency. To address these challenges, we propose a joint resource and intelligence optimization method for multi-vehicle collaborative perception, named C3I-JO. Based on slimmable network and accuracy-awareness, the method performs joint optimization over collaborative mode, resource allocation, and intelligent elasticity. C3I-JO minimizes overall resource consumption while satisfying both perception accuracy and delay constraints, thereby improving the overall system efficiency. Simulation results demonstrate that, compared with baseline methods, the proposed method achieves superior performance in terms of both resource consumption and perception quality.

Collaborative perception enables connected and autonomous vehicles (CAVs) to overcome individual viewpoint limitations by exchanging perception data, making effective resource allocation crucial for timely transmission. However, existing studies focus on resource allocation within a single collaborative perception group (CPG), limiting their effectiveness in multi-group collaborative perception systems. In such a system, each CPG contributes differently to the overall collaborative perception performance, and it is challenging to evaluate and represent CPG system-level utilities. Meanwhile, competition for shared spectrum resources leads to interference and complicates the joint optimization of collaboration mechanisms and spectrum allocation, which is intensified by their temporal scale misalignment. To address these challenges, we propose a Utility-Aware Hierarchical Reinforcement Learning method (UAHRL) to jointly optimize collaboration mechanisms and spectrum allocation. Specifically, we introduce a hierarchical framework to handle temporally misaligned decisions through joint training. The upper layer optimizes the collaborative relationship and granularity over a longer time scale to enhance system-level collaborative performance, while the lower layer allocates spectrum resources over a shorter time interval to fulfill individual CPG transmission demand and enhance system transmission efficiency. To represent and utilize system-level utility, we leverage a feature-based confidence map to assess CAVs’ perception capability and complementarity. A mixing network in the upper layer further decomposes global performance into individual CPG utilities, enabling utility-aware resource allocation. Simulations show that UAHRL outperforms baseline methods in system-level collaborative perception in multi-group systems.

Recent multi-agent deep reinforcement learning (MADRL) approaches have shown notable benefits in multi-intersection traffic signal control (M-TSC). However, spatial-temporal coupling and action heterogeneity are often overlooked. Interactions among traffic states and signal actions introduce complex coupling and hysteresis across space and time, while local actions vary due to fluctuating flows. We proposeAsyn-Light, a MADRL-based M-TSC model that captures spatial-temporal traffic features and leverages action repetition for asynchronous cooperative control. To address spatial-temporal coupling,Asyn-Lightemploys a feature extraction framework based on a spatial-temporal Transformer with a stacked spatial graph module. To handle action heterogeneity, it uses a repetition-enabled MADRL to generate asynchronous multi-step policies for each intersection. We evaluateAsyn-Lighton both synthetic and real-world datasets, analyzing each component’s contribution. Results show thatAsyn-Lightconsistently outperforms baselines in diverse settings, effectively balancing action smoothness with policy adaptability.

Efficient wireless resource allocation is essential for supporting multi-vehicle cooperation. The service data exchanged among intelligent vehicles is typically diverse, with varying transmission requirements that shift according to applications and traffic conditions, leading to major fluctuation in communication situations. Existing multi-agent reinforcement learning based resource allocation methods are often inefficient in handling such nonstationary communication situations due to their rigid cooperation patterns. To this end, we propose a ROle-TRansformable multi-agent resource allocation method, named ROTR. This method adopts a hierarchical decision-making process, where a high-level agent at a base station (BS) dynamically plans and distributes cooperation roles (CRs) and cooperation behaviors (CBs) in response to fluctuating communication situations. The Low-level agents within the transmitting vehicles (TVs) perform role transformations based on the assigned CRs and subsequently receive behavioral guidance according to CBs, enabling dynamic adjustments in cooperation patterns to adapt to variable communication situations and make resource allocation decisions. Additionally, we introduce a non-BS-assisted mode based on policy distillation, which enables a seamless transition to independent operation without the BS, relying solely on local states to generate CRs and CBs, thereby facilitating global resource cooperation. Extensive simulation experiments demonstrate that the proposed framework optimizes resource efficiency in nonstationary vehicular communications.

Point-voxel 3D object detectors have achieved impressive performance in complex traffic scenes. However, they utilize the 3D sparse convolution (spconv) layers with fixed receptive fields, such as voxel-based detectors, and inherit the fixed sphere radius from point-based methods for generating the features of keypoints, which make them weak in adaptively modeling various geometrical deformations and sizes of real objects. To tackle this issue, we propose a shape-adaptive set abstraction network (SASAN) for point-voxel 3D object detection. First, the proposal and offset generation module is adopted to learn the coordinates and confidences of 3D proposals and shape-adaptive offsets of the certain number of offset points for each voxel. Meanwhile, an extra offset supervision task is employed to guide the learning of shifting values of offset points, aiming at motivating the predicted offsets to preferably adapt to the various shapes of objects. Then, the shape-adaptive set abstraction module is proposed to extract multiscale keypoints features by grouping the neighboring offset points' features, as well as features learned from adjacent raw points and the 2-D bird-view map. Finally, the region of interest (RoI)-grid proposal refinement module is used to aggregate the keypoints features for further proposal refinement and confidence prediction. Extensive experiments on the competitive KITTI 3D detection benchmark demonstrate that the proposed SASAN gains superior performance as compared with state-of-the-art methods.

Vehicle platooning is a promising traffic model in intelligent transportation systems (ITSs), which can effectively improve resource utilization and reduce traffic congestion. The resource allocation for vehicle-to-everything (V2X) communications that consist of intraplatoon communications and interplatoon communications is crucial for safe operation of multiple vehicular platoons. Considering dynamic coordination pattern of vehicular platoons and layered architecture of vehicle platooning networks, a hierarchical resource decision-making framework is proposed in this article. In the proposed framework, the resource decision-making process is divided into two levels. The high level that generates and distributes coordination meta policy is deployed on base station (BS), and the low level that generates ego resource decisions is deployed in each platoon. To deal with optimization of resource allocation for multiplatoon V2X communications, a hierarchical reinforcement learning method (HierNet) is designed based on the proposed hierarchical decision-making framework. In HierNet, meta policy of the high level can be preserved and needs to be updated only when cooperative conditions of multiple platoons undergo distinct changes. Simulation experiments have demonstrated that our proposed method not only optimizes resource efficiency but also reduces the communication costs for resource decision making of vehicle platooning networks.

Autonomous driving vehicle (ADV) that is ready to transform our society and economy, is in desperate need of precise positioning over itself as well as surrounding environments. However, it is still a challenging issue for ADV to retrieve real-time positioning knowledge over road participants and dynamic surrounding environments, due to unsatisfied perception accuracy caused by sparse observations and limited perception range. Cooperative perception, which advocates cooperatively disseminating perception data among vehicles, has the potential to overcome the above limitations. To this end, this article proposes a novel edge-assisted multi-vehicle perception system to enhance vehicles’ awareness over surrounding environments, which is termed as EdgeCooper. EdgeCooper first schedules vehicles to share complementarity-enhanced and redundancy-minimized raw sensor data with an edge server, using multi-hop cooperative 5G V2X communications. Then, EdgeCooper merges vehicles’ individual views to form a holistic view with a higher resolution, thus enhancing perception robustness and enlarging perception range. We formulate multi-vehicle multi-hop cooperative data sharing as a minimum cost flow problem with conflict, and further prove that there exists no polynomial-time approximation algorithm with a constant performance ratio unless P = NP. Furthermore, a two-dimension graph coloring algorithm with guaranteed performance is proposed to eliminate conflict. We evaluate EdgeCooper by building a comprehensive simulation platform through a joint manipulation of SUMO, CARLA, NS3, and PyTorch. The experiment results show that, compared to a single vehicle’s perception, EdgeCooper performs effective and efficient in enhancing vehicular awareness, e.g., extending up to 3.6 times detection range and improving perception accuracy by 20%.

Unsupervised domain adaptive object detection methods aim to transfer knowledge from the label-sufficient domain to the unlabeled domain. Most existing works minimize domain disparity by concentrating on different levels through adversarial learning. However, adversarial learning do not consider the different influences on under-aligned and well-aligned samples as they merely match distinct distributions with consistent weight. To address this issue, we design a novel scale-disentangled and uncertainty-guided alignment for domain-adaptive object detection (SDUGA), consisting of three main components: (1) Disentangled scale coarse module, which decouples scale information from global image features and performs individual alignment across domains for the corresponding scale by training domain classifiers in an adversarial learning manner; (2) Disentangled scale fine module, which generalizes the disentangled scale alignment to instance-level adaptation, reinforcing the distribution alignment across domains from multi-scale local instance level; (3) Uncertainty-guided coarse-to-fine attention alignment, which adjusts weights for various samples adaptively by generating the uncertainty-guided attention map, thus enforcing the detector to converge more on alignment for under-aligned samples and avoid misaligning well-aligned ones. Extensive experiments over three challenging domain-shift object detection scenarios demonstrate that SDUGA gains superior performance compared to state-of-the-art methods.

Traffic prediction is a keystone for building smart cities in the new era and has found wide applications in traffic scheduling and management, environment policy making, public safety, and so on. Instead of creating a traffic predictor for each city, this article focuses on designing a unified network model that could be directly applied for traffic prediction in any city, by learning the essential spatial-temporal dependencies, i.e., the mutual relationship between traffic and the corresponding fine-grained road network. To achieve this goal, this article proposes a joint knowledge- and data-driven mechanism that novelly divides dependencies into three kinds of correlations, i.e., road segment, intra-intersection, and inter-intersection correlation, which capture the microcosmic, middle, and macroscopic dependencies between traffic and the road network, respectively. Specifically, we first construct traffic datasets that could cover all road segments from real-world trajectory datasets, which makes it possible to model the whole road network as a graph, with the help of fine-grained road topology. Then, we propose meta road segment learner, connection-aware spatial-temporal graph convolutional network (GCN), and multiscale residual networks for capturing the microcosmic, middle, and macroscopic dependencies, respectively. Our experiments on three real-world datasets demonstrate that our proposed method could: 1) achieve better prediction accuracy compared with several approaches and 2) capture the mutual relationship between traffic and the fine-grained road network since our model trained only using data from the source city achieves good performance when it is directly applied for traffic prediction in the target city, without any fine-tuning. The codes will be made publicly available.

Recent years have witnessed the growing popularity of connectionist temporal classification (CTC) and attention mechanism in scene text recognition (STR). CTC-based methods consume less time with few computational burdens, while they are not as effective as attention-based methods. To retain computational efficiency and effectiveness, we propose the global-local attention-augmented light Transformer (GLaLT), which adopts a Transformer-based encoder-decoder structure to orchestrate CTC and attention mechanism. The encoder integrates the self-attention module with the convolution module to augment the attention, where the self-attention module pays more attention to capturing long-term global dependencies and the convolution module focuses on local context modeling. The decoder consists of two parallel modules: one is the Transformer-decoder-based attention module and the other is the CTC module. The first one is removed in the testing phase and can guide the second one to extract robust features in the training phase. Extensive experiments on standard benchmarks demonstrate that GLaLT achieves state-of-the-art performance for both regular and irregular STR. In terms of tradeoffs, the proposed GLaLT is at or near the frontiers for maximizing speed, accuracy, and computational efficiency at the same time.

This paper proposes AlphaRoute, an AlphaGo inspired algorithm for coordinating large-scale routes, built upon graph attention reinforcement learning and Monte Carlo Tree Search (MCTS). We first partition the road network into regions and model large-scale coordinated route planning as a Markov game, where each partitioned region is treated as a player instead of each driver. Then, AlphaRoute applies a bilevel optimization framework, consisting of several region planners and a global planner, where the region planner coordinates the route choices for vehicles located in the region and generates several strategies, and the global planner evaluates the combination of strategies. AlphaRoute is built on graph attention network for evaluating each state and MCTS algorithm for dynamically visiting and simulating the future state for narrowing down the search space. AlphaRoute is capable of 1) bridging user fairness and system efficiency, 2) achieving higher search efficiency by alleviating the curse of dimensionality problems, and 3) making an effective and informed route planning by simulating over the future to capture traffic dynamics. Comprehensive experiments are conducted on two real-world road networks as compared with several baselines to evaluate the performance, and results show that AlphaRoute achieves the lowest travel time, and is efficient and effective for coordinating large-scale routes and alleviating the traffic congestion problem. The code will be publicly available.

The use of multi-agent reinforcement learning (MARL) methods in coordinating traffic lights (CTL) has become increasingly popular, treating each intersection as an agent. However, existing MARL approaches either treat each agent absolutely homogeneous, i.e., same network and parameter for each agent, or treat each agent completely heterogeneous, i.e., different networks and parameters for each agent. This creates a difficult balance between accuracy and complexity, especially in large-scale CTL. To address this challenge, we propose a grouped MARL method named GPLight. We first mine the similarity between agent environment considering both real-time traffic flow and static fine-grained road topology. Then we propose two loss functions to maintain a learnable and dynamic clustering, one that uses mutual information estimation for better stability, and the other that maximizes separability between groups. Finally, GPLight enforces the agents in a group to share the same network and parameters. This approach reduces complexity by promoting cooperation within the same group of agents while reflecting differences between groups to ensure accuracy. To verify the effectiveness of our method, we conduct experiments on both synthetic and real-world datasets, with up to 1,089 intersections. Compared with state-of-the-art methods, experiment results demonstrate the superiority of our proposed method, especially in large-scale CTL.

Multi-agent collaborative perception depends on sharing sensory information to improve perception accuracy and robustness, as well as to extend coverage. However, most collaborative perception methods ignore the limitations of communication networks, such as limited bandwidth and the possibility of wireless conflicts. To fill this gap, this paper proposes BlindSpotEliminator, a conflict-free scheduler over the cellular-V2X networks for supporting practical collaborative point cloud perception to eliminate blind spots. BlindSpotEliminator first identifies the blind spots for each vehicle, then lists the corresponding conflict relationships based on the distribution of the blind spots and communication conflicts, and finally designs an optimized point cloud data transmission strategy to eliminate the blind spots of each vehicle. Extensive experiments show that compared with greedy algorithm and random methods, BlindSpotEliminator achieves better efficiency, i.e., transmitting 20% more point cloud data.

In this paper, we propose a masked and sparse transformer to address the problem of point cloud registration with low overlap. The mask mechanism reduces the overall data, increasing the corresponding point ratio in the overlap region, while also reducing the computational cost to accelerate the algorithm's execution speed. Moreover, we combine spatial position encoding and sparse self-attention to establish relationships within the source point cloud, as well as the relationships and attention scores between the source and target point clouds. This approach is specifically designed for the task of point cloud registration. Finally, we search for the maximum overlap area by matching the spatial consistency between points and calculate the 3D transformation matrix to complete the registration process. Our method achieves an improvement in the inlier ratio and performs well on the 3DMatch and 3DLoMatch datasets, demonstrating high registration efficiency.

Traffic forecasting aims to capture complex spatial-temporal dependencies and non-linear dynamics, which plays an indispensable role in intelligent transportation systems and other domains like neuroscience, climate, etc. Most recent works rely on graph convolutional networks (GCN) to model the dependencies and the dynamics. However, the over-smoothing issue of GCN would produce indistinguishable features among nodes, leading to poor expressivity and weak capability of modeling complex dependencies and dynamics. To address this issue, we present a novel clustering spatial-temporal (ClusterST) unit, which incorporates unsupervised learning into GCN for extracting discriminative features. Specifically, we first exploit a neural network to learn a dynamic clustering, i.e., learning to partition the neighbors of each node into clusters at each time step. Two probabilistic losses are proposed to improve the separability of clusters. Then, the extracted features of different clusters can be distinguished. Based on the dynamically formed clusters, a vanilla GCN is applied to aggregate features within each cluster. By purely exploiting such a ClusterST unit, large improvements over the state-of-the-art are achieved. Furthermore, ClusterST units with a different number of clusters can be regarded as basic components to construct an inception-like ClusterST network for going deeper. We evaluate the framework on two real-world large-scale traffic datasets and observe an average improvement of 18.19% and 7.62% over state-of-the-art baselines, respectively. The code and models will be publicly available.

Metaverse and intelligent transportation system (ITS) are disruptive technologies that have the potential to transform the current transportation system by decreasing traffic accidents and improving driving safety. The integration of Metaverse and transportation technology, called metaverse transportation system (MTS), can greatly improve the intelligence of real transportation system. The digital models built in MTS help to simulate the full life cycle of physical entities, which equip the virtual space with controllability and flexibility. In this article, we concentrate on the field of environment perception, which is the basic function of intelligent vehicles in MTS. To overcome the poor scalability of traditional environment perception methods, we develop the framework of parallel vision for ITS in metaverse (PVITS), consisting of construction of virtual transportation space, model learning based on computational experiments, and feedback optimization based on parallel execution. This article highlights opportunities brought by PVITS in terms of model precision and generalization improvement. Then, the challenges of PVITS are discussed, i.e., distribution difference between virtual and real transportation space, structure design and theoretical interpretation of vision models, and data security and privacy in virtual transportation space. After that, we present several solutions to tackle the application challenges and fully exploit the superior characteristics of PVITS while attenuating their negative side effects. Some potential applications are also given to represent the effectiveness and reliability of PVITS.

Multi-agent collaborative perception depends on sharing sensory information to improve perception accuracy and robustness, as well as to extend coverage. The cooperative shared information between agents should achieve an equilibrium between redundancy and complementarity, thus creating a concise and composite representation. To this end, this paper presents a complementarity-enhanced and redundancy-minimized collaboration network (CRCNet), for efficiently guiding and supervising the fusion among shared features. Our key novelties lie in two aspects. First, each fused feature is forced to bring about a marginal gain by exploiting a contrastive loss, which can supervise our model to select complementary features. Second, mutual information is applied to measure the dependence between fused feature pairs and the upper bound of mutual information is minimized to encourage independence, thus guiding our model to select irredundant features. Furthermore, the above modules are incorporated into a feature fusion network CRCNet. Our quantitative and qualitative experiments in collaborative object detection show that CRCNet performs better than the state-of-the-art methods.

Accurate spatial-temporal prediction is a fundamental building block of many real-world applications such as traffic scheduling and management, environment policy making, and public safety. This problem is still challenging due to nonlinear, complicated, and dynamic spatial-temporal dependencies. To address these challenges, we propose a novel embedded spatial-temporal network (ESTNet), which extracts efficient features to model the dynamic correlations and then exploits three-dimension convolution to synchronously model the spatial-temporal dependencies. Specifically, we propose multi-range graph convolution networks for extracting multi-scale static features from the fine-grained road network. Meanwhile, dynamic features are extracted from real-time traffic using a gated recurrent unit network. These features can be applied to identify the dynamic and flexible correlations among sensors and make it possible to exploit a three-dimension convolution unit (3DCon) to simultaneously model the spatial-temporal dependencies. Furthermore, we propose a residual network by stacking multiple 3DCon to capture the nonlinear and complicated dependencies. The effectiveness and superiority of ESTNet are verified on two real-world datasets, and experiments show ESTNet outperforms the state-of-the-art with a significant margin. The code and models will be publicly available.

Object detection in traffic scenes has attracted considerable attention from both academia and industry recently. Modern detectors achieve excellent performance under a simple constrained environment while performing poorly under the actual complex and open traffic environment. Therefore, the capability of adapting to new and unseen domains is a key factor for the large-scale application and proliferation of detectors in autonomous driving. To this end, this paper proposes a novel category-induced coarse-to-fine domain adaptation approach (C2FDA) for cross-domain object detection, which consists of three pivotal components: (1) Attention-induced coarse-grained alignment module (ACGA), which strengthens the distribution alignment across disparate domains within the foreground features in category-agnostic way by the minimax optimization between the domain classifier and the backbone feature extractor; (2) Attention-induced feature selection module, which assists the model to emphasize the crucial foreground features and enables the ACGA to focus on the relevant and discriminative foreground features, without being affected by the distribution of inconsequential background features; (3) Category-induced fine-grained alignment module (CFGA), which reduces the domain shift in category-aware way by minimizing the distance of centroids with the same category from different domains and maximizing that of centroids with disparate categories. We evaluate the performance of our approach in various source/target domain pairs and comprehensive results demonstrate that C2FDA significantly outperforms the state-of-the-art on multiple domain adaptation scenarios, i.e., the synthetic-to-real adaptation, the weather adaptation, and the cross camera adaptation.

3D object detection has attracted a significant amount of attention and interest from both academia and industry due to its indispensable role in understanding 3D environments. By fusing the camera and LiDAR sensors, it is expected to improve both the accuracy and robustness of 3D object detection. However, existing fusion approaches are either limited by cascading processing, or easy to be influenced by the interference information in multi-sensors. To this end, this paper incorporates meta learning to fuse the camera and LiDAR data. Specifically, we first extract meta knowledge from images, and then apply it to generate the parameter weights of a set of convolution kernels, which are further exploited for feature extraction on LiDAR point clouds. Furthermore, we propose a meta fusion network (MF -Net), enabling accurate and robust 3D object detection. The superiority and effectiveness of MF -Net have been demonstrated by extensive experiments on KITTI 3D object detection dataset.

As artificial intelligence and the internet of vehicles are becoming mature, multi-agent reinforcement learning is utilized as an efficient way to coordinate vehicles to achieve safer and more efficient transportation. Communication between vehicles is essential for multi-vehicle cooperation to enhance the understanding of the environment state and the intentions of other vehicles. However, with limited communication resources, how to compress the message and reduce the number of messages that need to be transmitted while ensuring the coordination performance is an urgent problem to be solved. And evaluating whether the message is useful and identifying the valuable information from the received messages are huge challenges for the vehicle communication system. To this end, we propose an efficient communication method that can guarantee coordination performance with limited communication resources. In particular, the efficient communication method using the algorithm in variational auto-encoder to compress the message while guaranteeing the valuable information of the message is preserved in the compressed message. Additionally, the multi-head attention mechanism is utilized to extract valuable information from the received messages and help the vehicle to make the driving decision. To avoid contention of communication resources, the message will be scored before transmitted to other vehicles. So that communication resources can be reserved for valuable messages. The efficient communication method is evaluated in an unsignalized intersections scenario. Experimental results show that the efficient communication method achieves better performance under the bandwidth-constrained environment than the baselines.

Artificial intelligence-empowered smart things (e.g., robots, autonomous vehicles, and unmanned aerial vehicles) have been transforming the world. The “brains” of smart things can be abstracted as the agents or cybertwins residing on end devices and edge servers. The next-generation communication networks (i.e., 6G) will become the nervous system for these agents and natively support multiagent cooperation. By sharing local observations and intentions via communication channels, the agents could better understand the environments and make right decisions. Due to the limited channel bandwidth, the communication is considered as a bottleneck of multiagent cooperation. In this article, we propose a graph convolutional communication method (GraphComm) for multiagent cooperation to relive the bottleneck. Specifically, a variational information bottleneck is used to encode the observations and intentions compactly. Furthermore, a graph information bottleneck with the attention-based neighbor sampling mechanism is utilized to improve the effectiveness and robustness of the multiround communication process. The experimental results show that GraphComm can improve the effectiveness, robustness, and efficiency of communication in multiagent cooperative tasks as compared to baseline methods.

Multiagent reinforcement learning (MARL) has recently attracted considerable attention from both academics and practitioners. Core issues, e.g., the curse of dimensionality due to the exponential growth of agent interactions and nonstationary environments due to simultaneous learning, hinder the large-scale proliferation of MARL. These problems deteriorate with an increased number of agents. To address these challenges, we propose an adversarial collaborative learning method in a mixed cooperative-competitive environment, exploiting friend-or-foe Q-learning and mean-field theory. We first treat neighbors of agent i as two coalitions ( i 's friend and opponent coalition, respectively), and convert the Markov game into a two-player zero-sum game with an extended action set. By exploiting mean-field theory, this new game simplifies the interactions as those between a single agent and the mean effects of friends and opponents. A neural network is employed to learn the optimal mean effects of these two coalitions, which are trained via adversarial max and min steps. In the max step, with fixed policies of opponents, we optimize the friends' mean action to maximize their rewards. In the min step, the mean action of opponents is trained to minimize the friends' rewards when the policies of friends are frozen. These two steps are proved to converge to a Nash equilibrium. Then, another neural network is applied to learn the best response of each agent toward the mean effects. Finally, the adversarial max and min steps can jointly optimize the two networks. Experiments on two platforms demonstrate the learning effectiveness and strength of our approach, especially with many agents.

Object instance segmentation in traffic scenes is an important research topic. For training instance segmentation models, synthetic data can potentially complement real data, alleviating manual effort on annotating real images. However, the data distribution discrepancy between synthetic data and real data hampers the wide applications of synthetic data. In light of that, we propose a virtual-real interaction method for object instance segmentation. This method works over synthetic images with accurate annotations and real images without any labels. The virtual-real interaction guides the model to learn useful information from synthetic data while keeping consistent with real data. We first analyze the data distribution discrepancy from a probabilistic perspective, and divide it into image-level and instance-level discrepancies. Then, we design two components to align these discrepancies, i.e., global-level alignment and local-level alignment. Furthermore, a consistency alignment component is proposed to encourage the consistency between the global-level and the local-level alignment components. We evaluate the proposed approach on the real Cityscapes dataset by adapting from virtual SYNTHIA, Virtual KITTI, and VIPER datasets. The experimental results demonstrate that it achieves significantly better performance than state-of-the-art methods.

It is widely recognized that connected vehicles have the potential to further improve the road safety, transportation intelligence and enhance the in-vehicle entertainment. By leveraging the 5G enabled Vehicular Ad hoc NETworks (VANET) technology, which is referred to as 5G-VANET, a flexible software-defined communication can be achieved with ultra-high reliability, low latency, and high capacity. Many enabling applications in 5G-VANET rely on sharing mobile data among vehicles, which is still a challenging issue due to the extremely large data volume and the prohibitive cost of transmitting such data using 5G cellular networks. This article focuses on efficient cooperative data sharing in edge computing assisted 5G-VANET. First, to enable efficient cooperation between cellular communication and Dedicated Short-Range Communication (DSRC), we first propose a software-defined cooperative data sharing architecture in 5G-VANET. The cellular link allows the communications between OpenFlow enabled vehicles and the Controller to collect contextual information, while the DSRC serves as the data plane, enabling cooperative data sharing among adjacent vehicles. Second, we propose a graph theory based algorithm to efficiently solve the data sharing problem, which is formulated as a maximum weighted independent set problem on the constructed conflict graph. Specifically, considering the continuous data sharing, we propose a balanced greedy algorithm, which can make the content distribution more balanced. Furthermore, due to the fixed amount of computing resources allocated to this software-defined cooperative data sharing service, we propose an integer linear programming based decomposition algorithm to make full use of the computing resources. Extensive simulations in NS3 and SUMO demonstrate the superiority and scalability of the proposed software-defined architecture and cooperative data sharing algorithms.

The existing end-to-end methods make driving decisions mainly based on the vehicle's own perceived data, which cannot avoid hazards in blind zones. To fill this gap, vehicles should cooperate to construct a comprehensive environment perception by sharing information among each other, equipping each vehicle with see-through ability. While bringing more perceived information, data from other sources may also interfere feature selection and make decision making more difcult. To solve this problem, we propose a dual-attention-based neural network by utilizing two different attention modules. The first module is designed for each source to eliminate redundant features in perception and generate cognitive information for sharing. Since the influences of different cognition on the decision making are different under different circumstances, the second module is used to discriminate the importance of different cognition and focus on the dominant one as needed. Guided by the dual-attention-weighted features, the proposed network extracts the most salient features from the multi-source data, which leads to a signicant reduction of false response in steering angle controlling. Extensive experiments have demonstrated the superior performance of our proposed method, as compared with several state-of-the-arts.

This article investigates the coded cooperative data exchange (CCDE) problem, where a set of nodes initially hold a subset of packets and wish to retrieve all desired packets via direct wireless communication with neighbors. The CCDE problem in multihop wired networks has seen significant research recently and is proved to be NP-hard. The CCDE problem in multihop wireless networks (MWNs) must additionally consider half-duplex constraint, interference constraint, and channel constraint. Channel assignment brings new challenges to the CCDE problem in MWN, since it is also a well-known NP-hard problem even with one channel. In this article, we study the CCDE problem in MWN with multiple channels. We first construct a path network to evaluate the priority for each possible transmission and then construct a conflict graph (CG). This graph depicts the half-duplex, interference, and channel constraints. After that, a greedy channel assignment algorithm is proposed to obtain the nonconflict transmissions based on the constructed CG and assign a channel for each selected transmission. Finally, based on the selected transmissions, we construct a single-source multicast network exploiting the time expanded network, with which the network encoding and decoding schemes can be computed within polynomial time. Extensive simulations are conducted to demonstrate the efficiency of the proposed algorithm.

The infrastructure to vehicle (I2V) communication boosts a large number of prevailing vehicular services, which can provide vehicles with external information, storage, and computing power located at both mobile edge server (MES) and remote cloud. However, vehicle distribution is imbalanced due to the spatial inhomogeneity and temporal dynamics. As a consequence, the communication load for MES is imbalanced and vehicles may suffer from poor I2V communications where the MES is overloaded. In this paper, we propose a novel proactively load balancing approach that enables efficient cooperation among MESs, which is referred to as end-to-end load balancer (E2LB). E2LB schedules the cached data among MESs based on the predicted road traffic situation. First, a convolutional neural network (CNN) is applied to efficiently learn the spatio-temporal correlation in order to predict the road traffic situation. Then, we formulate the load balancing problem as a nonlinear programming (NLP) problem and a novel framework based on CNN is adopted to approximate the NLP optimization. Finally, we connect the above neural networks into an end-to-end neural network to jointly optimize the performance, where the input is the historical traffic situation while the output is the balanced scheduling solution. E2LB can guarantee the real-time scheduling, since the calling of a well-trained neural network only requires a small number of simple operations. Experiments on the trajectories of taxis and buses in Beijing demonstrate the efficiency and effectiveness of E2LB.

The performance of a vehicular ad hoc network (VANET) largely depends on the underlying medium access control (MAC), as it determines the schedule utility of physical resources. However, in existing time-division multiple access (TDMA)-based MAC protocols, a node usually acquires slots based on what each node senses, which is typically the coupling of the control and data plane. This coupling makes the TDMA protocols unable to rapidly and agilely deal with the challenges in VANETs, such as high mobility and dynamic network densities. Inspired by the software-defined network (SDN), we propose a novel SDN-based MAC protocol, named sdnMAC, to handle these challenges. A novel roadside openflow switch (ROFS) is designed as the roadside unit, controlled by the openflow controller. The sdnMAC can be divided into two tiers, the management of ROFSes (MA-ROFS) by the controller and the management of vehicles (MA-VEH) by ROFSes. In MA-ROFS, the controller schedules the cooperative sharing of time slot information among ROFSes. In MA-VEH, each ROFS allocates slots based on this shared information, thus decoupling of the control and data plane. This decoupling provides great rapidness and agility to sdnMAC, thus handling the rapid mobility and varying vehicle densities. Extensive simulations using network simulator NS3 and traffic simulator SUMO are performed. It is shown that the sdnMAC protocol can better meet the requirements of cooperative safety in VANETs.

We investigate the problem of physical layer security in a wireless cooperative network, where communication is assisted by the untrusted relay. It is impossible for the amplify-and-forward protocol to convey a confidential message from the source to the destination without the help of other nodes, where only an untrusted relay is available. Nonetheless, our results are quite optimistic when there are multiple untrusted relays, which amplify the received signal and forward it to the destination. We treat the untrusted relay nodes as eavesdroppers, despite the fact that they are the enablers of communication. First of all, we prove that a positive secrecy rate can always be ensured regardless of the transmit power and the channel condition of the untrusted relays, as long as there exists a sufficient number of untrusted relays. Furthermore, a closed-form solution to the upper bound of secrecy rate is obtained. Finally, a practical approach is proposed to acquire a positive secrecy rate, which brings nearly no extra communication cost. Simulations are conducted to demonstrate the validity of the proposed approach.

In this paper, we propose a hierarchical architecture based on software defined networking (SDN) to manage the physical resources in vehicular ad-hoc networks (VANETs), namely sdnMAC. First of all, a novel roadside unit (denoted by ROFS) is designed, which is an OpenFlow switch equipped with a wireless interface. Then, a hierarchical architecture is proposed for sdnMAC, consisting of two tiers, one is the management of the ROFSs by the Controller, the other is management of vehicles by ROFSs. Due to the cooperative share of slots information, sdnMAC can provide pre-warning of collisions and agility to topology change and varying densities of vehicles.