Vehicle-to-everything (V2X) technologies offer a promising paradigm to mitigate the limitations of constrained observability in single-vehicle systems. Prior work primarily focuses on single-frame cooperative perception, which fuses agents' information across different spatial locations but ignores temporal cues and temporal tasks (e.g., temporal perception and prediction). In this paper, we focus on the spatio-temporal fusion in V2X scenarios and design one-step and multi-step communication strategies (when to transmit) as well as examine their integration with three fusion strategies - early, late, and intermediate (what to transmit), providing comprehensive benchmarks with 11 fusion models (how to fuse). Furthermore, we propose V2XPnP, a novel intermediate fusion framework within one-step communication for end-to-end perception and prediction. Our framework employs a unified Transformer-based architecture to effectively model complex spatio-temporal relationships across multiple agents, frames, and high-definition maps. Moreover, we introduce the V2XPnP Sequential Dataset that supports all V2X collaboration modes and addresses the limitations of existing real-world datasets, which are restricted to single-frame or single-mode cooperation. Extensive experiments demonstrate that our framework outperforms state-of-the-art methods in both perception and prediction tasks.

The accurate identification of low-risk and high-risk lung tumors is essential for clinicians to develop lung cancer treatment strategies during surgery. Despite the achievements in deep learning, most medical image analysis applications are still hampered by the difficulty of obtaining large amounts of labeled data. In this paper, a semi-supervised deep learning framework, DS-FixMatch, is proposed for identifying lung tumors intra-operatively while alleviating the issue of sparse annotations. DS-FixMatch combines selective labeling and semi-supervised training: (1) For the acquired unlabeled images, a subset that best represents the distribution of the entire dataset is selected by an unsupervised algorithm. Compared to the traditional random labeling strategy, this method can avoid introducing samples that are highly influenced by the intraoperative environment for labeling. This subset is then sent to a human expert for labeling. (2) Supervised training is performed using labeled images. For the remaining unlabeled samples, DS-FixMatch utilizes model predictions to generate pseudo-labels for consistency regularization, further enhancing the model’s generalization ability. A dataset consisting of 2221 natural images, each capturing the Region of Interest (ROI) in lung tumors, is constructed to evaluate the effectiveness of the designed framework. Experiments show that DS-FixMatch leads in performance for the task of lung tumor recognition compared to other baselines.

Lung cancer is a most prevalent and deadly type of cancer worldwide. In clinical practice, it is crucial to accurately identify the risk level of lung cancer for diagnosing and treating lung lesions. Current AI methods for lung cancer image diagnosis mainly rely on CT images, histopathological images, and a few methods based on surgical lesion section images. However, there is a great need for methods for surgical lesion section images that can help surgeons quickly decide the surgical plan during the operation. Deep learning methods among them often supervise only the output layer of the network, while ignoring the constraints on the intermediate hidden layers, resulting in insufficient accuracy of high-risk and low-risk classification. This paper aims to design a method for the task of classifying surgical lesion section of lung tumours into high-risk and low-risk categories. The EDSNet model based on deep supervision is proposed, which enhances the performance of the hidden layer through deep supervision. Moreover, the MBConv module of baseline model is replaced with the SimAM-MBConv module proposed in this paper, which further enhances the representation of pathological features by deriving 3D attention weights for feature maps within one layer. Several experiments had been conducted on a clinical lung tumour surgical lesion section image dataset, achieving the classification accuracy of EDSNet is 95.90% with an AUC of 0.9935. Compared with most existing methods for similar tasks, our method is optimal in most of the evaluation metrics. The experimental results demonstrate the effectiveness and sophistication of our method for high-risk and low-risk classification of surgical section images of lung tumours, which can serve as a reference for doctors to determine the subsequent surgical plans and treatment strategies.

Cooperative perception algorithms based on fusing sensing data across multiple connected automated vehicles (CAVs) have shown promising performance to enhance the existing individual perception algorithm in terms of object detection tracking. However, existing cooperative perception algorithms are only developed offline given the constraints from data sharing and computational resources and none of them have been verified in real-time conditions. In this work, we propose a real-time cooperative perception framework called CooperFuse, which achieves cooperative perception in a late fusion scheme. Based on object detection and tracking results from individual vehicle, the late fusion cooperative perception algorithm considers object detection confidence score, kinematics, and dynamics consistency as well as scale consistency of detected objects. The algorithm computes the kinematic and dynamic consistency of the objects by solving for the energy consumption of inter-frame trajectories, and determines scale consistency by calculating inter-frame scale changes, enabling feature-based bounding box fusion. The experimental results demonstrate the real-time performance of the proposed algorithm and reveal its effective improvements in feature fusion and object detection accuracy when dealing with heterogeneous detection models across different cooperative intelligent agents.

Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance. With the rapid growth of autonomous vehicles and intelligent infrastructure, the V2X perception systems will soon be deployed at scale, which raises a safety-critical question: how can we evaluate and improve its performance under challenging traffic scenarios before the real-world deployment? Collecting diverse large-scale real-world test scenes seems to be the most straightforward solution, but it is expensive and time-consuming, and the collections can only cover limited scenarios. To this end, we propose the first open adversarial scene generator V2XP-ASG that can produce realistic, challenging scenes for modern LiDAR-based multi-agent perception systems. V2XP-ASG learns to construct an adversarial collaboration graph and simultaneously perturb multiple agents' poses in an adversarial and plausible manner. The experiments demonstrate that V2XP-ASG can effectively identify challenging scenes for a large range of V2X perception systems. Meanwhile, by training on the limited number of generated challenging scenes, the accuracy of V2X perception systems can be further improved by 12.3% on challenging and 4% on normal scenes. Our code will be released at https://github.com/XHwind/V2XP-ASG.

Modern autonomous systems are purposed for many challenging scenarios, where agents will face unexpected events and complicated tasks. The presence of disturbance noise with control command and unknown inputs can negatively impact robot performance. Previous research of joint input and state estimation separately studied the continuous and discrete cases without any prior information. This paper combines the continuous and discrete input cases into a unified theory based on the Expectation-Maximum (EM) algorithm. By introducing prior knowledge of events as the constraint, inequality optimization problems are formulated to determine a gain matrix or dynamic weights to realize an optimal input estimation with lower variance and more accurate decision-making. Finally, statistical results from experiments show that our algorithm owns 81% improvement of the variance than KF and 47% improvement than RKF in continuous space; a remarkable improvement of right decision-making probability of our input estimator in discrete space, identification ability is also analyzed by experiments.