High spatio‑temporal resolution novel‑view scene rendering is crucial for applications such as sports analysis and scientific experiments. However, existing Dynamic Scene Rendering (DSR) approaches typically rely on conventional RGB cameras with limited frame rates, making it difficult to achieve high spatio‑temporal resolution. In this paper, we present BulletTime4D, a high spatio‑temporal resolution DSR framework, which is the first trial to integrate a spike camera with binocular RGB cameras for dynamic scene reconstruction. Specifically, we first develop a hybrid camera prototype and build a real‑world dynamic scene reconstruction dataset. Then, BulletTime4D presents a multi‑timescale deformation representation by combining low‑frequency spatio‑temporal features with high‑frequency inter‑frame motion features. Finally, a rendering network is designed capable of projecting 4D Gaussians into the spike domain for spike rendering, and a cross‑domain supervision strategy is proposed to achieve high‑frame‑rate texture and color rendering. The results show that BulletTime4D outperforms state‑of‑the‑art methods on both simulated and real‑world datasets. In addition, BulletTime4D can synthesize 300 FPS novel‑view renderings using stereo RGB cameras at 30 FPS and a single spike camera.

Visual reinforcement learning (RL) suffers from poor sample efficiency due to high-dimensional observations in complex tasks. While existing works have shown that vision-language models (VLMs) can assist RL, they often focus on knowledge distillation from the VLM to RL, overlooking the potential of RL-generated interaction data to enhance the VLM. To address this, we propose COVR, a collaborative optimization framework that enables the mutual enhancement of the VLM and RL policies. Specifically, COVR fine-tunes the VLM with RL-generated data to enhance the semantic reasoning ability consistent with the target task, and uses the enhanced VLM to further guide policy learning via action priors. To improve fine-tuning efficiency, we introduce two key modules: (1) an Exploration-Driven Dynamic Filter module that preserves valuable exploration samples using adaptive thresholds based on the degree of exploration, and (2) a Return-Aware Adaptive Loss Weight module that improves the stability of training by quantifying the inconsistency of sampling actions via return signals of RL. We further design a progressive fine-tuning strategy to reduce resource consumption. Extensive experiments show that COVR achieves strong performance across various challenging visual control tasks.

Social media's openness fosters opinion exchange but complicates guiding users toward global consensus. Existing approaches are often invasive, modifying opinions or forcing cross-group interactions, undermining autonomy and triggering resistance. They also lack long-term planning, risking macro-level polarization despite local agreement. To address this, we propose H-NeiFi, a hierarchical, non-invasive opinion guidance framework. It models experts and non-experts separately and introduces a neighbor filtering mechanism that adaptively shapes communication without altering user opinions. Using multi-agent reinforcement learning with a long-term reward, H-NeiFi optimizes information flow while preserving interaction autonomy. Experiments show that it accelerates consensus by 22.0% to 30.7% and achieves global convergence even without experts, providing a natural and efficient paradigm for social network governance. Our code is available at: https://github.com/shijunguo44/H-NeiFi.

Existing end-to-end methods for steering angle prediction (SAP) primarily rely on RGB imagery from conventional cameras as input; however, they suffer from limitations such as poor performance in low-light conditions and motion blur. Recently, event cameras have garnered attention as complementary to RGB imagery, providing advantages such as high dynamic range and low latency. Nevertheless, earlier SAP methods that integrate event and RGB data may not fully exploit the spatio-temporal characteristics of events, resulting in performance degradation in low-light scenarios affected by noise interference. To address this limitation, we present a novel spatiotemporal-aware event-RGB fusion method for SAP, referred to as SERF, which aims to enhance the accuracy of event-based SAP. Specifically, SERF introduces three key components: 1) An innovative multi-layer Interaction Module based on attention mechanisms to fuse the multi-frame data, enabling more fine-grained feature processing; 2) a dynamic spatiotemporal mask mechanism, focusing RGB’s attention on spatially proximate events while diminishing the influence of temporally distant events, thereby reducing the impact of noise; and 3) a Memory Module that utilizes learnable tokens to accumulate essential latent fusion features through dynamic feature consolidation. Extensive experiments conducted on a variety of real-world and simulated datasets demonstrate the superior performance of SERF compared to the state-of-the-art methods. The experiments also validate the advantages of SERF in terms of inference performance, meeting the real-time requirements for actual deployment.

Rendering complex dynamic scenes offers the advantage of observing and understanding the real world. However, existing Dynamic Scene Rendering (DSR) methods remain challenged by suboptimal reconstruction fidelity. These limitations stem from relying on a single, unified deformation model, which struggles to capture complex motions involving multiple sub-motions and abrupt geometric transitions. While temporal decomposition methods could alleviate such shortcomings, they introduce the additional challenge of ignoring motion correlations and increasing storage requirements. To address these issues, we introduce Keyframe-driven Predictable Gaussian Splatting (KPGS)-an efficient framework for high-fidelity complex dynamic scene rendering. First, we present a patch-wise HSV clustering for extracting keyframes. Second, a prediction network based on the Transformer is utilized to calculate the deformable Gaussians at discrete keyframe times via voxelization. Third, we propose an inter-frame deformation network and a mutual supervision between adjacent segments to maintain the temporal continuity. Extensive experiments on our newly built dataset (MotionGS), as well as public benchmarks HyperNeRF and Neu3D, demonstrate that KPGS could achieve a higher average view synthesis performance than SOTA approaches, while maintaining a balance between storage cost and performance. More details of the demo and dataset are available at KPGS Supplementary.

Occupancy networks aim to reconstruct the surroundings with occupied semantic voxels. However, frequent object occlusions often occur in dynamic real-world scenarios, which cannot be captured by independent frames. Most existing occupancy networks generate results without explicitly considering past occupancy states and continuous visual changes over time, limiting their temporal accuracy. We tackle it by treating the task from a new continuous updating perspective, which considers historical data and continuous motion clues. We propose a new approach termed Continuous Motion clue exploitation for Occupancy Prediction (CMOP), which incorporates three key designs: (i) Propagator: which forecasts future occupancy states based on historical data; (ii) Tracker: which updates the occupancy on a per-frame basis using dynamic visual motion information; and (iii) Fuser: which aggregates results from the Propagator and Tracker into more robust and accurate occupancy results. Experiments on several benchmarks demonstrate that CMOP outperforms state-of-the-art baselines.

Vision-based Reinforcement Learning (VRL) attempts to establish associations between visual inputs and optimal actions through interactions with the environment. Given the high-dimensional and complex nature of visual data, it becomes essential to learn a policy based on high-quality state representation. To this end, existing VRL methods primarily rely on interaction-collected data, combined with selfsupervised auxiliary tasks. However, two key challenges remain: limited data samples and a lack of task-relevant semantic constraints. To tackle these challenges, we propose DGC, a method that Distills Guidance from Visual Language Models (VLMs) alongside self-supervised learning into a Compact VRL agent. Notably, we leverage the state representation capabilities of VLMs, rather than their decision-making abilities. Within DGC, a novel promptingreasoning pipeline is designed to convert historical observations and actions into usable supervision signals, enabling semantic understanding within the compact visual encoder. By leveraging these distilled semantic representations, the VRL agent achieves significant improvements in sample efficiency. Extensive experiments on the Carla benchmark demonstrate our state-of-the-art performance.

Spike camera with high temporal resolution offers a new perspective on high-speed dynamic scene rendering. Most existing rendering methods rely on Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) for static scenes using a monocular spike camera. However, these methods struggle with dynamic motion, while a single camera suffers from limited spatial coverage, making it challenging to reconstruct fine details in high-speed scenes. To address these problems, we propose Spike4DGS, the first high-speed dynamic scene rendering framework with 4D Gaussian Splatting using spike camera arrays. Technically, we first build a multi-view spike camera array to validate our solution, then establish both synthetic and real-world multi-view spike-based reconstruction datasets. Then, we design a multi-view spike-based dense initialization module that obtains dense point clouds and camera poses from continuous spike streams. Finally, we propose a spike-pixel synergy constraint supervision to optimize Spike4DGS, incorporating both rendered image quality loss and dynamic spatiotemporal spike loss. The results show that our Spike4DGS outperforms state-of-the-art methods in terms of novel view rendering quality on both synthetic and real-world datasets. More details are available at https://github.com/Qinghongye/Spike4DGS.

The task of live video analytics relies on real-time object tracking that typically involves computationally expensive deep neural network (DNN) models. In practice, it has become essential to process video data on edge devices deployed near the cameras. However, these edge devices often have very limited computing resources and thus suffer from poor tracking accuracy. Through a measurement study, we identify three major factors contributing to the performance issue: outdated detection results, tracking error accumulation, and ignorance of new objects. We introduce a novel approach, called Predict & Correct based Tracking, orPCTrack, to systematically address these problems. Our design incorporates three innovative components: 1) a Predictive Detection Propagator that rapidly updates outdated object bounding boxes to match the current frame through a lightweight prediction model; 2) a Frame Difference Corrector that refines the object bounding boxes based on frame difference information; and 3) a New Object Detector that efficiently discovers newly appearing objects during tracking. Experimental results show that our approach achieves remarkable accuracy improvements, ranging from 19.4% to 34.7%, across diverse traffic scenarios, compared to state of the art methods.

Modern intelligent vehicles (IVs) are equipped with a variety of sensors and communication modules, empowering Advanced Driver Assistance Systems (ADAS) and enabling inter-vehicle connectivity. This paper focuses on multi-vehicle cooperative perception, with a primary objective of achieving low latency. The task involves nearby cooperative vehicles sending their camera data to an edge server, which then merges the local views to create a global traffic view. While multi-camera perception has been actively researched, existing solutions often rely on deep learning models, resulting in excessive processing latency. In contrast, we propose leveraging thestate estimationtechnique from the robotics field for this task. We explicitly model and solve for the system state, addressing additional challenges brought by object mobility and vision obstruction. Furthermore, we introduce aprogressive state estimationpipeline to further accelerate system state notifications, supported by a motion prediction method that optimizes position accuracy and perception smoothness. Experimental results demonstrate the superiority of our approach over the deep learning method, with 12.0 × to 27.4 × reductions in server processing delay, while maintaining mean absolute errors below 1 m.

Multi-agent trajectory prediction is essential in autonomous driving, risk avoidance, and traffic flow control. However, the heterogeneous traffic density on interactions, which caused by physical laws, social norms and so on, is often overlooked in existing methods. When the density varies, the number of agents involved in interactions and the corresponding interaction probability change dynami-cally. To tackle this issue, we propose a new method, called Density-Adaptive Model based on Motif Matrix for Multi-Agent Trajectory Prediction (DAMM), to gain insights into multi-agent systems. Here we leverage the motif matrix to represent dynamic connectivity in a higher-order pattern, and distill the interaction information from the perspectives of the spatial and the temporal dimensions. Specifically, in spatial dimension, we utilize multi-scale feature fusion to adaptively select the optimal range of neighbors participating in interactions for each time slot. In temporal dimension, we extract the temporal interaction features and adapt a pyramidal pooling layer to generate the interaction probability for each agent. Experimental results demonstrate that our approach surpasses state-of-the-art methods on autonomous driving dataset.

We explore visual reinforcement learning (RL) using two complementary visual modalities: frame-based RGB cam-era and event-based Dynamic Vision Sensor (DVS). Ex-isting multi-modality visual RL methods often encounter challenges in effectively extracting task-relevant information from multiple modalities while suppressing the in-creased noise, only using indirect reward signals instead of pixel-level supervision. To tackle this, we propose a Decomposed Multi-Modality Representation (DMR) framework for visual RL. It explicitly decomposes the inputs into three distinct components: combined task-relevant features (co-features), RGB-specific noise, and DVS-specific noise. The co-features represent the full information from both modalities that is relevant to the RL task; the two noise components, each constrained by a data reconstruction loss to avoid information leak, are contrasted with the co-features to maximize their difference. Extensive experiments demonstrate that, by explicitly separating the different types of information, our approach achieves substan-tially improved policy performance compared to state-of-the-art approaches.

In autonomous driving, cooperative perception through vehicle-to-vehicle (V2V) communication is considered crucial for enhancing traffic safety and efficiency. However, existing methods often simplify the handling of perception data from multiple vehicles. In these approaches, the egovehicle aggregates observations from all neighboring connected cooperative vehicles (CCV), without considering the interactions between the vehicles or making differentiated use of the acquired sensing data. This approach can result in suboptimal performance due to the increase of noise and large transmission delay. In this paper, we introduce a novel approach to cooperative perception. By fusing both the road topology and trajectory histories of neighboring CCVs, our model learns an interaction score for each CCV. These scores prioritize vehicles that are most relevant to the current driving scenario, offering valuable guidance for selective fusion of sensor data, thereby enhancing driving decision-making. The proposed method is validated through experiments conducted on the CARLA simulator. Results demonstrate that our approach surpasses existing methods in terms of performance and robustness.

Nowadays, Connected Automated Vehicles (CAVs) have emerged as powerful infrastructures for the next-generation Intelligent Transportation System (ITS) as the rapid technological advancements of communication networks and vehicular intelligence. While prospective platoon-based techniques in CAVs, the heterogeneous traffic condition poses a challenge for platoon control in the self-organized traffic bottleneck, thus making an urgent need for a practical sustainable transportation architecture. To address this problem, we propose a software defined architecture that leverages multi-agent techniques to mobile-edge computing networks for multi-vehicle adaptive platoon, which is called SD-M3ASP. The architecture supports centralized and decentralized management of vehicular edge communication resources between mobile vehicles and edge devices, and underpins sustainable vehicular platooning capabilities. Then, we propose cluster-based kinematic models by grouping vehicles into multi-vehicle clusters (MVCs) to facilitate efficient platoon control with collision avoidance. Furthermore, we propose three-stage platoon control algorithms to adaptively balance the size of MVCs and form stable platoons in heterogeneous traffic flows. The intra-platoon and inter-platoon convergence are analyzed by using the Routh stability criterion and Lyapunov technique. A CAV simulation software is developed for demonstration purposes which is available online athttps://qgailab.com/cav-sim. Extensive numerical simulation results have shown the superiority of the proposed method, which can greatly eliminate the self-organized congestion caused by heterogeneous traffic flow.

With the rapid development of wireless communication and localization technologies, the easier collection of trajectory data can bring potential data-driven value. Recently, there has been an increasing interest in how to publish trajectory dataset without revealing personal information. However, since the large-scale and real-world sequential trajectory dataset presents a heterogeneous regional distribution, the existing study ignores the relationship between privacy budget allocation and spatial characteristics, resulting in unreasonable continuity and mapping distortion, and thus lowering the utility of the synthetic dataset. To address this problem, we propose a probability distribution model named Adaptive grid-based Weighted Differential Privacy (AWDP). First, trajectories are adaptively discretized into the multi-resolution grid structures to make trajectories more uniformly distributed and less disturbed by the noise. Second, we allocate different weighted budgets for different grids according to density-based regional characteristics. Third, a spatio-temporal continuity maintenance method is designed to solve unrealistic direction- and density-based continuity deviations of synthetic trajectories. An application system is developed for demonstration purposes which is available online athttp://qgailab.com/awdp/. The extensive experiments on three datasets demonstrate that AWDP performs significantly better than the state-of-the-art model in preserving the density distribution of the original trajectories with differential privacy guarantee and high utility.

Recent developments in multiagent consensus problems have heightened the role of network topology when the agent number increases largely. The existing works assume that the convergence evolution typically proceeds over a peer-to-peer architecture where agents are treated equally and communicate directly with perceived one-hop neighbors, thus resulting in slower convergence speed. In this article, we first extract the backbone network topology to provide a hierarchical organization over the original multiagent system (MAS). Second, we introduce a geometric convergence method based on the constraint set (CS) under periodically extracted switching-backbone topologies. Finally, we derive a fully decentralized framework named hierarchical switching-backbone MAS (HSBMAS) that is designed to conduct agents converge to a common stable equilibrium. Provable connectivity and convergence guarantees of the framework are provided when the initial topology is connected. Extensive simulation results on different-type and varying-density topologies have shown the superiority of the proposed framework.

Efficient motion and appearance modeling are critical for vision-based Reinforcement Learning (RL). However, existing methods struggle to reconcile motion and appearance information within the state representations learned from a single observation encoder. To address the problem, we present Synergizing Interactive Motion-appearance Understanding (Simoun), a unified framework for vision-based RL Given consecutive observation frames, Simoun deliberately and interactively learns both motion and appearance features through a dual-path network architecture. The learning process collaborates with a structural interactive module, which explores the latent motion-appearance structures from the two network paths to leverage their complementarity. To promote sample efficiency, we further design a consistency-guided curiosity module to encourage the exploration of under-learned observations. During training, the curiosity module provides intrinsic rewards according to the consistency of environmental temporal dynamics, which are deduced from both motion and appearance network paths. Experiments conducted on Deep-Mind control suite and CARLA automatic driving benchmarks demonstrate the effectiveness of Simoun, where it performs favorably against state-of-the-art methods.

Integrating RGB frames with alternative modality inputs is gaining increasing traction in many vision-based reinforcement learning (RL) applications. Existing multi-modal vision-based RL methods usually follow a Global Value Estimation (GVE) pipeline, which uses a fused modality feature to obtain a unified global environmental description. However, such a feature-level fusion paradigm with a single critic may fall short in policy learning as it tends to overlook the distinct values of each modality. To remedy this, this paper proposes a Local modality-customized Value Estimation (LVE) paradigm, which dynamically estimates the contribution and adjusts the importance weight of each modality from a value-level perspective. Furthermore, a task-contextual re-fusion process is developed to achieve a task-level re-balance of estimations from both feature and value levels. To this end, a Hierarchical Adaptive Value Estimation (HAVE) framework is formed, which adaptively coordinates the contributions of individual modalities as well as their collective efficacy. Agents trained by HAVE are able to exploit the unique characteristics of various modalities while capturing their intricate interactions, achieving substantially improved performance. We specifically highlight the potency of our approach within the challenging landscape of autonomous driving, utilizing the CARLA benchmark with neuromorphic event and depth data to demonstrate HAVE's capability and the effectiveness of its distinct components.