3D reconstruction is a cornerstone for creating immersive digital experiences in metaverse. Owing to explicit scene representation and efficient rendering, Gaussian splatting (GS) has become a prominent research focus in 3D reconstruction. However, the input images for GS are often imperfect, as those collected via highly-interfered wireless environment (HIWE) tend to be distorted, thereby undermining the accuracy of 3D reconstruction and limiting scalability. This paper proposes a novel Gaussian semantic splatting (GSS) scheme, designed for agile, reliable, and communication-efficient 3D reconstruction in the metaverse. Specifically, the semantic communication encoder/decoder (SCED) within GSS performs sequential semantic encoding and channel encoding using the proposed reliable and efficient semantic communication (RESC) algorithm, enabling the receiver to recover images with near-perfect accuracy. These images are then processed by the memory-efficient Gaussian renderer (MEGR), which employs an agile Gaussian splatting rendering (AGSR) algorithm to complete the 3D reconstruction and render a series of new viewpoint images. Additionally, a semantic control unit (SCU) is designed to oversee the components, enhancing the overall efficiency of the 3D reconstruction process. Experimental results demonstrate that GSS achieves competitive 3D reconstruction quality in HIWE, delivering real-time rendering speeds of 145 FPS at an$800\times 800$resolution while reducing storage memory overhead by more than 12 times compared to the state-of-the-art (SOTA) scheme.

Recently, there has been growing interest in using low-cost sensor combinations, such as cameras and IMUs, to achieve accurate localization within pre-built pointcloud maps. In this paper, we propose a novel hybrid visual-Lidar mapping and visual-only re-localization framework, specifically designed for UAVs with limited computational resources operating in challenging environments. Keyframes function as a bridge in our system, associating images with pointcloud to facilitate efficient and accurate pose estimation. Besides, our system creates omnidirectional keyframes at the mapping stage, enabling effective re-localization from any orientation, which enhance the robustness and practicability of our system. Experiments show that the proposed algorithm achieves high localization accuracy on pre-built maps and is capable of running in real-time on UAVs for autonomous navigation tasks. The source code will be made publicly available soon.

Facial expression recognition (FER) is a challenging task due to pervasive occlusion and dataset biases. Especially when facial information is partially occluded, existing FER models struggle to extract effective facial features, leading to inaccurate classifications. In response, we present ORSANet, which introduces the following three key contributions: First, we introduce auxiliary multi-modal semantic guidance to disambiguate facial occlusion and learn high-level semantic knowledge, which is two-fold: 1) we introduce semantic segmentation maps as dense semantics prior to generate semantics-enhanced facial representations; 2) we introduce facial landmarks as sparse geometric prior to mitigate intrinsic noises in FER, such as identity and gender biases. Second, to facilitate the effective incorporation of these two multi-modal priors, we customize a Multi-scale Cross-interaction Module (MCM) to adaptively fuse the landmark feature and semantics-enhanced representations within different scales. Third, we design a Dynamic Adversarial Repulsion Enhancement Loss (DARELoss) that dynamically adjusts the margins of ambiguous classes, further enhancing the model's ability to distinguish similar expressions. We further construct the first occlusion-oriented FER dataset to facilitate specialized robustness analysis on various real-world occlusion conditions, dubbed Occlu-FER. Extensive experiments on both public benchmarks and Occlu-FER demonstrate that our proposed ORSANet achieves SOTA recognition performance. Code is publicly available at https://github.com/Wenyuzhy/ORSANet-master.

The Approximate Nearest Neighbor (ANN) search is a cornerstone in high-dimensional data processing, which, however, remains challenging in outsourced scenarios due to potential result and data tampering. Recent studies on verifiable search have partially offered verification guarantees, while they suffer from limited scalability or fail to address result integrity simultaneously. In this paper, we propose VLAH, a lightweight and verifiable framework for ANN search. VLAH integrates locality-sensitive hashing with a compressed trie structure for efficient bucket verification and employs the in-bucket HNSW index to accelerate candidate retrieval. To ensure result integrity, we design a hybrid verification object that combines Merkle-based prefix proofs, neighbor-bucket consistency, and graph search trace validation. By integrating blockchain technology, we further enhance the integrity of the scheme. We theoretically analyze the complexity of our scheme and the correctness and integrity guarantees of the query results in lightweight verification. Experimental results on real high-dimensional datasets show that VLAH achieves competitive search performance with significantly reduced verification costs, bridging the gap between efficiency and verifiability in large-scale ANN systems.

The research on signal coordination has been greatly enriched over the last decade. However, existing contributions face inherent limitations such as weak connection between objectives and common measurements of effectiveness (MOEs) caused by insufficient modeling of traffic dynamics, invariable phase splits, and great demand on hyperparameters. Meanwhile, nearly all related works are concentrated on scenarios with only under-saturated phases. Therefore, an arterial signal coordination model for minimum level of over-saturation and stops is proposed. Unlike most related works, the proposed model focuses on minimizing phase over-saturation and total stops by estimating queue profile for all phases under variable signal plans. The model is initially formulated as a mixed-integer nonlinear programming (MINLP). By applying linearization techniques, it is then transformed into a mixed-integer linear programming (MILP). Simulation experiments are carried out in SUMO, where an artery is built with eight scenarios of different traffic demand. The results indicate that the model is more competent in reducing average delay (AD), average stops (AS) and average total travel time (ATTT) than Yang’s multi-path progression model for all scenarios. It is also verified to best MP-BAND by managing obvious reduction in AS and showing advantage in decreasing AD and ATTT in most scenarios. Additionally, the proposed model is able to alleviate the level of over-saturation for an intersection by re-allocating phase splits properly, resulting in less over-saturated phases. Intuitive illustrations attest to the effectiveness of the queue estimation in the proposed model, highlighting the theoretical importance of modeling queue length as a variable.

In recent years, 3D Gaussian splatting (3D-GS) has emerged as a novel scene representation approach. However, existing vision-only 3D-GS methods often rely on hand-crafted heuristics for point-cloud densification and face challenges in handling occlusions and high GPU memory and computation consumption [1]. LiDAR-Inertial-Visual (LIV) sensor configuration has demonstrated superior performance in precise localization and dense mapping by leveraging complementary sensing characteristics: rich texture information from cameras, precise geometric measurements from LiDAR, and high-frequency motion data from IMU [2]-[8]. Inspired by this, we propose a novel real-time Gaussian-based simultaneous localization and mapping (SLAM) system. Our map system comprises a global Gaussian map and a sliding window of Gaussians, along with an IESKF-based real-time odometry utilizing Gaussian maps. The structure of the global Gaussian map consists of hash-indexed voxels organized in a recursive octree. This hierarchical structure effectively covers sparse spatial volumes while adapting to different levels of detail and scales in the environment. The Gaussian map is efficiently initialized through multi-sensor fusion and optimized with photometric gradients. Our system incrementally maintains a sliding window of Gaussians with minimal graphics memory usage, significantly reducing GPU computation and memory consumption by only optimizing the map within the sliding window, enabling real-time optimization. Moreover, we implement a tightly coupled multi-sensor fusion odometry with an iterative error state Kalman filter (IESKF), which leverages real-time updating and rendering of the Gaussian map to achieve competitive localization accuracy. Our system represents the first real-time Gaussian-based SLAM framework deployable on resource-constrained embedded systems (all implemented in C++/CUDA for efficiency), demonstrated on theNVIDIA Jetson Orin NXplatform. The framework achieves real-time performance while maintaining robust multi-sensor fusion capabilities. All implementation algorithms, hardware designs, and CAD models and demo video of our GPU-accelerated system will be publicly available athttps://github.com/HKUST-Aerial-Robotics/GS-LIVO.

Autonomous parking is a crucial task in the intelligent driving field. Traditional parking algorithms are usually implemented using rule-based schemes. However, these methods are less effective in complex parking scenarios due to the intricate design of the algorithms. In contrast, neural-network-based methods tend to be more intuitive and versatile than the rule-based methods. By collecting a large number of expert parking trajectory data and emulating human strategy via learning-based methods, the parking task can be effectively addressed. In this paper, we employ imitation learning to perform end-to-end planning from RGB images to path planning by imitating human driving trajectories. The proposed end-to-end approach utilizes a target query encoder to fuse images and target features, and a transformer-based decoder to autoregressively predict future waypoints. We conduct extensive experiments in real-world scenarios, and the results demonstrate that the proposed method achieved an average parking success rate of 87.8% across four different real-world garages. Real-vehicle experiments further validate the feasibility and effectiveness of the method proposed in this paper. The code can be found at: https://github.com/qintonguav/ParkingE2E.

By leveraging multi-view inputs to synthesize novel-view images, Neural Radiance Fields (NeRF) have emerged as a prominent technique in the realm of 3D object reconstruction. However, existing methods primarily focus on global scene reconstruction using large datasets, which necessitate substantial computational resources and impose high-quality requirements on input images. Nevertheless, in practical applications, users prioritize the 3D reconstruction results of on-demand specific object (OSO) based on their individual demands . Furthermore, the collected images transmitted through high-interference wireless environment (HIWE) leads to negatively impact the accuracy of NeRF reconstruction, thereby limiting its scalability. In this paper, we propose a novel on-demand Semantic Neural Radiance Fields (OSNeRF) scheme, which offers fast and robust 3D object reconstruction for diverse tasks. Within OSNeRF, semantic encoder is employed to extract core semantic features of OSOs from the collected scene images, semantic decoder is utilized to facilitate robust image recovery under HIWE conditions, lightweight renderer is employed for fast and efficient object reconstruction. Moreover, a semantic control unit (SCU) is introduced to guide above components, thereby enhancing the efficiency of reconstruction. Demonstrative experiments demonstrate that the proposed OSNeRF enables fast and robust object reconstruction in HIWE, surpassing the performance of state-of-the-art (SOTA) methods in terms of reconstruction quality.

This paper proposes an adversarial reinforcement learning (RL)-based traffic control strategy to improve the traffic efficiency of an integrated network with expressway and adjacent surface streets. The proposed adversarial RL integrates adversarial learning into a multi-agent RL model, namely the multi-agent advantage actor-critic (MA2C), so as to enhance the generalization of the control strategy against the mismatch between an offline-training environment and the real traffic process. In the adversarial RL, the RL model is trained to maximize the network throughput, while the adversarial network, which produces disturbances to observed traffic states, is trained based on the opposite reward of the RL model. The proposed control strategy is tested using the microscopic traffic simulation software, SUMO. To reproduce the difference between an offline-training environment and the real traffic process, two different traffic models in SUMO are used for offline-training and online-testing purposes, respectively. Simulation results demonstrate that the proposed approach performs better in reducing total time spent than the original MA2C approach, as well as a conventional feedback based controller.

Recently, Neural Radiance Fields (NeRF) achieved impressive results in novel view synthesis. Block-NeRF showed the capability of leveraging NeRF to build large city-scale models. For large-scale modeling, a mass of image data is necessary. Collecting images from specially designed data-collection vehicles can not support large-scale applications. How to acquire massive high-quality data remains an opening problem. Noting that the automotive industry has a huge amount of image data, crowd-sourcing is a convenient way for large-scale data collection. In this paper, we present a crowd-sourced framework, which utilizes substantial data captured by production vehicles to reconstruct the scene with the NeRF model. This approach solves the key problem of large-scale reconstruction, that is where the data comes from and how to use them. Firstly, the crowd-sourced massive data is filtered to remove redundancy and keep a balanced distribution in terms of time and space. Then a structure-from-motion module is performed to refine camera poses. Finally, images, as well as poses, are used to train the NeRF model in a certain block. We highlight that we presents a comprehensive framework that integrates multiple modules, including data selection, sparse 3D reconstruction, sequence appearance embedding, depth supervision of ground surface, and occlusion completion. The complete system is capable of effectively processing and reconstructing high-quality 3D scenes from crowd-sourced data. Extensive quantitative and qualitative experiments were conducted to validate the performance of our system. Moreover, we proposed an application, named first-view navigation, which leveraged the NeRF model to generate 3D street view and guide the driver with a synthesized video.