3D visual grounding aims to localize target objects in point clouds based on free-form natural language, which often describes both target and reference objects. Effective alignment between visual and text features is crucial for this task. However, existing two-stage methods that rely solely on object-level features can yield suboptimal accuracy, while one-stage methods that align only point-level features can be prone to noise. In this paper, we propose DDPA-3DVG, a novel framework that progressively aligns visual locations and language descriptions at multiple granularities. Specifically, we decouple natural language descriptions into distinct representations of target objects, reference objects, and their mutual relationships, while disentangling 3D scenes into object-level, voxel-level, and point-level features. By progressively fusing these dual-decoupled features from coarse to fine, our method enhances cross-modal alignment and achieves state-of-the-art performance on three challenging benchmarks—ScanRefer, Nr3D, and Sr3D. The code will be released at https://github.com/HDU-VRLab/DDPA-3DVG.

Neural Radiance Field (NeRF) has emerged as a leading method for reconstructing 3D human avatars with exceptional rendering capabilities, particularly for novel view and pose synthesis. However, current approaches for editing these avatars are limited, typically allowing only global geometry adjustments or texture modifications via neural texture maps. This paper introduces Unified Volumetric Avatar, a novel framework enabling independent and simultaneous global and local editing of both geometry and texture of 3D human avatars and user-friendly manipulation. The proposed approach seamlessly integrates implicit neural fields with an explicit polygonal mesh , leveraging distinct geometry and appearance latent codes attached to the body mesh for precise local edits. These trackable latent codes permeate through the 3D space via barycentric interpolation, mitigating spatial ambiguity with the aid of a local signed height indicator. Furthermore, our method enhances surface illumination representation across different poses by incorporating a pose-dependent shading factor instead of relying on view-dependent radiance color. Experimental results on multiple human avatars demonstrate its efficacy in achieving competitive results for novel view synthesis and novel pose rendering, showcasing its potential for versatile human representation. The source code will be made publicly available.

Safe urban driving is challenging due to the high density of traffic flow and various potential hazards, such as the sudden appearance of unknown objects. Traditional rule-based approaches and imitation learning methods struggle to address the diverse driving scenarios encountered in urban environments. Reinforcement learning (RL), which adapts to a wide range of driving scenarios through continuous interaction with the environment, has demonstrated success in autonomous driving. Making safety decisions when driving in urban environments necessitates a comprehensive perception of the current scene and the ability to predict the evolution of the dynamic scene. In this paper, we present a novel cascade deep reinforcement learning framework, CDRP3, designed to enhance the safety decision-making capabilities of self-driving vehicles in complex scenarios and emergencies. We leverage a multi-modal spatio-temporal perception (MmSTP) module to fuse multi-modal sensor data and introduce temporal perception to capture spatio-temporal information about dynamic driving environments, and a future state prediction (FSP) module to model complex interactions between different traffic participants and explicitly predict their future states. Subsequently, in the PPO-based planning module, we use the comprehensive environmental information obtained from perception and prediction to decode an optimized driving strategy using a lateral and longitudinal separated multi-branch network structure guided by a customized reward function. This approach enables knowledge transfer from the perception and prediction components to planning, and planning-oriented enhancement of safety decision-making capabilities to improve driving safety. Our experiments demonstrate that CDRP3 outperforms state-of-the-art methods, providing superior driving safety in complex urban environments.

Although human reconstruction typically results in human-specific avatars, recent 3D scene reconstruction techniques utilizing pixel-aligned features show promise in generalizing to new scenes. Applying these techniques to human avatar reconstruction can result in a volumetric avatar with generalizability but limited animatability due to rendering only being possible for static representations. In this paper, we propose AniPixel, a novel animatable and generalizable human avatar reconstruction method that leverages pixel-aligned features for body geometry prediction and RGB color blending. Technically, to align the canonical space with the target space and the observation space, we propose a bidirectional neural skinning field based on skeleton-driven deformation to establish the target-to-canonical and canonical-to-observation correspondences. Then, we disentangle the canonical body geometry into a normalized neutral-sized body and a subject-specific residual for better generalizability. As the geometry and appearance are closely related, we introduce pixel-aligned features to facilitate the body geometry prediction and detailed surface normals to reinforce the RGB color blending. We also devise a pose-dependent and view direction-related shading module to represent the local illumination variance. Experiments show that AniPixel renders comparable novel views while delivering better novel pose animation results than state-of-the-art methods. Code will be released at AniPixel. https://github.com/loong8888/AniPixel.

Deep learning has demonstrated its power in image rectification by leveraging the representation capacity of deep neural networks via supervised training based on a large-scale synthetic dataset. However, the model may overfit the synthetic images and generalize not well on real-world fisheye images due to the limited universality of a specific distortion model and the lack of explicitly modeling the distortion and rectification process. In this paper, we propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of a same scene from different lenses should be the same. Specifically, we devise a new network architecture with a shared encoder and several prediction heads, each of which predicts the distortion parameter of a specific distortion model. We further leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters and exploit the intra- and inter-model consistency between them during training, thereby leading to a self-supervised learning scheme without the need for ground-truth distortion parameters or normal images. Experiments on synthetic dataset and real-world fisheye images demonstrate that our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art (SOTA) methods. The proposed self-supervised method also provides a possible way to improve the universality of distortion models while keeping their self-consistency. Code and datasets will be available at https://github.com/loong8888/SIR.