Direct 3D scene stylization from sparse views remains a significant challenge, as existing optimization-based methods are prohibitively slow and require dense inputs to prevent geometric corruption. While recent direct methods accelerate this process, their rigid decoupling of a static geometry from appearance often leads to visual artifacts, where stylistic textures conflict with and distort the underlying scene structure. To address these limitations, we introduce GeoStyler, a direct framework that generates high-fidelity, multi-view consistent stylized 3D scenes in seconds. Our approach reformulates the conventional pipeline by first leveraging a diffusion model to generate a set of geometrically consistent stylized 2D images. The core of this stage is a novel hybrid query formulation for the self-attention mechanism. Specifically, cross-view geometric information is directly embedded into the query to enforce 3D consistency, while style information is independently injected via the key and value to preserve scene structure. This process is further stabilized by a geometrically-aware latent initialization that provides a coherent starting point for the denoising process. Subsequently, a decoupled reconstruction network lifts these 2D stylized images to 3D Gaussians. A geometry branch predicts a robust 3D scaffold from the original content images, while a parallel style branch predicts the final appearance from our generated stylized images, ensuring structural integrity is not compromised. Extensive experiments on large-scale benchmarks, including RealEstate10K and ACID, demonstrate that GeoStyler significantly outperforms prior arts in stylization quality and multi-view consistency, achieving state-of-the-art performance with a dramatic speedup. Our project page: https://huhuhuxiao.github.io/Geo-Styler/.

Although 3D Gaussian Splatting (3DGS) has demonstrated promising results in novel view synthesis, its performance degrades dramatically with sparse inputs and generates undesirable artifacts. As the number of training views decreases, the novel view synthesis task degrades to a highly under-determined problem such that existing methods suffer from the notorious overfitting issue. Interestingly, we observe that models with fewer Gaussian primitives exhibit less overfitting under spare inputs. Inspired by this observation, we propose a Random Dropout Regularization (RDR) to exploit the advantages of low-complexity models to alleviate overfitting. In addition, to remedy the lack of high-frequency details for these models, an Edge-guided Splitting Strategy (ESS) is developed. With these two techniques, our method (termed DropoutGS) provides a simple yet effective plug-in approach to improve the generalization performance of existing 3DGS methods. Extensive experiments show that our DropoutGS produces state-of-the-art performance under sparse views on benchmark datasets including Blender, LLFF, and DTU. The project page is at: https://xuyx55.github.io/DropoutGS/.

Embodied artificial intelligence requires a wide variety of large-scale simulated environments for development. Previous scene reconstruction approaches based on multiview images can produce high-fidelity 3D scenes but lack diversity. In contrast, existing prompt-based scene generation approaches can produce diverse scenes but lack fine-grained controls. To bridge these two fields, the scene graph provides the key relationships within a scene, while offering flexible controls. However, 3D scene generation from scene graphs is challenging and under-explored. In this paper, we propose a scene graph-based 3D indoor scene generation method for the efficient simulated environment creation, that maintains both high diversity and fine-grained control. Specifically, we first introduce an interaction-aware scene graph to merge object nodes with hierarchical interaction relationships, which alleviates levitation and interference issues during the scene generation. Then, we employ the large language model (LLM) for instruction-driven 3D layout generation with carefully designed prompts. Finally, a 3D large generation model is utilized to generate the content for each node of the interaction-aware scene graph, which is then transformed based on the corresponding bounding box in the 3D layout. The experiments demonstrate that the proposed method achieves state-of-the-art performance on 3D indoor scene generation. Additionally, the proposed method exhibits fine-grained controls at the object level, while providing a high diversity of layouts, geometry, and textures.

Semantic instance completion aims to recover the complete 3D shapes of foreground objects together with their labels from a partial 2.5D scan of a scene. Previous works have relied on full supervision, which requires ground-truth annotations, in the form of bounding boxes and complete 3D objects. This has greatly limited their real-world application because the acquisition of ground-truth data is very costly and time-consuming. To address this bottleneck, we propose a Weakly-Supervised Semantic Instance Completion Network (WSSIC-Net), which learns real-world partial point cloud object completion without requiring the ground truth of complete 3D objects. Instead, WSSIC-Net leverages 3D ground-truth bounding boxes, partial objects of a raw scene, and unpaired synthetic 3D point clouds. More specifically, a 3D detector is used to encode partial point clouds into proposal features, which are then fed into two branches. The first branch uses fully supervised box prediction based on proposal features. The second branch, hereinafter called instance completion, leverages the proposal features as partial object features to achieve weakly-supervised instance completion. A Generative Adversarial Network (GAN) completes the partial features of the 2.5D foreground objects of real-world scenes using only unpaired but semantically-consistent complete synthetic point clouds. In our experiments, we demonstrate that the fully-supervised 3D detection and the weakly-supervised instance completion complement one another. The qualitative and quantitative evaluations on the ScanNet v2 dataset demonstrate that the proposed "weakly-supervised" approach consistently achieves comparable performance to the state-of-the-art "fully supervised" methods.

With the growth of VR and AR industry, 3D reconstruction has become a more and more important topic in multimedia. Although 3D Gaussian Splatting achieves state-of-the-art in 3D Reconstruction, a large number of Gaussians are needed to fit a 3D scene due to the Gibbs Phenomenon. The pursuit of compressing 3D Gaussian Splatting and reducing memory overhead has long been a focal point. Embarking on this trajectory, our study delves into this domain, aiming to mitigate these challenges. Inspired by the tangram, a Chinese ancient puzzle, we introduce a novel methodology (Tangram-Splatting) that leverages shape priors to optimize 3D scene fitting. Central to our approach is a pioneering technique that diversifies Gaussian function types while preserving algorithmic efficiency. Through exhaustive experimentation, we demonstrate that our method achieves a remarkable average reduction of 62.4% in memory consumption used to store optimized parameters and decreases the training time by at least 10 minutes, with only marginal sacrifices in PSNR performance, typically under 0.3 dB, and our algorithm is even better on some datasets. This reduction in memory burden is of paramount significance for real-world applications, mitigating the substantial memory footprint and transmission burden traditionally associated with such algorithms. Our algorithm underscores the profound potential of Tangram-Splatting in advancing multimedia applications.

Novel view synthesis aims at rendering any posed images from sparse observations of the scene. Recently, neural radiance fields (NeRF) have demonstrated their effectiveness in synthesizing novel views of a bounded scene. However, most existing methods cannot be directly extended to 360° unbounded scenes where the camera orientations and scene depths are unconstrained with large variations. In this paper, we present a spherical radiance field (SRF) for efficient novel view synthesis in 360° unbounded scenes. Specifically, we represent a 3D scene as multiple concentric spheres with different radii. In particular, each sphere encodes its corresponding layered scene into implicit representations and is parameterized with an equirectangular projection image. A shallow multi-layer perceptron (MLP) is then used to infer the density and color from these sphere representations for volume rendering. Moreover, an occupancy grid is introduced to cache the density field and guide the ray sampling, which accelerates the training and rendering procedures by reducing the number of samples along the ray. Experiments show that our method can well fit 360° unbounded scenes and produces state-of-the-art results on three benchmark datasets with less than 30 minutes of training time on a 3090 GPU, surpassing Mip-NeRF 360 with a 400× speedup. In addition, our method achieves competitive performance in terms of both accuracy and efficiency on a bounded dataset. Project page: https://minglin-chen.github.io/SphericalRF.

Semantic instance reconstruction attracts increasing attention in several areas such as mobile mapping, scene reconstruction, and robot navigation. Although much progresses have been made in recent years, the reconstruction performance is highly sensitive to occlusions and noises. To address these issues, we incorporate point cloud completion into a novel semantic instance reconstruction network PMNet, which consists of a 3-D object detection module, a point cloud completion module, and a mesh generation module. Based on the candidate instance proposals and their proposal features obtained in the object detection module, a point encoder layer is proposed to learn the local geometric features from the point cloud belonging to the detected instances, and a feature transformation layer is utilized to align the proposal features with the local geometric features. These two types of features are then fused and fed into the point cloud decoder to predict the complete point cloud of each instance. The mesh is finally reconstructed for each instance by the mesh generation module. Quantitative and qualitative experiments conducted on the ScanNetv2 dataset demonstrate that the proposed PMNet achieves the best reconstruction performance on real-world point clouds.

Pedestrian detection is an important problem for autonomous driving. It is still chanllenging to detect and segment pedestrians from point clouds. In this paper, we propose a method named SurfaceNet to detect and segment pedestrians from point clouds. Specifically, we propose a novel representation, named surface map, to represent a point cloud as a 2D pseudo-image. For pedestrian detection, the proposed method comprises of four modules: 1) a grid feature encoder that can processes arbitrary number of points within each grid; 2) a surface feature convolutional module that employs a set of 2D convolutional layers to extract high level features; 3) a view transform module that transforms features from front view to bird's eye view; and 4) an anchor-free 3D object detection head that produces rotated 3D bounding box predictions. For semantic segmentation, the 2D pseudo-image is used for semantic segmentation and the segmentation results are re-projected to the original point cloud to achieve point cloud segmentation. Experimental results on the KITTI dataset show that our method achieves promising performance on pedestrian detection and segmentation in point clouds.