Recently single-view 3D generation via Gaussian splatting has emerged and developed quickly. They learn 3D Gaussians from 2D RGB images generated from pre-trained multi-view diffusion (MVD) models, and have shown a promising avenue for 3D generation through a single image. Despite the current progress, these methods still suffer from the inconsistency jointly caused by the geometric ambiguity in the 2D images, and the lack of structure of 3D Gaussians, leading to distorted and blurry 3D object generation. In this paper, we propose to fix these issues by GS-RGBN, a new RGBN-volume Gaussian Reconstruction Model designed to generate high-fidelity 3D objects from single-view images. Our key insight is a structured 3D representation can simultaneously mitigate the afore-mentioned two issues. To this end, we propose a novel hybrid Voxel-Gaussian representation, where a 3D voxel representation contains explicit 3D geometric information, eliminating the geometric ambiguity from 2D images. It also structures Gaussians during learning so that the optimization tends to find better local optima. Our 3D voxel representation is obtained by a fusion module that aligns RGB features and surface normal features, both of which can be estimated from 2D images. Extensive experiments demonstrate the superiority of our methods over prior works in terms of high-quality reconstruction results, robust generalization, and good efficiency.

We propose a novel model‐free unsupervised learning paradigm to tackle the unfavorable prevailing problem of real‐world image deraining, dubbed MUL‐Derain. Beyond existing unsupervised deraining efforts, MUL‐Derain leverages a model‐free Multiscale Attentive Filtering (MSAF) to handle multiscale rain streaks. Therefore, formulation of any rain imaging is not necessary, and it requires neither iterative optimization nor progressive refinement operations. Meanwhile, MUL‐Derain can efficiently compute spatial coherence and global interactions by modeling long‐range dependencies, allowing MSAF to learn useful knowledge from a larger or even global rain region. Furthermore, we formulate a novel multiloss function to constrain MUL‐Derain to preserve both color and structure information from the rainy images. Extensive experiments on both synthetic and real‐world datasets demonstrate that our MUL‐Derain obtains state‐of‐the‐art performance over un/semisupervised methods and exhibits competitive advantages over the fully‐supervised ones.

Object detection remains challenging in hazy weather conditions due to the poor visibility of captured images. There are currently two types of detectors capable of adapting to varying weather conditions: (i) low‐level adaptation methods that combine one detector with an additional dehazing network and (ii) high‐level adaptation methods that explore various kinds of domain adaptation knowledge. However, neither of these approaches can achieve desirable performance due to their inherent limitations. We raise an intriguing question—if combining both low‐level adaptation and high‐level adaptation, can improve the generalization ability of a detector in hazy weather conditions? To answer it, we propose a Joint High‐Low Adaptation Object Detection paradigm (HLA‐HOD) in hazy weather conditions. By combining both low‐level adaptation and high‐level adaptation, HLA‐HOD achieves superior performance on hazy images without requiring ground‐truth bounding boxes or clean images. Extensive experiments demonstrate that our method outperforms state‐of‐the‐art low‐level and high‐level adaptation methods by a large margin both quantitatively and qualitatively.

LiDAR and camera, as two different sensors, supply geometric (point clouds) and semantic (RGB images) information of 3-D scenes. However, it is still challenging for existing methods to fuse data from the two cross sensors, making them complementary for quality 3-D object detection (3OD). We propose ImLiDAR, a new 3OD paradigm to narrow the cross-sensor discrepancies by progressively fusing the multiscale features of camera Images and LiDAR point clouds. ImLiDAR enables to provide the detection head with cross-sensor yet robustly fused features. To achieve this, two core designs exist in ImLiDAR. First, we propose a cross-sensor dynamic message propagation (CDMP) module to combine the best of the multiscale image and point features. Second, we raise a direct set prediction problem that allows designing an effective set-based detector (SD) to tackle the inconsistency of the classification and localization confidences, and the sensitivity of hand-tuned hyperparameters. Besides, the novel SD can be detachable and easily integrated into various detection networks. Comparisons on the KITTI, nuScenes, and SUN-RGBD datasets all show clear visual and numerical improvements of our ImLiDAR over 45 state-of-the-art 3OD methods.

Rain severely degrades the visibility of scene objects, especially when images are captured through the glass under rainy weather. We observe three intriguing phenomena: 1) rain is a mixture of raindrops, rain streaks and rainy haze; 2) the depth from the camera determines the degree of object visibility, where objects nearby and far away are visually blocked by rain streaks and rainy haze, respectively; and 3) raindrops on the glass randomly affect the object visibility of the whole image space. However, existing solutions and benchmark datasets lack full consideration of the mixture of rain (MOR). In this paper, we originally consider that the overall object visibility is determined by MOR, and enrich the RainCityscapes by considering real-world raindrops to construct the MOR dataset, named RainCityscapes++. To solve the practical rain removal problem arisen from MOR, we formulate a new rain imaging model and propose a multi-branch attention generative adversarial network (MBA-RainGAN). Extensive experiments show clear improvements of our approach over SOTAs on RainCityscapes++.

Abstract Real‐world rain is a mixture of rain streaks and rainy haze. However, current efforts formulate image rain streaks removal and rainy haze removal as separated models, worsening the loss of image details. This paper attempts to solve the mixture of rain removal problem in a single model by estimating the scene depths of images. To this end, we propose a novel SEMI‐ supervised M ixture O f rain RE moval G enerative A dversarial N etwork (Semi‐MoreGAN). Unlike most of existing methods, Semi‐MoreGAN is a joint learning paradigm of mixture of rain removal and depth estimation; and it effectively integrates the image features with the depth information for better rain removal. Furthermore, it leverages unpaired real‐world rainy and clean images to bridge the gap between synthetic and real‐world rain. Extensive experiments show clear improvements of our approach over twenty representative state‐of‐the‐arts on both synthetic and real‐world rainy images. Source code is available at https://github.com/syy-whu/Semi-MoreGAN .