Semantic Scene Completion (SSC) aims to simultaneously predict the volumetric occupancy and semantic category of a 3D scene. In this paper, we propose a real-time semantic scene completion method with a feature aggregation strategy and conditioned prediction module. Feature aggregation fuses feature with different receptive fields and gathers context to improve scene completion performance. And the conditioned prediction module adopts a two-step prediction scheme that takes volumetric occupancy as a condition to enhance semantic completion prediction. We conduct experiments on three recognized benchmarks NYU, NYUCAD, and SUNCG. Our method achieves competitive performance at a speed of 110 FPS on one GTX 1080 Ti GPU.

Convolutional neural networks (CNN) have achieved great success in RGB semantic segmentation. RGB-D images provide additional depth information, which can improve segmentation performance. To take full advantages of the 3D geometry relations provided by RGB-D images, in this paper, we propose 2.5D convolution, which mimics one 3D convolution kernel by several masked 2D convolution kernels. Our 2.5D convolution can effectively process spatial relations between pixels in a manner similar to 3D convolution while still sampling pixels on 2D plane, and thus saves computational cost. And it can be seamlessly incorporated into pretrained CNNs. Experiments on two challenging RGB-D semantic segmentation benchmarks NYUDv2 and SUN-RGBD validate the effectiveness of our approach.

In RGB-D semantic segmentation tasks, it has been shown that HHA embeddings effectively encode rich depth features and using HHA together with RGB images can improve segmentation performance. In this paper, we propose a novel method to effectively integrate RGB and HHA features. By replacing identity mappings in ResNet-based two-stream network with idempotent mappings, we can couple the originally separated two branches to mix features from two modalities, while still keep the good information flow nature of ResNet. Moreover, our method does not bring any additional network blocks or parameters, and only needs very small modification on basic two-stream networks. We conduct experiments on two challenging RGB-D semantic segmentation datasets NYUDv2 and SUN-RGBD. The experiment results show that our method can significantly improve segmentation performance and our method achieves the state-of-the-art on these two datasets.

Deep convolutional neural networks (CNNs) have gained their reputation for the success in various tasks in computer vision, including salient objects detection. However, it remains a challenge that the CNNs have repeated downsample operators and always create low-resolution predictions, which tend to loss details and finer structure of images. To detect and segment the salient objects well, it is also necessary to merge high-level semantic information and low-level fine details simultaneously. Thus, we propose a novel network structure with stage-wise refinement sub-structures. In addition, we exploit the essence of salient objects detection by encoding the global image context in a specifically designed module, which is applied to every stage of the refinement structure. So the coarse saliency map generated from the base CNN can be refined with low-level feature and global context information step-by-step. Experimental results have demonstrated that the proposed method outperforms the state-of-the-art approaches on four benchmark datasets.