Bird’s-Eye-View (BEV) segmentation is a essential technology for safe and efficient navigation. This is even more necessary in indoor driving, where there are dynamic and unstructured objects such as people and robot. However, since there is no way to generate training data, most of the researches have been conducted mainly in outdoor environments. In this paper, we propose an innovative approach to address this challenge. We automatically generate BEV training data for indoor environments based on the physics engine of a simulator. This eliminates the needs for tons of real data and virtual environments. We also propose a lightweight network architecture capable of BEV panoptic segmentation in real-time. Based on this network and simple image processing, our framework shows fast but also robust performance in real-world environments with no gap between simulation and reality. The network can inference at a speed of about 61 FPS on AGX Orin in FP16 mode. Furthermore, it outperforms existing algorithms in the semantic segmentation task, achieving 14% higher mIoU.

Omni-directional images are becoming more prevalent for understanding the scene of all directions around a camera, as they provide a much wider field-of-view (FoV) compared to conventional images. In this work, we present a novel approach to represent omni-directional images and suggest how to apply CNNs on the proposed image representation. The proposed image representation method utilizes a spherical polyhedron to reduce distortion introduced inevitably when sampling pixels on a non-Euclidean spherical surface around the camera center. To apply convolution operation on our representation of images, we stack the neighboring pixels on top of each pixel and multiply with trainable parameters. This approach enables us to apply the same CNN architectures used in conventional euclidean 2D images on our proposed method in a straightforward manner. Compared to the previous work, we additionally compare different designs of kernels that can be applied to our proposed method. We also show that our method outperforms in monocular depth estimation task compared to other state-of-the-art representation methods of omni-directional images. In addition, we propose a novel method to fit bounding ellipses of arbitrary orientation using object detection networks and apply it to an omni-directional real-world human detection dataset.

Omni-directional cameras have many advantages over conventional cameras in that they have a much wider field-of-view (FOV). Accordingly, several approaches have been proposed recently to apply convolutional neural networks (CNNs) to omni-directional images for various visual tasks. However, most of them use image representations defined in the Euclidean space after transforming the omni-directional views originally formed in the non-Euclidean space. This transformation leads to shape distortion due to nonuniform spatial resolving power and the loss of continuity. These effects make existing convolution kernels experience difficulties in extracting meaningful information. This paper presents a novel method to resolve such problems of applying CNNs to omni-directional images. The proposed method utilizes a spherical polyhedron to represent omni-directional views. This method minimizes the variance of the spatial resolving power on the sphere surface, and includes new convolution and pooling methods for the proposed representation. The proposed method can also be adopted by any existing CNN-based methods. The feasibility of the proposed method is demonstrated through classification, detection, and semantic segmentation tasks with synthetic and real datasets.