Semi-supervised video object segmentation (VOS) is to predict the segment of a target object in a video when a ground truth segmentation mask for the target is given in the first frame. Recently, space-time memory networks (STM) have received significant attention as a promising approach for semi-supervised VOS. However, an important point has been overlooked in applying STM to VOS: The solution (=STM) is non-local, but the problem (=VOS) is predominantly local. To solve this mismatch between STM and VOS, we propose new VOS networks called kernelized memory network (KMN) and KMN with multiple kernels (KMN$^{M}$). Our networks conduct not onlyQuery-to-Memorymatching but alsoMemory-to-Querymatching. InMemory-to-Querymatching, a kernel is employed to reduce the degree of non-localness of the STM. In addition, we present a Hide-and-Seek strategy in pre-training to handle occlusions effectively. The proposed networks surpass the state-of-the-art results on standard benchmarks by a significant margin (+4% in$\mathcal {J_{M}}$on DAVIS 2017 test-dev set). The runtimes of our proposed KMN and KMN$^{M}$on DAVIS 2016 validation set are 0.12 and 0.13 seconds per frame, respectively, and the two networks have similar computation times to STM.

With the development of deep learning, semantic segmentation has received considerable attention within the robotics community. For semantic segmentation to be applied to mobile robots or autonomous vehicles, real-time processing is essential. In this article, a new real-time semantic segmentation network, called the adjacent feature propagation network (AFPNet), is proposed to achieve high performance and fast inference. AFPNet executes in real time on a commercial embedded GPU. The network includes two new modules. The local memory module (LMM) is the first; it improves the upsampling accuracy by propagating the high-level features to the adjacent grids. The cascaded pyramid pooling module (CPPM) is the second; it reduces computational time by changing the structure of the pyramid pooling module. Using these two modules, the proposed AFPNet achieved 76.4% mean intersection-over-union on the Cityscapes test dataset, outperforming other real-time semantic segmentation networks. Furthermore, AFPNet was successfully deployed on an embedded board Jetson AGX Xavier and applied to the real-world navigation of a mobile robot, proving that AFPNet can be effectively used in a variety of real-time applications.

In this paper, we tackle the unsupervised domain adaptation (UDA) for semantic segmentation, which aims to segment the unlabeled real data using labeled synthetic data. The main problem of UDA for semantic segmentation relies on reducing the domain gap between the real image and synthetic image. To solve this problem, we focused on separating information in an image into content and style. Here, only the content has cues for semantic segmentation, and the style makes the domain gap. Thus, precise separation of content and style in an image leads to effect as supervision of real data even when learning with synthetic data. To make the best of this effect, we propose a zero-style loss. Even though we perfectly extract content for semantic segmentation in the real domain, another main challenge, the class imbalance problem, still exists in UDA for semantic segmentation. We address this problem by transferring the contents of tail classes from synthetic to real domain. Experimental results show that the proposed method achieves the state-of-the-art performance in semantic segmentation on the major two UDA settings.

When pedestrian detection (PD) is implemented on a central processing unit (CPU), performing real-time processing using a classical sliding window is difficult. Therefore, an efficient proposal generation method is required. A new generation method, named additive kernel binarized normed gradient (AKBING), is proposed herein, and this method is applied to the PD for real-time implementation on a CPU. The AKBING is based on an additive kernel support vector machine (AKSVM) and is implemented using the binarized normed gradient. The proposed PD can operate in real time because all AKSVM computations are approximated via simple atomic operations. In the suggested kernelized proposal method, the popular features and a classifier are combined, and the method is tested on a Caltech Pedestrian dataset and KITTI dataset. The experimental results show that the detection system with the proposed method improved the speed with minor degradation in detection accuracy.

When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365

For intelligent vehicle applications, detecting pedestrian technique must be robust and perform in real time. In pedestrian detection, support vector machine (SVM) is one of the popular classifiers because of its robust performance. In this paper, we propose the new method to implement cascade SVM that enables fast rejection of negative samples. The proposed method is tested with INRIA person dataset and show better rejection performance of negative samples than conventional method.

Object detection is one of the most interesting branches in computer vision. Accurate detection systems can be utilized to various areas. There are two steps in detection, feature extraction and classification. In this paper, new feature extraction method is proposed. Histogram Oriented Gradient (HOG) is famous, fast and accurate feature, but it is not rotation invariant. This paper proposes a new shape of HOG for fast detection of rotated objects. The proposed method is faster than conventional method in rotational object detection.