Active Label Correction (ALC) has emerged as a promising solution to the high cost and error-prone nature of manual pixel-wise annotation in semantic segmentation, by actively identifying and correcting mislabeled data. Although recent work has improved correction efficiency by generating pseudo-labels using foundation models, substantial inefficiencies still remain. In this paper, we introduce A²LC, an Active and Automated Label Correction framework for semantic segmentation, where manual and automatic correction stages operate in a cascaded manner. Specifically, the automatic correction stage leverages human feedback to extend label corrections beyond the queried samples, thereby maximizing cost efficiency. In addition, we introduce an adaptively balanced acquisition function that emphasizes underrepresented tail classes, working in strong synergy with the automatic correction stage. Extensive experiments on Cityscapes and PASCAL VOC 2012 demonstrate that A²LC significantly outperforms previous state-of-the-art methods. Notably, A²LC exhibits high efficiency by outperforming previous methods with only 20% of their budget, and shows strong effectiveness by achieving a 27.23% performance gain under the same budget on Cityscapes.

We present a novel, training-free approach to scene change detection. Our method leverages tracking models, which inherently perform change detection between consecutive frames of video by identifying common objects and detecting new or missing objects. Specifically, our method takes advantage of the change detection effect of the tracking model by inputting reference and query images instead of consecutive frames. Furthermore, we focus on the content gap and style gap between two input images in change detection, and address both issues by proposing adaptive content threshold and style bridging layers, respectively. Finally, we extend our approach to video, leveraging rich temporal information to enhance the performance of scene change detection. We compare our approach and baseline through various experiments. While existing train-based baseline tend to specialize only in the trained domain, our method shows consistent performance across various domains, proving the competitiveness of our approach.

Neural Radiance Fields (NeRF) presented a novel way to represent scenes, allowing for high-quality 3D reconstruction from 2D images. Following its remarkable achievements, global localization within NeRF maps is an essential task for enabling a wide range of applications. Recently, Loc-NeRF demonstrated a localization approach that combines traditional Monte Carlo Localization with NeRF, showing promising results for using NeRF as an environment map. However, despite its advancements, Loc-NeRF encounters the challenge of a time-intensive ray rendering process, which can be a significant limitation in practical applications. To address this issue, we introduce Fast Loc-NeRF, which enhances efficiency and accuracy in NeRF map-based global localization. We propose a particle rejection weighting strategy that estimates the uncertainty of particles by leveraging NeRF's inherent characteristics and incorporates them into the particle weighting process to reject abnormal particles. Additionally, Fast Loc-NeRF employs a coarse-to-fine approach, matching rendered pixels and observed images across multiple resolutions from low to high. As a result, it speeds up the costly particle update process while enhancing precise localization results. Our Fast Loc-NeRF establishes new state-of-the-art localization performance on several bench-marks, demonstrating both its accuracy and efficiency. The code is available at this url.

While most people associate LiDAR primarily with its ability to measure distances and provide geometric information about the environment (via point clouds), LiDAR also captures additional data, including reflectivity or intensity values. Unfortunately, when LiDAR is applied to Place Recognition (PR) in mobile robotics, most previous works on LiDAR-based PR rely only on geometric measurements, neglecting the additional reflectivity information that LiDAR provides. In this paper, we propose a novel descriptor for 3D PR, named RE-TRIP (REflectivity-instance augmented TRIangle descriPtor). This new descriptor leverages both geometric measurements and reflectivity to enhance robustness in challenging scenarios such as geometric degeneracy, high geometric similarity, and the presence of dynamic objects. To implement RE-TRIP in real-world applications, we further propose (1) keypoint extraction method, (2) key instance segmentation method, (3) RE-TRIP matching method, and (4) reflectivity combined loop verification method. Finally, we conduct a series of experiments to demonstrate the effectiveness of RE-TRIP. Applied to public datasets (i.e., HELIPR, FusionPortable) containing diverse scenarios-including long corridors, bridges, large-scale urban areas, and highly dynamic environments-our experimental results show that the proposed method outperforms existing state-of-the-art methods in terms of Scan Context, Intensity Scan Context and STD. Our code is available at: https://github.com/pycS714IRE-TRIP.

In this paper, we propose a novel image retrieval network named Correlation Verification Network (CVNet) to replace the conventional geometric re-ranking with a 4D convolutional neural network that learns diverse geometric matching possibilities. To enable efficient cross-scale matching, we construct feature pyramids and establish cross-scale feature correlations in a single inference, thereby replacing the costly multi-scale inference. Additionally, we employ curriculum learning with the Hide-and-Seek strategy to handle challenging samples. Our proposed CVNet demonstrates state-of-the-art performance on several image retrieval benchmarks by a large margin. From an implementation perspective, however, CVNet has one drawback: it requires high memory usage because it needs to store dense features of all database images. This high memory requirement can be a significant limitation in practical applications. To address this issue, we introduce an extension of CVNet called Dense-to-Sparse CVNet (CVNet), which can significantly reduce memory usage by sparsifying the features of the database images. The sparsification module in CVNet learns to select the relevant parts of image features end-to-end using a Gumbel estimator. Since the sparsification is performed offline, CVNet does not increase online extraction and matching times. CVNet dramatically reduces the memory footprint while preserving performance levels nearly identical to CVNet.

Three-dimensional light detection and ranging (3D LiDAR) sensors are widely used in autonomous vehicles. Among various perception problems using a 3D LiDAR, the identification of drivable region (DR) is one of the most important problems. In this paper, the DR identification using a 3D LiDAR is reformulated as a completion problem. In other words, the problem is to generate the virtual points so that the virtual points cover the DR as densely and uniformly as possible, thereby representing the DR well enough. The problem considered herein is named the DR completion problem. To solve the DRC, a new network called the Drivable Region Completion Network (DRCN) is proposed. The proposed DRCN consists of an encoder part and a decoder part. The encoder part consists of two encoders. The first one is the Multi-Layer Perceptron (MLP) encoder, and it captures the global feature of the given LiDAR points. The second one is named the Point Pyramid Network (PPN) encoder, and it extracts local features between the points in the point cloud. The decoder part takes the coarse-to-fine structure, and it consists of the coarse and refinement decoders. The coarse decoder predicts seed points for the DR. The refinement decoder refines the seed points to predict the final dense DR points. Finally, the DRCN is applied to the SemanticKITTI dataset for evaluation. The proposed DRCN is validated by showing that our DRCN outperforms other DR detection methods in terms of accuracy.

We propose a novel solution for unpaired image-to-image (I2I) translation. To translate complex images with a wide range of objects to a different domain, recent approaches often use the object annotations to perform per-class source-to-target style mapping. However, there remains a point for us to exploit in the I2I. An object in each class consists of multiple components, and all the sub-object components have different characteristics. For example, a car in CAR class consists of a car body, tires, windows and head and tail lamps, etc., and they should be handled separately for realistic I2I translation. The simplest solution to the problem will be to use more detailed annotations with sub-object component annotations than the simple object annotations, but it is not possible. The key idea of this paper is to bypass the sub-object component annotations by leveraging the original style of the input image because the original style will include the information about the characteristics of the sub-object components. Specifically, for each pixel, we use not only the per-class style gap between the source and target domains but also the pixel’s original style to determine the target style of a pixel. To this end, we present Style Harmonization for unpaired I2I translation (SHUNIT). Our SHUNIT generates a new style by harmonizing the target domain style retrieved from a class memory and an original source image style. Instead of direct source-to-target style mapping, we aim for source and target styles harmonization. We validate our method with extensive experiments and achieve state-of-the-art performance on the latest benchmark sets. The source code is available online: https://github.com/bluejangbaljang/SHUNIT.

Despite advances in global image representation, existing image retrieval approaches rarely consider geometric structure during the global retrieval stage. In this work, we revisit the conventional self-similarity descriptor from a convolutional perspective, to encode both the visual and structural cues of the image to global image representation. Our proposed network, named Structural Embedding Network (SENet), captures the internal structure of the images and gradually compresses them into dense self-similarity descriptors while learning diverse structures from various images. These self-similarity descriptors and original image features are fused and then pooled into global embedding, so that global embedding can represent both geometric and visual cues of the image. Along with this novel structural embedding, our proposed network sets new state-of-the-art performances on several image retrieval benchmarks, convincing its robustness to look-alike distractors. The code and models are available: https://github.com/sungonce/SENet.

Ego-localization is one of the most critical functions in autonomous vehicles. This paper presents a novel WiFi-based localization system for autonomous driving designed to augment onboard localization systems during critical failures or complement GNSS-denied scenarios such as parking lots. The system leverages the existing WiFi network infrastructure to provide global localization using a WiFi interface and a publicly available WiFi RSS and AP database created through survey efforts with conventional mobile devices. An LSTM-based architecture is trained to estimate the device’s position from the history of WiFi RSS, leveraging temporal correlations in the sequences. The results suggest that this system is a viable alternative even when no strong requirements are set for the quality of the GNSS measurements in the surveying phase.

The network trained for domain adaptation is prone to bias toward the easy-to-transfer classes. Since the ground truth label on the target domain is unavailable during training, the bias problem leads to skewed predictions, forgetting to predict hard-to-transfer classes. To address this problem, we propose Cross-domain Moving Object Mixing (CMOM) that cuts several objects, including hard-to-transfer classes, in the source domain video clip and pastes them into the target domain video clip. Unlike image-level domain adaptation, the temporal context should be maintained to mix moving objects in two different videos. Therefore, we de-sign CMOM to mix with consecutive video frames, so that unrealistic movements are not occurring. We additionally propose Feature Alignment with Temporal Context (FATC) to enhance target domain feature discriminability. FATC exploits the robust source domain features, which are trained with ground truth labels, to learn discriminative target do-main features in an unsupervised manner by filtering unreliable predictions with temporal consensus. We demonstrate the effectiveness of the proposed approaches through extensive experiments. In particular, our model reaches mIoU of 53.81% on VIPER → Cityscapes-Seq benchmark and mIoU of 56.31% on SYNTHIA-Seq → Cityscapes-Seq benchmark, surpassing the state-of-the-art methods by large margins.

Semi-supervised video object segmentation (VOS) is to predict the segment of a target object in a video when a ground truth segmentation mask for the target is given in the first frame. Recently, space-time memory networks (STM) have received significant attention as a promising approach for semi-supervised VOS. However, an important point has been overlooked in applying STM to VOS: The solution (=STM) is non-local, but the problem (=VOS) is predominantly local. To solve this mismatch between STM and VOS, we propose new VOS networks called kernelized memory network (KMN) and KMN with multiple kernels (KMN$^{M}$). Our networks conduct not onlyQuery-to-Memorymatching but alsoMemory-to-Querymatching. InMemory-to-Querymatching, a kernel is employed to reduce the degree of non-localness of the STM. In addition, we present a Hide-and-Seek strategy in pre-training to handle occlusions effectively. The proposed networks surpass the state-of-the-art results on standard benchmarks by a significant margin (+4% in$\mathcal {J_{M}}$on DAVIS 2017 test-dev set). The runtimes of our proposed KMN and KMN$^{M}$on DAVIS 2016 validation set are 0.12 and 0.13 seconds per frame, respectively, and the two networks have similar computation times to STM.

Unsupervised video object segmentation (UVOS) is a per-pixel binary labeling problem which aims at separating the foreground object from the background in the video without using the ground truth (GT) mask of the foreground object. Most of the previous UVOS models use the first frame or the entire video as a reference frame to specify the mask of the foreground object. Our question is why the first frame should be selected as a reference frame or why the entire video should be used to specify the mask. We believe that we can select a better reference frame to achieve the better UVOS performance than using only the first frame or the entire video as a reference frame. In our paper, we propose Easy Frame Selector (EFS). The EFS enables us to select an "easy" reference frame that makes the subsequent VOS become easy, thereby improving the VOS performance. Furthermore, we propose a new framework named as Iterative Mask Prediction (IMP). In the framework, we repeat applying EFS to the given video and selecting an "easier" reference frame from the video than the previous iteration, increasing the VOS performance incrementally. The IMP consists of EFS, Bi-directional Mask Prediction (BMP), and Temporal Information Updating (TIU). From the proposed framework, we achieve state-of-the-art performance in three UVOS benchmark sets: DAVIS16, FBMS, and SegTrack-V2.

In this paper, a new deep learning network named as graph-based point tracker (GPT) is proposed for 3D object tracking in point clouds. GPT is not based on Siamese network applied to template and search area, but it is based on the transfer of target clue from the template to the search area. GPT is end-to-end trainable. GPT has two new modules: graph feature augmentation (GFA) and improved target clue (ITC) module. The key idea of GFA is to exploit one-to-many relationship between template and search area points using a bipartite graph. In GFA, edge features of the bipartite graph are generated by transferring the target clues of template points to search area points through edge convolution. It captures the relationship between template and search area points effectively from the perspective of geometry and shape of two point clouds. The second module is ITC. The key idea of ITC is to embed the information of the center of the target into the edges of the bipartite graph via Hough voting, strengthening the discriminative power of GFA. Both modules significantly contribute to the improvement of GPT by transferring geometric and shape information including target center from target template to search area effectively. Experiments on the KITTI tracking dataset show that GPT achieves state-of-the-art performance and can run in real-time.

We present a new domain generalized semantic segmentation network named WildNet, which learns domain-generalized features by leveraging a variety of contents and styles from the wild. In domain generalization, the low generalization ability for unseen target domains is clearly due to overfitting to the source domain. To address this problem, previous works have focused on generalizing the domain by removing or diversifying the styles of the source domain. These alleviated overfitting to the source-style but overlooked overfitting to the source-content. In this paper, we propose to diversify both the content and style of the source domain with the help of the wild. Our main idea is for networks to naturally learn domain-generalized semantic information from the wild. To this end, we diversify styles by augmenting source features to resemble wild styles and enable networks to adapt to a variety of styles. Further-more, we encourage networks to learn class-discriminant features by providing semantic variations borrowed from the wild to source contents in the feature space. Finally, we regularize networks to capture consistent semantic information even when both the content and style of the source domain are extended to the wild. Extensive experiments on five different datasets validate the effectiveness of our WildNet, and we significantly outperform state-of-the-art methods. The source code and model are available online: https://github.com/suhyeonlee/WildNet.

Geometric verification is considered a de facto solution for the re-ranking task in image retrieval. In this study, we propose a novel image retrieval re-ranking network named Correlation Verification Networks (CVNet). Our proposed network, comprising deeply stacked 4D convolutional layers, gradually compresses dense feature correlation into image similarity while learning diverse geometric matching patterns from various image pairs. To enable cross-scale matching, it builds feature pyramids and constructs cross-scale feature correlations within a single inference, replacing costly multi-scale inferences. In addition, we use curriculum learning with the hard negative mining and Hide-and-Seek strategy to handle hard samples without losing generality. Our proposed re-ranking network shows state-of-the-art performance on several retrieval benchmarks with a significant margin (+12.6% in mAP on ROxford-Hard+1M set) over state-of-the-art methods. The source code and models are available online: ht tps: / /gi thub. com/ sungonce/CVNet.

Despite the great success of attention mechanisms on object recognition, scene recognition remains a challenging problem. The reason is that discriminative regions are not evident in a scene image. For example, a tree in an image can be a cue to recognize a scene, but the tree cannot be the only cue for recognizing the scene. That means several scene categories (e.g. mountain, marsh, and river) can contain a tree. Thus sometimes, overall regions, rather than specific regions, need to be considered for scene recognition. To solve the problem, we propose Spatial-Channel Transformer (SC-Transformer). The SC-Transformer is a simple yet effective module that uses a new attention mechanism by incorporating the importance between the spatial and the channel domain for a given scene image. If the given scene image should be considered only within some specific regions, SC-Transformer turns off the channel attention, and vice versa. Furthermore, the attention mechanism used in our proposed method is advanced from previous approaches. Previous spatial and channel attention mechanisms were designed in a sequential or parallel manner. These mechanisms eventually combine spatial and channel attention together, so spatial and channel attention may often interfere with each other. In contrast to the previous works, we present a new mechanism that simultaneously considers spatial and channel attentions. We validate our approach on a large-scale scene recognition dataset and outperform the previous state-of-the-art spatial-channel attention mechanism. Experimental results demonstrate the efficacy of our attention mechanism for scene recognition.

Indoor place category recognition for a cleaning robot is a problem in which a cleaning robot predicts the category of the indoor place using images captured by it. This is similar to scene recognition in computer vision as well as semantic mapping in robotics. Compared with scene recognition, the indoor place category recognition considered in this article differs as follows: 1) the indoor places include typical home objects; 2) a sequence of images instead of an isolated image is provided because the images are captured successively by a cleaning robot; and 3) the camera of the cleaning robot has a different view compared with those of cameras typically used by human beings. Compared with semantic mapping, indoor place category recognition can be considered as a component in semantic SLAM. In this article, a new method based on the combination of a probabilistic approach and deep learning is proposed to address indoor place category recognition for a cleaning robot. Concerning the probabilistic approach, a new place-object fusion method is proposed based on Bayesian inference. For deep learning, the proposed place-object fusion method is trained using a convolutional neural network in an end-to-end framework. Furthermore, a new recurrent neural network, called the Bayesian filtering network (BFN), is proposed to conduct time-domain fusion. Finally, the proposed method is applied to a benchmark dataset and a new dataset developed in this article, and its validity is demonstrated experimentally.

With the development of deep learning, semantic segmentation has received considerable attention within the robotics community. For semantic segmentation to be applied to mobile robots or autonomous vehicles, real-time processing is essential. In this article, a new real-time semantic segmentation network, called the adjacent feature propagation network (AFPNet), is proposed to achieve high performance and fast inference. AFPNet executes in real time on a commercial embedded GPU. The network includes two new modules. The local memory module (LMM) is the first; it improves the upsampling accuracy by propagating the high-level features to the adjacent grids. The cascaded pyramid pooling module (CPPM) is the second; it reduces computational time by changing the structure of the pyramid pooling module. Using these two modules, the proposed AFPNet achieved 76.4% mean intersection-over-union on the Cityscapes test dataset, outperforming other real-time semantic segmentation networks. Furthermore, AFPNet was successfully deployed on an embedded board Jetson AGX Xavier and applied to the real-world navigation of a mobile robot, proving that AFPNet can be effectively used in a variety of real-time applications.

In this paper, we tackle the unsupervised domain adaptation (UDA) for semantic segmentation, which aims to segment the unlabeled real data using labeled synthetic data. The main problem of UDA for semantic segmentation relies on reducing the domain gap between the real image and synthetic image. To solve this problem, we focused on separating information in an image into content and style. Here, only the content has cues for semantic segmentation, and the style makes the domain gap. Thus, precise separation of content and style in an image leads to effect as supervision of real data even when learning with synthetic data. To make the best of this effect, we propose a zero-style loss. Even though we perfectly extract content for semantic segmentation in the real domain, another main challenge, the class imbalance problem, still exists in UDA for semantic segmentation. We address this problem by transferring the contents of tail classes from synthetic to real domain. Experimental results show that the proposed method achieves the state-of-the-art performance in semantic segmentation on the major two UDA settings.

We present Hierarchical Memory Matching Network (HMMN) for semi-supervised video object segmentation. Based on a recent memory-based method [33], we propose two advanced memory read modules that enable us to perform memory reading in multiple scales while exploiting temporal smoothness. We first propose a kernel guided memory matching module that replaces the non-local dense memory read, commonly adopted in previous memory-based methods. The module imposes the temporal smoothness constraint in the memory read, leading to accurate memory retrieval. More importantly, we introduce a hierarchical memory matching scheme and propose a top-k guided memory matching module in which memory read on a fine-scale is guided by that on a coarse-scale. With the module, we perform memory read in multiple scales efficiently and leverage both high-level semantic and low-level fine-grained memory features to predict detailed object masks. Our network achieves state-of-the-art performance on the validation sets of DAVIS 2016/2017 (90.8% and 84.7%) and YouTube-VOS 2018/2019 (82.6% and 82.5%), and test-dev set of DAVIS 2017 (78.6%). The source code and model are available online: https://github.com/Hongje/HMMN.

Global localization using a monocular camera is one of the most challenging problems in computer vision and intelligent robotics. In this article, a new deep neural network named Mixture Density (MD)-PoseNet is proposed to address this problem. Unlike existing learning-based global localization methods that return a single guess for the camera pose, MD-PoseNet returns multiple guesses represented in the form of a Gaussian mixture (GM). The key idea of MD-PoseNet is that the network returns the distribution of all probable camera poses instead of the most probable camera pose, and the distribution represents the multiple guesses for the camera pose. The multiple guesses returned by MD-PoseNet are, consequently, exploited in the probabilistic framework of particle filters. Finally, the proposed method is applied to four different environments, and its validity is demonstrated via experiments.

Vehicle bounding box tracking (VBBT) is a new problem that is becoming increasingly important in autonomous driving. It is defined as a problem in which not only the position but also the size of a target vehicle is estimated using a sensor. In this paper, novel VBBT using a low-end three-dimensional (3D) laser scanner is proposed. Compared to previous methods, the proposed VBBT has three distinctions: (1) the center of a rectangular vehicle is defined as its position, and the motion model that uses the center of the vehicle as the state is developed; (2) a new measurement model is proposed that models the measured size of the target vehicle as a sample from a uniform distribution; and (3) a Bayesian filter for the proposed motion and measurement model is developed and it is named as the Pareto Kalman filter (PKF). Finally, the proposed method is applied to six scenarios, and its validity is demonstrated through experimentation.

Road detection and segmentation is a crucial task in computer vision for safe autonomous driving. With this in mind a new net architecture (3D-DEEP) and its end-to-end training methodology for CNN-based semantic segmentation is described along this paper for. The method relies on disparity filtered and LiDAR projected images for three-dimensional information and image feature extraction through fully convolutional networks architectures. The developed models were trained and validated over Cityscapes dataset using just fine annotation examples with 19 different training classes, and over KITTI road dataset. 72.32% mean intersection over union (mIoU) has been obtained for the 19 Cityscapes training classes using the validation images. On the other hand, over KITTI dataset the model has achieved an F1 error value of 97.85% in validation and 96.02% using the test images.

When pedestrian detection (PD) is implemented on a central processing unit (CPU), performing real-time processing using a classical sliding window is difficult. Therefore, an efficient proposal generation method is required. A new generation method, named additive kernel binarized normed gradient (AKBING), is proposed herein, and this method is applied to the PD for real-time implementation on a CPU. The AKBING is based on an additive kernel support vector machine (AKSVM) and is implemented using the binarized normed gradient. The proposed PD can operate in real time because all AKSVM computations are approximated via simple atomic operations. In the suggested kernelized proposal method, the popular features and a classifier are combined, and the method is tested on a Caltech Pedestrian dataset and KITTI dataset. The experimental results show that the detection system with the proposed method improved the speed with minor degradation in detection accuracy.

When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365

In this paper, a novel normal distribution mixture matching based model free object tracking algorithm using 2D LIDAR is proposed. Each target object is modeled as a normal distribution mixture that captures the distribution of the points scanned from the surface of the object. This novel representation enables normal distribution transform (NDT) to accurately estimate the motion of objects, even if the shape of the points differs depending on where it is observed. Our evaluation of the proposed algorithm shows good performance in practical applications. In addition, we provides an alternative way of segmentation and data association using occupancy grid map to avoid a problem that defines a distance metric between the mixture and the point cloud. As a result, the proposed algorithm works in real time in our experiments.

High Definition map (HD Map) is an important part of autonomous driving vehicle. Most conventional method to generate HD map requires expensive system and postprocessing of observed data. In this paper, we propose automatic HD map generating algorithm using just monocular camera without further human labors. The proposed algorithm detects road lane from image and classifies the type of road lane at pixel-level with Fully Convolutional Network (FCN) which outperforms the other semantic segmentation methods. The segmentation results are used to extract lane features, and the features are used for loop-closure detection. Final map is generated with graph-based Simultaneous Localization and Mapping (SLAM) algorithm. The experiment is done with monocular camera mounted on mobile vehicle. In this paper, final map generated by proposed method is compared with aerial view data. The results show that the proposed method can generate reliable map that is comparable to real roads even only the low-cost sensor is used.

The 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, dubbed CoVieW'18, is held in Seoul, Korea on October 22, 2018, in conjuction with ACM Multimedia 2018. The workshop aims to solve the joint and comprehensive understanding problem in untrimmed videos with a particular emphasis on joint action and scene recognition. The workshop encourages researchers to participate in joint action and scene recognition challenge in untrimmed videos and to report their results. The workshop program includes 1 keynote speech, 2 invited speakers, 6 regular and challenge papers. The developments made in the workshop will deliver a step change in a variety of video applications.

These days, laser scanners becomes the primary sensor for advanced driver assistance system (ADAS). The most important theme of ADAS is to distinguish surroundings of egovehicle because notification of situation is the beginning of ADAS such as path planning, mapping and tracking. In this paper, we present approach for object classification by using a laser scanner mounted in vehicle. For object classification, we suggest Recurrent Neural Network (RNN) which is widely used in linguistic study or language model. We rearrange laser scanner data to equivalent theta intervals and apply recurrent neural network model to identify of class about laser scanner point. The proposed method is implemented on a real vehicle, and its performance is tested in a real-world environment. The experiments indicate that the proposed method has good performance in real-life situation.

For reliable driving assistance or automated driving, pedestrian detection must be robust and performed in real time. In pedestrian detection, a linear support vector machine (linSVM) is popularly used as a classifier but exhibits degraded performance due to the multipostures of pedestrians. Kernel SVM (KSVM) could be a better choice for pedestrian detection, but it has a disadvantage in that it requires too much more computation than linSVM. In this paper, the cascade implementation of the additive KSVM (AKSVM) is proposed for the application of pedestrian detection. AKSVM avoids kernel expansion by using lookup tables, and it is implemented in cascade form, thereby speeding up pedestrian detection. The cascade implementation is trained by a genetic algorithm such that the computation time is minimized, whereas the detection accuracy is maximized. In experiments, the proposed method is tested with the INRIA dataset. The experimental results indicate that the proposed method has better detection accuracy and reduced computation time compared with conventional methods.

Negligence of a driver or a sudden stop of a forward vehicle can cause rear-end collision. In this paper, we propose a new situation assessment algorithm to determine collision probability to prevent the rear-end collision. The proposed algorithm consists of two phases: coarse assessment and fine assessment. In the coarse assessment, the algorithm selects a target vehicle with the highest possibility of collision by using fuzzy logic. In fine assessment, it determines collision probability based on a statistical approach considering driving maneuvers; it models the driving maneuvers to enable the driver to operate the vehicle in conditions toward the collision and calculates the collision probability as the ratio between the total driving maneuvers and the driving maneuvers in possible collisions. To reduce the simulation time complexity, we adapt a neural network. Since there exist variance of widths for different vehicles, we also apply neural network ensemble to cope with the variance. Numerical evaluation of the proposed method is provided through simulations and practical tests.

For intelligent vehicle applications, detecting pedestrian technique must be robust and perform in real time. In pedestrian detection, support vector machine (SVM) is one of the popular classifiers because of its robust performance. In this paper, we propose the new method to implement cascade SVM that enables fast rejection of negative samples. The proposed method is tested with INRIA person dataset and show better rejection performance of negative samples than conventional method.

Object detection is one of the most interesting branches in computer vision. Accurate detection systems can be utilized to various areas. There are two steps in detection, feature extraction and classification. In this paper, new feature extraction method is proposed. Histogram Oriented Gradient (HOG) is famous, fast and accurate feature, but it is not rotation invariant. This paper proposes a new shape of HOG for fast detection of rotated objects. The proposed method is faster than conventional method in rotational object detection.

Support vector regression has been considered as one of the most important regression or function approximation methodologies in a variety of fields. In this paper, two new general dimensional multiple output support vector regressions (MSVRs) named SOCPL1 and SOCPL2 are proposed. The proposed methods are formulated in the dual space and their relationship with the previous works is clearly investigated. Further, the proposed MSVRs are extended into the multiple kernel learning and their training is implemented by the off-the-shelf convex optimization tools. The proposed MSVRs are applied to benchmark problems and their performances are compared with those of the previous methods in the experimental section.

In this paper, a new on-road vehicle detection method is presented. First, a new feature named the Position and Intensity-included Histogram of Oriented Gradients (PIHOG or πHOG) is proposed. Unlike the conventional HOG, πHOG compensates the information loss involved in the construction of a histogram with position information, and it improves the discriminative power using intensity information. Second, a new search space reduction (SSR) method is proposed to speed up the detection and reduce the computational load. The SSR additionally decreases the false positive rate. A variety of classifiers, including support vector machine, extreme learning machine, and k-nearest neighbor, are used to train and classify vehicles using πHOG. The validity of the proposed method is demonstrated by its application to Caltech, IR, Pittsburgh, and Kitti datasets. The experimental results demonstrate that the proposed vehicle detection method not only improves detection performance but also reduces computation time.

In this study, we developed and evaluated a tactile stimulation device for upper extremity motion guidance. The developed device stimulates skin pressing directly using “tapping.” A minimal number of actuators are used in the tactile stimulation device that is worn on the wrist. The device consists of six Tiny Ultrasonic Linear Actuator (TULA) modules, a control circuit, an upper case, and a lower case. We estimated motions through kinematic analysis of the upper extremities for motion guidance and our driving algorithm applied a tactile illusion to generate directional information cues and tapped one point using a tactile stimulation device to guide upper extremity motion. To evaluate the developed device, an experiment was conducted to test whether directional information can be successfully displayed by the device. As a result, it was found that the directional information cues could be reliably conveyed through the wrist with tactile stimulation using a “tapping” method that is based on tactile illusion, though the number of actuators that display continuous tactile stimulation is limited.

This paper proposes an effective hypothesis generation for detection multi-vehicle using a monocular camera fixed on the host vehicle. In hypothesis generation (HG) step, we use linear model between the distance and vehicle size by using recursive least square. It generates effective image patches and improves the detection performance. In addition, it also reduces the computation time compared with sliding-window approach. In hypothesis verification (HV) step, we use the Histogram of Oriented Gradient (HOG) feature and Support Vector Machine (SVM). In our experiment, Caltech and IR datasets are used. The experimental result shows the improvement of running time and detection performance.

Biometric researchers have recently found considerable applicability of gait recognition in visual surveillance systems. This study proposes a probabilistic framework for gait modelling that is applied to gait recognition. The basic idea of this framework is to consider the silhouette shape as a multivariate random variable and model it in a full probabilistic framework. The Bernoulli mixture model is employed to model silhouette distribution and recursive algorithms are provided for silhouette image and sequence classification. Finally, the proposed probabilistic method is applied to benchmark databases and its validity is demonstrated through experiments.

A novel range-free localization algorithm based on the multidimensional support vector regression (MSVR) is proposed in this paper. The range-free localization problem is formulated as a multidimensional regression problem, and a new MSVR training method is proposed to solve the regression problem. Unlike standard support vector regression, the proposed MSVR allows multiple outputs and localizes the sensors without resorting to multilateration. The training of the MSVR is formulated directly in primal space and it can be solved in two ways. First, it is formulated as a second-order cone programming and trained by convex optimization. Second, its own training method is developed based on the Newton-Raphson method. A simulation is conducted for both isotropic and anisotropic networks, and the proposed method exhibits excellent and robust performance in both isotropic and anisotropic networks.

Many classification methods have been reported and the most popular ones among them are multilayer perceptron (MLP), nearest neighbor (NN), and support vector machine (SVM), etc. All of them have the weakness that they are not transparent or not clearly understandable to human beings. Sometimes, however, linguistically understandable classifiers could be preferred to the nontransparent models. Especially, when we are given a large set of data and we have to draw concise but interpretable hypothesis or conclusion, linguistically understandable classifiers should be required. In this paper, a linguistically understandable fuzzy classifier is presented and a new training method is proposed. To handle the uncertainties stemming from the problem or the measurement, the fuzzy classifier, the consequent part outputs the degree of truth for the assignment of each fuzzy set to the classes.

Particle filters perform the nonlinear estimation and have received much attention from many engineering fields over the past decade. Unfortunately, there are some cases in which most particles are concentrated prematurely at a wrong point, thereby losing diversity and causing the estimation to fail. In this paper, genetic algorithms (GAs) are incorporated into a particle filter to overcome this drawback of the filter. By using genetic operators, the premature convergence of the particles is avoided and the search region of particles enlarged. The GA-inspired proposal distribution is proposed and the corresponding importance weight is derived to approximate the given target distribution. Finally, a computer simulation is performed to show the effectiveness of the proposed method.

This paper presents an adaptive approach for synchronization of Takagi–Sugeno (T–S) fuzzy chaotic systems. T–S fuzzy model can represent a general class of nonlinear system and we employ it for fuzzy modeling of the chaotic drive system. Since the output of the drive system is only available for synchronization, the response system is designed based on fuzzy adaptive observer for uncertain parameters and parameter mismatch cases. We analyze the stability of the overall fuzzy synchronization system by applying Lyapunov stability theory and derive stability conditions by solving linear matrix inequalities (LMIs) problem. The adaptive law is derived to estimate the uncertain parameters or parameter mismatch. Numerical examples are given to demonstrate the validity of the proposed fuzzy adaptive synchronization approach.

This paper presents a flying display system using an autonomous blimp (small indoor airship) with a visual tracking system and an image projection system. The real-time visual tracking system tracks the blimp while it flies along a given spatial path to follow a wall. The image projection system projects still images or a video stream, whose rectangular shape is pre-compensated to look natural and flat using an image warping algorithm, on the surface of the blimp. The blimp is designed to have holonomic dynamics and it can maintain a stable pose and position in the presence of bounded air flow disturbances during the wall following motion. The real-time visual tracking system tracks the blimp and calculates its position in three dimensional space. Finally, we verify the capability of the autonomous blimp, the real-time visual tracking and the image projection system by experiments in the public exhibition environment. We also verify that the system is useful for transferring information and advertising in a crowded public area such as an exhibition hall or a department store

This paper is concerned with the robust tracking control of an electrically driven robot with the model uncertainties in the robot dynamics and the motor dynamics. The motors driving the joints of the robot are assumed to be equipped with only the joint position and the current measurement devices. Adaptive fuzzy logic and adaptive backstepping method are employed to provide the solution to the control problem. The suggested method does not require the measurement of the velocity nor the acceleration. Simulation results from a two-link electrically driven robot show the satisfactory performance of the proposed control scheme even in the presence of internal model uncertainties in both the robot and motor dynamics and external disturbances.

One of the most important objectives in the design of control systems is to achieve the good tracking performance in the presence of the internal parameter uncertainty and external disturbance. In this paper, a new multiple-input-multiple-output (MIMO) fuzzy disturbance observer (FDO) based on output measurement is developed to achieve the goal. A filtered signal is introduced to resolve the algebraic loop encountered in the conventional FDO. The contribution of the disturbance observation error /spl zeta/ to updating the parameters of the fuzzy system is analyzed in the sense of L/sub 2/ and L/sub /spl infin//. Then, the MIMO FDO is modified and the high gain observer (HGO) is employed to implement the output tracking control system. It is shown in a rigorous manner that the disturbance observation error, the tracking error and the state observation error converge to a compact set of which size can be kept arbitrarily small. Finally, the suggested method is applied to the speed control of a permanent magnet synchronous motor (PMSM) in the presence of the internal parameter uncertainty and external disturbance. The effectiveness and the feasibility of the suggested method are demonstrated by computer simulation.

A novel analysis and design method for affine fuzzy systems is proposed. Both continuous-time and discrete-time cases are considered. The quadratic stability and stabilizability conditions of the affine fuzzy systems are derived and they are represented in the formulation of bilinear matrix inequalities (BMIs). Two diffeomorphic state transformations (one is linear and the other is nonlinear) are introduced to convert the plant into more tractable affine form. The conversion makes the stability and stabilizability problems of the affine fuzzy systems convex and makes the problems solvable directly by the convex linear matrix inequality (LMI) technique. The bias terms of the fuzzy controller are solved simultaneously together with the gains. Finally, the applicability of the suggested method is demonstrated via an example and computer simulation.

A new robust TSK fuzzy modeling algorithm is proposed. The proposed algorithm is the modified version of noise clustering algorithm. Various robust approaches to deal with the data containing noise or outliers in real applications were proposed, but most algorithms process clustering of data first and then conduct fuzzy regression. We propose the algorithm that parameters of the premise part and the consequent part are obtained simultaneously. The proposed algorithm shows good performance against noise or outliers. Without adaptation of parameters, the proposed algorithm shows the superior performance over other approaches.

Many robot controllers require not only joint position measurements but also joint velocity measurements; however, most robotic systems are only equipped with joint position measurement devices. In this paper, a new output feedback tracking control approach is developed for the robot manipulators with model uncertainty. The approach suggested herein does not require velocity measurements and employs the adaptive fuzzy logic. The adaptive fuzzy logic allows us to approximate uncertain and nonlinear robot dynamics. Only one fuzzy system is used to implement the observer-controller structure of the output feedback robot system. It is shown in a rigorous manner that all the signals in a closed loop composed of a robot, an observer, and a controller are uniformly ultimately bounded. Finally, computer simulation results on three-link robot manipulators are presented to show the results which indicate good position tracking performance and robustness against payload uncertainty and external disturbances.

In this paper, stability analysis and synthesis of the type-II (Singleton-type) linguistic fuzzy control system are addressed. First, stability theorems are given for both continuous-time case and discrete-time case and they are recast in the formulation of bilinear matrix inequalities (BMIs). Continuous and discrete iterative linear matrix inequality methods are presented to obtain the feasible solution for the stability conditions represented as the BMI. The interesting feature of this paper is that the suggested method can be applied not only to the stability analysis but also to the controller synthesis. When it is applied to the synthesis of the type-II fuzzy logic controller, the resulting one can theoretically guarantee the stability of the closed-loop system. Finally, the effectiveness of the suggested method is illustrated via computer simulation.

This paper presents a new approach to robust tracking control of the nonlinear sampled systems using a discrete-time fuzzy disturbance observer (DFDO). Novel update and control laws are proposed to guarantee that all the signals in the closed-loop control system are uniformly ultimately bounded (UUB) in a compact set. No persistence of excitation (PE) condition, nor the assumption on the slowness of the change of the fuzzy parameters, is required. In addition, a robustifying controller is designed to improve the tracking performance. Finally, a computer simulation example is presented to illustrate the effectiveness and the applicability of the suggested method.

In this paper, a discrete-time fuzzy disturbance observer (FDO) is developed and its application to the control of a nonlinear system in the presence of the internal and external disturbances is presented. To construct the discrete-time FDO, a novel tuning method is proposed and shown to be useful in adjusting the parameters of the FDO. The tuning method uses the augmented error to guarantee that the FDO monitors the disturbance and the control objective is achieved. The design parameters such as a learning rate used in constructing the discrete-time FDO are determined in a numerical manner by solving a linear matrix inequality. It is shown in a rigorous manner that both the disturbance observation error and the control error converge to a compact set of which size can be kept arbitrarily small. In addition, the relationships between the suggested FDO-based control and the conventional adaptive fuzzy controls reported in the previous literatures are discussed. Finally, some examples and computer simulation results are presented to illustrate the effectiveness and the applicability of the FDO.

For original paper see ibid., vol.10 , p.412-414 (2002).In the aforementioned note, the authors insisted that the robot example reported in a previous paper can be effectively controlled without the need for the adaptive fuzzy component. In this note, we point out the technical flaw of the aforementioned note and demonstrate the value of the adaptive fuzzy systems in controlling the robots based on the simulation results.

In this paper, a fuzzy disturbance observer (FDO) is developed and its application to the control of a nonlinear system under the internal and external disturbances is presented. To construct the FDO, two parameter tuning methods are proposed and shown to be useful in adjusting the parameters of the FDO. The first tuning method employs the disturbance observation error to guarantee that the FDO monitors the unknown disturbance. The next one enlarges the concept of error and introduces augmented error to guarantee that the FDO monitors the disturbance and the control objective is achieved. In addition, the relationships between the suggested FDO-based control and the conventional adaptive fuzzy controls reported in the previous literatures are discussed and it is shown in a rigorous manner that the disturbance observation error or the augmented error converges to a region of which size can be kept arbitrarily small. Finally, some examples and computer simulation results are presented to illustrate the effectiveness and the applicability of the FDO.

A new stability analysis and controller synthesis methodology for a continuous affine fuzzy system is proposed in this paper. The method suggested is based on the numerical convex optimization techniques. In analysis, the stability condition under which the affine fuzzy system is quadratically stable is derived and is recast in the formulation of linear matrix inequalities (LMIs). The emphasis of this paper, however, is on the synthesis of fuzzy controller based on the derived stability condition. In the synthesis, the stabilizability condition turns out to be in the formulation of bilinear matrix inequalities and is solved numerically in an iterative manner. Fuzzy local controllers also assume the affine form and their bias terms are solved in a numerical manner simultaneously together with the gains. Continuous iterative LMI (ILMI) approach is presented to obtain a feasible solution for the synthesis of the affine fuzzy system.

The paper presents numerical methodology for the stability analysis of a fuzzy control system. The fuzzy control system analyzed is a closed-loop system controlled by a fuzzy logic controller (FLC) with singleton consequents. Compared with previous works based on numerical approaches (E. Kim et al., 1999), the method proposed in the paper employs two new strategies to release the conservatism of the previous methods: region-wise affine transformation and piecewise quadratic Lyapunov function. Finally, the effectiveness of the stability analysis is illustrated by a numerical example and its computer simulation.

This paper develops a stability analysis and controller synthesis methodology for a discrete affine fuzzy system based on the convex optimization techniques. In analysis, the stability condition under which the affine fuzzy system is quadratically stable is derived. Then, the condition Is recast in the formulation of Linear Matrix Inequalities (LMI) and numerically addressed. The emphasis of this paper, however, is on the synthesis of fuzzy controller based on the derived stability condition. In synthesis, the stabilizability condition turns out to be in the formulation of nonconvex matrix inequalities and is solved numerically in an iterative manner. Discrete iterative LMI (ILMI) approach is proposed to obtain the feasible solution for the synthesis of the affine fuzzy system. Finally, the applicability of the suggested methodology is demonstrated via some examples and computer simulations.

This paper deals with the quadratic stability conditions of fuzzy control systems that relax the existing conditions reported in the previous literatures. Two new conditions are proposed and shown to be useful in analyzing and designing fuzzy control systems. The first one employs the S-procedure to utilize information regarding the premise parts of the fuzzy systems. The next one enlarges the class of fuzzy control systems, whose stability is ensured by representing the interactions among the fuzzy subsystems in a single matrix and solving it by linear matrix inequality. The relationships between the suggested stability conditions and the conventional well-known stability conditions reported in the previous literatures are also discussed, and it is shown in a rigorous manner that the second condition of this paper includes the conventional conditions. Finally, some examples and simulation results are presented to illustrate the effectiveness of the stability conditions.

A limit-cycle is the phenomenon that can be observed in systems composed of nonlinear elements. The phenomenon is of fundamental importance in nonlinear systems and, as far as the design of a nonlinear system is concerned, it should be considered along with the stability analysis. In the paper, the limit-cycle of a system controlled by a fuzzy logic controller (FLC) is addressed via some of the classical control techniques used to analyze nonlinear systems in the frequency domain. First, reasonable assumptions are made on the structure of the FLC by using fuzzy basis functions (FBFs) and the describing function of the FLC is derived to analyze and predict the existence of the limit-cycle of the closed-loop system including the FLC. Finally computer simulation is performed to show how the analysis given in the paper is used to predict the existence of the limit-cycle of the fuzzy control system.

This paper proposes a numerical stability analysis methodology for the singleton-type linguistic fuzzy control systems based on optimization techniques. First, it demonstrates that a singleton-type linguistic fuzzy logic controller (FLC) can be converted into a region-wise sector-bounded controller or, more generally, a polytopic system by quadratic programming (QP). Next, the convex optimization technique called linear matrix inequalities (LMI) is used to analyze the closed loop of the converted polytopic system. Finally, the applicability of the suggested methodology is highlighted via simulation results.

This paper proposes a new variable structure controller (VSC) for the accurate tracking control of a manipulator using the variable boundary layer. Up to now, VSC applying the variable boundary layer did not remove chattering from an arbitrary initial state of the system trajectory because VSC has used the fixed sliding surface. But, by using the linear time-varying sliding surfaces, the scheme has the robustness against chattering from all states. This scheme can be applied to the second-order nonlinear systems with parameter uncertainty and extraneous disturbances, and have better tracking performance than the conventional method. To demonstrate its performance, the proposed control algorithm is applied to a two-link manipulator.

This paper presents an explanation of a fuzzy model considering the correlation among components of input data. Generally, fuzzy models have a capability of dividing an input space into several subspaces compared to a linear model. But hitherto suggested fuzzy modeling algorithms have not taken into consideration the correlation among components of sample data and have addressed them independently, which results in an ineffective partition of the input space. In order to solve this problem, this paper proposes a new fuzzy modeling algorithm, which partitions the input space more effectively than conventional fuzzy modeling algorithms by taking into consideration the correlation among components of sample data. As a way to use the correlation and divide the input space, the method of principal component is used. Finally, the results of the computer simulation are given to demonstrate the validity of this algorithm.

This paper proposes a new approach to fuzzy modeling. The suggested fuzzy model can express a given unknown system with a few fuzzy rules as well as Takagi and Sugeno's model (1985), because it has the same structure as that of Takagi and Sugeno's model. It is also as easy to implement as Sugeno and Yasukawa's model (1993) because its identification mimics the simple identification procedure of Sugeno and Yasukawa's model. The suggested algorithm is composed of two steps: coarse tuning and fine tuning. In coarse tuning, fuzzy C-regression model (FCRM) clustering is used, which is a modified version of fuzzy C-means (FCM). In fine tuning, gradient descent algorithm is used to precisely adjust parameters of the fuzzy model instead of nonlinear optimization methods used in other models. Finally, some examples are given to demonstrate the validity of this algorithm.

A fuzzy adaptive controller which is able to solve the problems of classical adaptive controllers and conventional fuzzy adaptive controller is suggested. The architecture of a fuzzy adaptive controller using the robust property of a fuzzy controller is explained. A design procedure which can be carried out mathematically and systematically from the model of an objective system is suggested, and related mathematical theorems and their proofs are also given. The performance of the proposed adaptive control algorithm is analyzed through a DC motor control simulation and experiments on a compliant robot system.