Accurate localization is essential for autonomous driving, but GNSS-based methods struggle in challenging environments such as urban canyons. Cross-view pose optimization offers an effective solution for localization by directly estimating vehicle pose using satellite-view images. However, existing methods primarily rely on cross-view features at a given pose, neglecting fine-grained contexts for precision and global contexts for robustness against large initial pose errors. To overcome these limitations, we propose PIDLoc, a novel cross-view pose optimization approach inspired by the proportional-integral-derivative (PID) controller. Using RGB images and LiDAR data, the PIDLoc models cross-view feature relationships through the PID branches and estimates pose via the spatially aware pose estimator (SPE). To enhance localization accuracy, the PID branches leverage feature differences for local context (P), aggregated feature differences for global context (I), and gradients of feature differences for fine-grained context (D). Integrated with the PID branches, the SPE captures spatial relationships within the PID-branch features for consistent localization. Experimental results demonstrate that the PIDLoc achieves state-of-the-art performance in cross-view pose estimation for the KITTI dataset, reducing position error by 37.8% compared with the previous state-of-the-art. Our code is available at https://github.com/url-kaist/PIDLoc

Prompt tuning, which adapts vision-language models by freezing model parameters and opti- mizing only the prompt, has proven effective for task-specific adaptations. The core challenge in prompt tuning is improving specialization for a specific task and generalization for unseen domains. However, frozen encoders often produce misaligned features, leading to confusion between classes and limiting specialization. To overcome this issue, we propose a confusion-aware loss (CoA-loss) that improves specialization by refining the decision boundaries between confusing classes. Additionally, we mathematically demonstrate that a mixture model can enhance generalization without compromising specialization. This is achieved using confidence-aware weights (CoA- weights), which adjust the weights of each prediction in the mixture model based on its confidence within the class domains. Extensive experiments show that CoCoA-Mix, a mixture model with CoA-loss and CoA-weights, outperforms state-of-the-art methods by enhancing specialization and generalization. Our code is publicly available at https://github.com/url-kaist/CoCoA-Mix

Recent advances in quadrupedal robots have demonstrated impressive agility and the ability to traverse diverse terrains. However, hardware issues, such as motor overheating or joint locking, may occur during long-distance walking or traversing through rough terrains leading to locomotion failures. Although several studies have proposed fault-tolerant control methods for quadrupedal robots, there are still challenges in traversing unstructured terrains. In this paper, we propose DreamFLEX, a robust fault-tolerant locomotion controller that enables a quadrupedal robot to traverse complex environments even under joint failure conditions. DreamFLEX integrates an explicit failure estimation and modulation network that jointly estimates the robot's joint fault vector and utilizes this information to adapt the locomotion pattern to faulty conditions in real-time, enabling quadrupedal robots to maintain stability and performance in rough terrains. Experimental results demonstrate that DreamFLEX outperforms existing methods in both simulation and real-world scenarios, effectively managing hardware failures while maintaining robust locomotion performance.

While global point cloud registration systems have advanced significantly in all aspects, many studies have focused on specific components, such as feature extraction, graph-theoretic pruning, or pose solvers. In this paper, we take a holistic view on the registration problem and develop an open-source and versatile C++ library for point cloud registration, called KISS-Matcher. KISS-Matcher combines a novel feature detector, Faster-PFH, that improves over the classical fast point feature histogram (FPFH). Moreover, it adopts a k-core-based graph-theoretic pruning to reduce the time complexity of rejecting outlier correspondences. Finally, it combines these modules in a complete, user-friendly, and ready-to-use pipeline. As verified by extensive experiments, KISS-Matcher has superior scalability and broad applicability, achieving a substantial speed-up compared to state-of-the-art outlier-robust registration pipelines while preserving accuracy. Our code will be available at https://github.com/MIT-SPARK/KISS-Matcher.

In recent years, robust matching methods using deep learning-based approaches have been actively studied and improved in computer vision tasks. However, there remains a persistent demand for both robust and fast matching techniques. To address this, we propose a novel Mamba-based local feature matching approach, called MambaGlue, where Mamba is an emerging state-of-the-art architecture rapidly gaining recognition for its superior speed in both training and inference, and promising performance compared with Transformer architectures. In particular, we propose two modules: a) MambaAttention mixer to simultaneously and selectively understand the local and global context through the Mamba-based self-attention structure and b) deep confidence score regressor, which is a multi-layer perceptron (MLP)-based architecture that evaluates a score indicating how confidently matching predictions correspond to the ground-truth correspondences. Consequently, our MambaGlue achieves a balance between robustness and efficiency in real-world applications. As verified on various public datasets, we demonstrate that our MambaGlue yields a substantial performance improvement over baseline approaches while maintaining fast inference speed. Our code will be available on https://github.com/url-kaist/MambaGlue.

Multi-robot localization is a crucial task for implementing multi-robot systems. Numerous researchers have proposed optimization-based multi-robot localization methods that use camera, IMU, and UWB sensors. Nevertheless, characteristics of individual robot odometry estimates and distance measurements between robots used in the optimization are not sufficiently considered. In addition, previous researches were heavily influenced by the odometry accuracy that is estimated from individual robots. Consequently, long-term drift error caused by error accumulation is potentially inevitable. In this paper, we propose a novel visual-inertial-range-based multi-robot localization method, named SaWa-ML, which enables geometric structure-aware pose correction and weight adaptation-based robust multi-robot localization. Our contributions are twofold: (i) we leverage UWB sensor data, whose range error does not accumulate over time, to first estimate the relative positions between robots and then correct the positions of each robot, thus reducing long-term drift errors, (ii) we design adaptive weights for robot pose correction by considering the characteristics of the sensor data and visual-inertial odometry estimates. The proposed method has been validated in real-world experiments, showing a substantial performance increase compared with state-of-the-art algorithms.

Scene graphs have emerged as a powerful tool for robots, providing a structured representation of spatial and semantic relationships for advanced task planning. Despite their potential, conventional 3D indoor scene graphs face critical limitations, particularly under- and over-segmentation of room layers in structurally complex environments. Under-segmentation misclassifies non-traversable areas as part of a room, often in open spaces, while over-segmentation fragments a single room into overlapping segments in complex environments. These issues stem from naive voxel-based map representations that rely solely on geometric proximity, disregarding the structural constraints of traversable spaces and resulting in inconsistent room layers within scene graphs. To the best of our knowledge, this work is the first to tackle segmentation inconsistency as a challenge and address it with Traversability-Aware Consistent Scene Graphs (TACS-Graphs), a novel framework that integrates ground robot traversability with room segmentation. By leveraging traversability as a key factor in defining room boundaries, the proposed method achieves a more semantically meaningful and topologically coherent segmentation, effectively mitigating the inaccuracies of voxel-based scene graph approaches in complex environments. Furthermore, the enhanced segmentation consistency improves loop closure detection efficiency in the proposed Consistent Scene Graph-leveraging Loop Closure Detection (CoSG-LCD) leading to higher pose estimation accuracy. Experimental results confirm that the proposed approach outperforms state-of-the-art methods in terms of scene graph consistency and pose graph optimization performance.

Depth information which specifies the distance between objects and current position of the robot is essential for many robot tasks such as navigation. Recently, researchers have proposed depth completion frameworks to provide dense depth maps that offer comprehensive information about the surrounding environment. However, existing methods show significant trade-offs between computational efficiency and accuracy during inference. The substantial memory and computational requirements make them unsuitable for real-time applications, highlighting the need to improve the completeness and accuracy of depth information while improving processing speed to enhance robot performance in various tasks. To address these challenges, in this paper, we propose CHADET (cross-hierarchical-attention depth-completion transformer), a lightweight depth-completion network that can generate accurate dense depth maps from RGB images and sparse depth points. For each pair, its feature is extracted from the depthwise blocks and passed to the equally lightweight transformer-based decoder. In the decoder, we utilize the novel cross-hierarchical-attention module that refines the image features from the depth information. Our approach improves the quality and reduces memory usage of the depth map prediction, as validated in both KITTI, NYUv2, and VOID datasets.

Single-domain generalization (S-DG) aims to generalize a model to unseen environments with a single-source domain. However, most S-DG approaches have been conducted in the field of classification. When these approaches are applied to object detection, the semantic features of some objects can be damaged, which can lead to imprecise object localization and misclassification. To address these problems, we propose an object-aware domain generalization (OA-DG) method for single-domain generalization in object detection. Our method consists of data augmentation and training strategy, which are called OA-Mix and OA-Loss, respectively. OA-Mix generates multi-domain data with multi-level transformation and object-aware mixing strategy. OA-Loss enables models to learn domain-invariant representations for objects and backgrounds from the original and OA-Mixed images. Our proposed method outperforms state-of-the-art works on standard benchmarks. Our code is available at https://github.com/WoojuLee24/OA-DG.

Deep neural networks have shown remarkable performance in image classification. However, their performance significantly deteriorates with corrupted input data. Domain generalization methods have been proposed to train robust models against out-of-distribution data. Data augmentation in the frequency domain is one of such approaches that enable a model to learn phase features to establish domain-invariant representations. This approach changes the amplitudes of the input data while preserving the phases. However, using fixed phases leads to susceptibility to phase fluctuations because amplitudes and phase fluctuations commonly occur in out-of-distribution. In this study, to address this problem, we introduce an approach using finite variation of the phases of input data rather than maintaining fixed phases. Based on the assumption that the degree of domain-invariant features varies for each phase, we propose a method to distinguish phases based on this degree. In addition, we propose a method called vital phase augmentation (VIPAug) that applies the variation to the phases differently according to the degree of domain-invariant features of given phases. The model depends more on the vital phases that contain more domain-invariant features for attaining robustness to amplitude and phase fluctuations. We present experimental evaluations of our proposed approach, which exhibited improved performance for both clean and corrupted data. VIPAug achieved SOTA performance on the benchmark CIFAR-10 and CIFAR-100 datasets, as well as near-SOTA performance on the ImageNet-100 and ImageNet datasets. Our code is available at https://github.com/excitedkid/vipaug.

Despite great improvements in semantic segmentation, challenges persist because of the lack of local/global contexts and the relationship between them. In this paper, we propose Contextrast, a contrastive learning-based semantic segmentation method that allows to capture local/global contexts and comprehend their relationships. Our proposed method comprises two parts: a) contextual contrastive learning (CCL) and b) boundary-aware negative (BANE) sampling. Contextual contrastive learning obtains local/global context from multi-scale feature aggregation and inter/intra-relationship of features for better discrimination capabilities. Meanwhile, BANE sampling selects embedding features along the boundaries of incorrectly predicted regions to employ them as harder negative samples on our contrastive learning, resolving segmentation issues along the boundary region by exploiting fine-grained details. We demonstrate that our Contextrast substantially enhances the performance of semantic segmentation networks, outper-forming state-of-the-art contrastive learning approaches on diverse public datasets, e.g. Cityscapes, CamVid, PASCALC, COCO-Stuff, and ADE20K, without an increase in computational cost during inference.

Moving object segmentation (MOS) using a 3D light detection and ranging (LiDAR) sensor is crucial for scene understanding and identification of moving objects. Despite the availability of various types of 3D LiDAR sensors in the market, MOS research still predominantly focuses on 3D point clouds from mechanically spinning omnidirectional LiDAR sensors. Thus, we are, for example, lacking a dataset with MOS labels for point clouds from solid-state LiDAR sensors which have irregular scanning patterns. In this paper, we present a labeled dataset, called HeLiMOS, that enables to test MOS approaches on four heterogeneous LiDAR sensors, including two solid-state LiDAR sensors. Furthermore, we introduce a novel automatic labeling method to substantially reduce the labeling effort required from human annotators. To this end, our framework exploits an instance-aware static map building approach and tracking-based false label filtering. Finally, we provide experimental results regarding the performance of commonly used state-of-the-art MOS approaches on HeLiMOS that suggest a new direction for a sensor-agnostic MOS, which generally works regardless of the type of LiDAR sensors used to capture 3D point clouds. Our dataset is available at https://sites.google.com/view/helimos.

Recognizing traversable terrain from 3D point cloud data is critical, as it directly impacts the performance of autonomous navigation in off-road environments. However, existing segmentation algorithms often struggle with challenges related to changes in data distribution, environmental specificity, and sensor variations. Moreover, when encountering sunken areas, their performance is frequently compromised, and they may even fail to recognize them. To address these challenges, we introduce B-TMS, a novel approach that performs map-wise terrain modeling and segmentation by utilizing Bayesian generalized kernel (BGK) within the graph structure known as the tri-grid field (TGF). Our experiments encompass various data distributions, ranging from single scans to partial maps, utilizing both public datasets representing urban scenes and off-road environments, and our own dataset acquired from extremely bumpy terrains. Our results demonstrate notable contributions, particularly in terms of robustness to data distribution variations, adaptability to diverse environmental conditions, and resilience against the challenges associated with parameter changes.

With the rapid development of autonomous driving and SLAM technology, the performance of autonomous systems using multimodal sensors highly relies on accurate extrinsic calibration. Addressing the need for a convenient, maintenance-friendly calibration process in any natural environment, this paper introduces Galibr, a fully automatic targetless LiDAR-camera extrinsic calibration tool designed for ground vehicle platforms in any natural setting. The method utilizes the ground planes and edge information from both LiDAR and camera inputs, streamlining the calibration process. It encompasses two main steps: an initial pose estimation algorithm based on ground planes (GP-init), and a refinement phase through edge extraction and matching. Our approach significantly enhances calibration performance, primarily attributed to our novel initial pose estimation method, as demonstrated in unstructured natural environments, including on the KITTI dataset and the KAIST quadruped dataset.

Radar sensors are emerging as solutions for perceiving surroundings and estimating ego-motion in extreme weather conditions. Unfortunately, radar measurements are noisy and suffer from mutual interference, which degrades the performance of feature extraction and matching, triggering imprecise matching pairs, which are referred to as outliers. To tackle the effect of outliers on radar odometry,$a$novel outlier-robust method called ORORA is proposed, which is an abbreviation of Outlier-RObust RAdar odometry. To this end, a novel decoupling-based method is proposed, which consists of graduated non-convexity (GNC)-based rotation estimation and anisotropic component-wise translation estimation (A-COTE). Furthermore, our method leverages the anisotropic characteristics of radar measurements, each of whose uncertainty along the azimuthal direction is somewhat larger than that along the radial direction. As verified in the public dataset, it was demonstrated that our proposed method yields robust ego-motion estimation performance compared with other state-of-the-art methods. Our code is available at https://github.com/url-kaist/outlier-robust-radar-odometry.

Quadrupedal robots resemble the physical ability of legged animals to walk through unstructured terrains. However, designing a controller for quadrupedal robots poses a significant challenge due to their functional complexity and requires adaptation to various terrains. Recently, deep reinforcement learning, inspired by how legged animals learn to walk from their experiences, has been utilized to synthesize natural quadrupedal locomotion. However, state-of-the-art methods strongly depend on a complex and reliable sensing framework. Furthermore, prior works that rely only on proprioception have shown a limited demonstration for overcoming challenging terrains, especially for a long distance. This work proposes a novel quadrupedal locomotion learning framework that allows quadrupedal robots to walk through challenging terrains, even with limited sensing modalities. The proposed framework was validated in real-world outdoor environments with varying conditions within a single run for a long distance.

Line information from urban structures can be exploited as an additional geometrical feature to achieve robust vision-based simultaneous localization and mapping (SLAM) systems in textureless scenes. Sometimes, however, conventional line tracking methods fail to track caused by image blur or occlusion. Even though these lost line features are just a subset of plenty of features, the failure in feature tracking can potentially lead to performance degradation of the SLAM system, particularly in textureless environments. To tackle this problem, we propose a robust line-tracking method for line-based monocular visual-inertial odometry. The proposed method generates a semi-dense map composed of depth and sparsity mesh using estimated 3D features. By leveraging the semi-dense map, our method performs a range-adaptive epipo-lar search to match the lines, allowing for robust line tracking while simultaneously reducing false positives. Furthermore, an algorithm to avoid conflicts is proposed, which occurs when the tracked lines from consecutive matching do not accord with the lines matched by our method. This algorithm discriminately maintains line features while appropriately aggregating lines spread across multiple frames. As evaluated in the EuRoC dataset and a more challenging textureless corridor scene, our proposed method shows substantial performance increases compared with other line-based visual (-inertial) approaches.

Deep neural networks (DNNs) are vulnerable to adversarial examples that are carefully designed to cause the deep learning model to make mistakes. Adversarial examples of 2D images and 3D point clouds have been extensively studied, but studies on event-based data are limited. Event-based data can be an alternative to a 2D image under high-speed movements, such as autonomous driving. However, the given adversarial events make the current deep learning model vulnerable to safety issues. In this work, we generate adversarial examples and then train the robust models for event-based data, for the first time. Our algorithm shifts the time of the original events and generates additional adversarial events. Additional adversarial events are generated in two stages. First, null events are added to the event-based data to generate additional adversarial events. The perturbation size can be controlled with the number of null events. Second, the location and time of additional adversarial events are set to mislead DNNs in a gradient-based attack. Our algorithm achieves an attack success rate of 97.95% on the N-Caltech101 dataset. Furthermore, the adversarial training model improves robustness on the adversarial event data compared to the original model.

Global registration using 3D point clouds is a crucial technology for mobile platforms to achieve localization or manage loop-closing situations. In recent years, numerous researchers have proposed global registration methods to address a large number of outlier correspondences. Unfortunately, the degeneracy problem, which represents the phenomenon in which the number of estimated inliers becomes lower than three, is still potentially inevitable. To tackle the problem, a degeneracy-robust decoupling-based global registration method is proposed, called Quatro. In particular, our method employs quasi-SO(3) estimation by leveraging the Atlanta world assumption in urban environments to avoid degeneracy in rotation estimation. Thus, the minimum degree of freedom (DoF) of our method is reduced from three to one. As verified in indoor and outdoor 3D LiDAR datasets, our proposed method yields robust global registration performance compared with other global registration methods, even for distant point cloud pairs. Furthermore, the experimental results confirm the applicability of our method as a coarse alignment. Our code is available: https://github.com/url-kaist/quatro

Contrary to other standard cameras, event cam-eras interpret the world in an entirely different manner; as a collection of asynchronous events. Despite event camera's unique data output, many event feature detection and tracking algorithms have shown significant progress by making detours to frame-based data representations. This paper questions the need to do so and proposes a novel event data-friendly method that achieve simultaneous feature detection and tracking, called event Clustering-based Detection and Tracking (eCDT). Our method employs a novel clustering method, named as k-NN Classifier-based Spatial Clustering and Applications with Noise (KCSCAN), to cluster adjacent polarity events to retrieve event trajectories. With the aid of a Head and Tail Descriptor Matching process, event clusters that reappear in a different polarity are continually tracked, elongating the feature tracks. Thanks to our clustering approach in spatio-temporal space, our method automatically solves feature detection and feature tracking simultaneously. Also, eCDT can extract feature tracks at any frequency with an adjustable time window, which does not corrupt the high temporal resolution of the original event data. Our method achieves 30 % better feature tracking ages compared with the state-of-the-art approach while also having a low error approximately equal to it.

In the field of 3D perception using 3D LiDAR sensors, ground segmentation is an essential task for various purposes, such as traversable area detection and object recognition. Under these circumstances, several ground segmentation methods have been proposed. However, some limitations are still encountered. First, some ground segmentation methods require fine-tuning of parameters depending on the surroundings, which is excessively laborious and time-consuming. Moreover, even if the parameters are well adjusted, a partial under-segmentation problem can still emerge, which implies ground segmentation failures in some regions. Finally, ground segmentation methods typically fail to estimate an appropriate ground plane when the ground is above another structure, such as a retaining wall. To address these problems, we propose a robust ground segmentation method called Patchwork++, an extension of Patchwork. Patchwork++ exploits adaptive ground likelihood estimation (A-GLE) to calculate appropriate parameters adaptively based on the previous ground segmentation results. Moreover, temporal ground revert (TGR) alleviates a partial under-segmentation problem by using the temporary ground property. Also, region-wise vertical plane fitting (R-VPF) is introduced to segment the ground plane properly even if the ground is elevated with different layers. Finally, we present reflected noise removal (RNR) to eliminate virtual noise points efficiently based on the 3D LiDAR reflection model. We demonstrate the qualitative and quantitative evaluations using a SemanticKITTI dataset. Our code is available at https://github.com/url-kaist/patchwork-plusplus

In this paper, a degeneracy avoidance method for a point and line based visual SLAM algorithm is proposed. Visual SLAM predominantly uses point features. However, point features lack robustness in low texture and illuminance variant environments. Therefore, line features are used to compensate the weaknesses of point features. In addition, point features are poor in representing discernable features for the naked eye, meaning mapped point features cannot be recognized. To overcome the limitations above, line features were actively employed in previous studies. However, since degeneracy arises in the process of using line features, this paper attempts to solve this problem. First, a simple method to identify degenerate lines is presented. In addition, a novel structural constraint is proposed to avoid the degeneracy problem. At last, a point and line based monocular SLAM system using a robust optical-flow based lien tracking method is implemented. The results are verified using experiments with the EuRoC dataset and compared with other state-of-the-art algorithms. It is proven that our method yields more accurate localization as well as mapping results.

Multirotors can be effectively applied to various tasks, such as transportation, investigation, exploration, and lifesaving, depending on the type of payload. However, due to the nature of multirotors, the payload loaded on the multirotor is limited in its position and weight, which presents a major disadvantage when the multirotor is used in various fields. In this paper, we propose a novel method that greatly improves the restrictions on payload position and weight using a morphing quadrotor system. Our method can estimate the drone’s weight, center of gravity position, and inertia tensor in real-time, which change depending on payload, and determine the optimal morphology for efficient and stable flight. An adaptive control method that can reflect the change in flight dynamics by payload and morphing is also presented. Experiments were conducted to confirm that the proposed morphing quadrotor improves the stability and efficiency in various situations of transporting payloads compared with the conventional quadrotor systems.

Exploring an unknown environment without colliding with obstacles is one of the essentials of autonomous vehicles to perform diverse missions such as structural inspections, rescues, deliveries, and so forth. Therefore, unmanned aerial vehicles (UAVS), which are fast, agile, and have high degrees of freedom, have been widely used. However, previous approaches have two limitations: a) First, they may not be appropriate for exploring large-scale environments because they mainly depend on random sampling-based path planning that causes unnecessary movements. b) Second, they assume the pose estimation is accurate enough, which is the most critical factor in obtaining an accurate map. In this paper, to explore and map unknown large-scale environments rapidly and accurately, we propose a novel exploration method that combines the pre-calculated Peacock Trajectory with graph-based global exploration and active loop-closing. Because the two-step trajectory that considers the kinodynamics of UAVs is used, obstacle avoidance is guaranteed in the receding-horizon manner. In addition, local exploration that considers the frontier and global exploration based on the graph maximizes the speed of exploration by minimizing unnecessary revisiting. In addition, by actively closing the loop based on the likelihood, pose estimation performance is improved. The proposed method’s performance is verified by exploring 3D simulation environments in comparison with the state-of-the-art methods. Finally, the proposed approach is validated in a real-world experiment.

Recently, needs for unmanned aerial vehicles (UAVs) that are attachable to the wall have been highlighted. As one of the ways to address the need, researches on various tilting multirotors that can increase maneuverability has been employed. Unfortunately, existing studies on the tilting multirotors require considerable amounts of prior information on the complex dynamic model. Meanwhile, reinforcement learning on quadrotors has been studied to mitigate this issue. Yet, these are only been applied to standard quadrotors, whose systems are less complex than those of tilting multirotors. In this paper, a novel reinforcement learning-based method is proposed to control a tilting multirotor on real-world applications, which is the first attempt to apply reinforcement learning to a tilting multirotor. To do so, we propose a novel reward function for a neural network model that takes power efficiency into account. The model is initially trained over a simulated environment and then fine-tuned using real-world data in order to overcome the sim-to-real gap issue. Furthermore, a novel, efficient state representation with respect to the goal frame that helps the network learn optimal policy better is proposed. As verified on real-world experiments, our proposed method shows robust controllability by overcoming the complex dynamics of tilting multirotors.

Localization is one of the most important technologies needed to use Unmanned Aerial Vehicles (UAVs) in actual fields. Currently, most UAVs use GNSS to estimate their position. Recently, there have been attacks that target the weaknesses of UAVs that use GNSS, such as interrupting GNSS signal to crash the UAVs or sending fake GNSS signals to hijack the UAVs. To avoid this kind of situation, this paper proposes an algorithm that deals with the localization problem of the UAV in GNSS-denied environments. We propose a localization method, named as BRM (Building Ratio Map based) localization, for a UAV by matching an existing numerical map with UAV images. The building area is extracted from the UAV images. The ratio of buildings that occupy in the corresponding image frame is calculated and matched with the building information on the numerical map. The position estimation is started in the range of several km2area, so that the position estimation can be performed without knowing the exact initial coordinate. Only freely available maps are used for training data set and matching the ground truth. Finally, we get real UAV images, IMU data, and GNSS data from UAV flight to show that the proposed method can achieve better performance than the conventional methods.

Interests in exploration of new energy resources are increasing due to the exhaustion of existing resources. To explore new energy sources, various studies have been conducted to improve the drilling performance of drilling equipment for deep and strong ground. However, with better performance, the modern drilling equipment is bulky and, furthermore, has become inconvenient in both installation and operation, for it takes complex procedures for complex terrains. Moreover, environmental issues are also a concern because of the excessive use of mud and slurry to remove excavated soil. To overcome these limitations, a mechanism that combines an expandable drill bit and link structure to simulate the function of the teeth and forelimbs of a mole is proposed. In this paper, the proposed expandable drill bit simplifies the complexity and high number of degrees of freedom of the animal head. In addition, a debris removal mechanism mimicking a shoulder structure and forefoot movement is proposed. For efficient debris removal, the proposed mechanism enables the simultaneous rotation and expanding/folding motions of the drill bit by using a single actuator. The performance of the proposed system is evaluated by dynamic simulations and experiments.

In this study, a deep-learning-based multi-stage network architecture called Multi-Stage Depth Prediction Network (MSDPN) is proposed to predict a dense depth map using a 2D LiDAR and a monocular camera. Our proposed network consists of a multi-stage encoder-decoder architecture and Cross Stage Feature Aggregation (CSFA). The proposed multi-stage encoder-decoder architecture alleviates the partial observation problem caused by the characteristics of a 2D LiDAR, and CSFA prevents the multi-stage network from diluting the features and allows the network to learn the interspatial relationship between features better. Previous works use sub-sampled data from the ground truth as an input rather than actual 2D LiDAR data. In contrast, our approach trains the model and conducts experiments with a physically-collected 2D LiDAR dataset. To this end, we acquired our own dataset called KAIST RGBD-scan dataset and validated the effectiveness and the robustness of MSDPN under realistic conditions. As verified experimentally, our network yields promising performance against state-of-the-art methods. Additionally, we analyzed the performance of different input methods and confirmed that the reference depth map is robust in untrained scenarios.

In this study, a three-layered bidirectional Long Short-term Memory (Bi-LSTM) with residual attention, named as RONet, is proposed to achieve localization using range measurements. Accordingly, we acquired our own datasets and tested RONet using realistic conditions. It is shown that the RONet can estimate the position of the mobile robot in real time using the Nvidia Jetson AGX Xavier based only on range measurements. We also analyzed the sequence length of LSTM as a type of hyperparameters. We found that optimal sequence length is eight for more than eight anchors and twelve for fewer anchors compared to sequences with different lengths, given that construction of the network with the optimal sequence length estimates the position precisely and accounts for uncertainties. As verified experimentally, RONet yields more precise performance and results in increased robustness against outliers compared to a conventional range-only approach based on a particle filtering and the other conventional deep-learning-based approaches. We set three cases, reduced the number of anchors, and verified that the RONet was a robust solution. We also confirmed that it is the best solution that yields the smallest Root-Mean-Square-Error (RMSE) values, equal to 4.466 cm, 3.210 cm, and 3.090 cm, in the cases where three, five, and eight anchors were deployed, respectively.

In this paper, a novel 3D coverage path planning method, which is efficient and practical for inspection of high-rise structures such as buildings or towers, using an unmanned aerial vehicle (UAV) is presented. Our approach basically focuses on developing a model-based path planner for structural inspection with a prior map, which is opposite to a non-model based exploration. The proposed method uses a volumetric map which is made before the path planning. With the map, the whole structure is divided into several layers for efficient path planning. Firstly, in each layer, a set of the normal vectors of the center point of every voxel is calculated, and then the opposing vectors become viewpoints. Due to too many viewpoints and an overlapped inspection surface, we down-sample them with a voxel grid filter. Then, the shortest tour connecting the reduced viewpoints must be computed with the Traveling Salesman Problem (TSP) solver. Lastly, all the paths in each layer are combined to form the complete path. The results are verified using simulations with a rotary wing UAV and compared with other state-of-the-art algorithm. It is proven that our method performs much better for structural inspection with respect to computation time as well as the coverage completeness.

With the increase in developments in underwater infrastructure, the demand for development of autonomous vehicle navigation system in structured environment is also increased. However, the localization in a structured environment is a challenging problem due to signal uncertainties and distortions. In order to overcome these problems, we propose the camera and sonar aided integrated navigation system. In the proposed sensor-fusion-based localization scheme, the AUV estimates its own position continuously using artificial landmarks. The artificial landmarks for image sonar is deployed along the path to guide the AUV to the structure. The active vision markers are installed on the jacket structure, and they function as both landmarks and waypoints. This approach prevents the inherent drift of dead-reckoning velocities and collision with structures. The proposed approach was verified through a real sea experiment. The AUV conducted the full autonomous navigation from the dock to the jacket structure, and then returned to the dock without collision or significant localization error. These results show the feasibility of full autonomous navigation in a structured environment.

Recent advances in 3D depth sensors have created many opportunities for security, surveillance, and entertainment. The 3D depth sensors provide more powerful monitoring systems for dangerous situations irrespective of lighting conditions in buildings or production facilities. To robustly recognize emergency actions or hazardous situations of workers at a production facility, we present human joint estimation and behavior recognition algorithms that solely use depth information in this paper. To estimate human joints on a low cost computing platform, we propose a human joint estimation algorithm that integrates a geodesic graph and a support vector machine (SVM). The human feature points are extracted within a range of geodesic distance from a geodesic graph. The geodesic graph is used for optimizing the estimation result. The SVM-based human joint estimator uses randomly selected human features to reduce computation. Body parts that typically involve many motions are then estimated by the geodesic distance value. The proposed algorithm can work for any human without calibration, and thus the system can be used with any subject immediately even with a low cost computing platform. In the case of the behavior recognition algorithm, the algorithm should have a simple behavior registration process, and it also should be robust to environmental changes. To meet these goals, we propose a template matching-based behavior recognition algorithm. Our method creates a behavior template set that consists of weighted human joint data with scale and rotation invariant properties . A single behavior template consists of the joint information that is estimated per frame. Additionally, we propose adaptive template rejection and a sliding window filter to prevent misrecognition between similar behaviors. The human joint estimation and behavior recognition algorithms are evaluated individually through several experiments and the performance is proven through a comparison with other algorithms. The experimental results show that our method performs well and is applicable in real environments.

This paper proposes a cooperative coevolutionary approach to multi-robot formation control. To deal with the formation control problem, the concept of a cooperative coevo-lution (CC) framework is incorporated with model predictive control (MPC) such that candidates of all robots coevolve toward a Nash equilibrium in a distributed way. Using the Nash-equilibrium strategy, the robots can quickly move to a desired formation from their initial locations. The stability is guaranteed via a novel repair algorithm that enforces each candidate to satisfy a derived condition for asymptotic stability. The cooperative coevolutionary particle swarm optimization (CCPSO) is adopted and modified to fit into the formation control problem. Simulations are performed on a group of nonholonomic mobile robots to demonstrate the effectiveness of the CC-based MPC. Also, the proposed MPC shows a better performance compared to sequential quadratic programming (SQP)-based MPC.

In this paper, a novel cooperative coevolution (CC)-based model predictive control (MPC) is proposed to deal with formation control problem. While conventional evolutionary algorithm (EA)-based MPC approaches cannot guarantee the stability, the proposed CC-based MPC guarantees the asymptotic stability regardless of the optimality of the solution that the CC-based algorithm generates with a small number of individuals. To guarantee the stability, a terminal state constraint is found, and then a repair algorithm is applied to all candidate solutions to meet the constraint. The cooperatively coevolving particle swarm optimization (CCPSO), most recently developed algorithm among CC-based EAs, is adopted and modified to fit into the formation control problem. Experiments are performed on a group of nonholonomic mobile robots to demonstrate the effectiveness of the CC-based MPC.

Many studies have been conducted to achieve natural interaction between robots and humans, and this area of research is becoming more important. Except for several works, however, most gesture recognition algorithms are designed with an assumption that the sensor does not move. To resolve this problem, we propose an algorithm that can robustly recognize gestures while the sensor is moving.

Nowadays, as the building structures are getting taller and taller, the importance of maintenance or inspection of these structures is being increased. However, it has some problems due to the lack of professional manpower and there is a risk in maintaining those areas that are hard to reach, besides the high maintenance cost. The unmanned wall-climbing robots for the areas hard to reach have been researched to solve the problems. The infrastructure-based wall-climbing robots have high payload and safety but the robots need the infrastructure that should be installed on the target structure. The infrastructure is not preferred by the architects since it can be harmful to the exterior of the structure. For this reason, wall-climbing robots that do not need any infrastructure are being researched. Nevertheless, most of the non-infrastructure-based wall-climbing robots are in the laboratory level since the payload, safety and maneuverability are not satisfactory. To overcome these problems, a flight-possible wall-climbing robot mechanism is proposed in this paper. The robot is based on the quadrotor system that is a well-known aerial vehicle using four rotors. It uses thrust forces induced by the four rotors not only to fly but also to stick on the wall. The flight capability makes its maneuverability and safety greatly enhanced. The feasibility of the mechanism is shown through simulations and experiments with a prototype.

Existing solutions for dead reckoning cannot provide accurate positioning when a robot suffers from changing dynamics such as wheel slip. In this paper, we propose a fuzzy-logic-assisted interacting multiple model (FLAIMM) framework to detect and compensate for wheel slip. We designed two different types of extended Kalman filter (EKF) to consider both no-slip and slip dynamics of mobile robots. Then a fuzzy inference system (FIS) model for slip estimation is constructed using adaptive neuro-fuzzy inference system (ANFIS). The trained model is utilized along with the two EKFs in the FLAIMM framework. The approach is evaluated using real data sets acquired with a robot driving in an indoor environment. The experimental results show that our approach improves position accuracy compared to the conventional multiple model approach.

Most path planning algorithms for a marine surface vehicle in the ocean environment have been developed only for a 2-D (x;y) plane without considering the vehicle heading angle. As a result, the vehicle has a difficulty in following the path correctly. In this paper, we propose a limit-cycle circle set that applies to the Theta* algorithm. The minimum turning radius of a marine surface vehicle is calculated using a limit-cycle circle set, and circles of this radius is used to generate a configuration space of an occupancy grid map. After applying Theta* to this configuration space, the limit-cycle circle set is also applied to the start and end nodes to find the appropriate path with specified heading angles. The benefit of this algorithm is its fast computation time compared to other 3-D (x;y;θ) path planning algorithms, along with the fact that it can be applied to the 3-D kinematic state of the vehicle. We simulate the proposed algorithm and compare it with 3-D A* and 3-D A* with post smoothing algorithms.

This paper presents a novel interval type-2 fuzzy logic control architecture for flocking system when the system has noisy sensor measurements. The traditional type-1 fuzzy logic controller (FLC) using precise type-1 fuzzy sets cannot fully model and handle the uncertainties of sensor data. However, type-2 FLC using type-2 fuzzy sets with a footprint of uncertainty (FOU) produces better performances under noisy environments. In this paper, therefore, we present a reactive control architecture for flocking algorithm that is based on interval type 2 FLC to implement the flocking behaviors consisting separation, obstacle avoidance, and velocity matching behaviors. The type-2 based control system could cope with the uncertainties of noisy sensor measurements and resulted in good performances that outperformed the type-1 FLC.

User localization is one of the key technologies for mobile robots to successfully interact with humans. Among various localization methods using radio frequency (RF) signals, time of arrival (TOA) based localization is popular since the target coordinates can be directly calculated from the accurate range measurements. In complex indoor environment, however, RF ranging-based localization is quite challenging since the range measurements suffer not only from signal noise but also from signal blockages and reflections. A set of range measurements taken in complex indoor environment verifies that almost all measurements are non-line-of-sight (NLOS) ranges which have striking difference to the line-of-sight (LOS) distances. These NLOS range measurements make severe degradation in the accuracy of trilateration based localizations if used without any compensation. In this paper we propose a particle filter-based localization algorithm which utilizes indoor geometry from a given map to estimate the NLOS signal path and compensates for the range measurements. The algorithm is verified with experiments performed in real indoor environments.

The odometry information used in localization can be quite erroneous when the robot follows the curved path or suffers from slippage. Thus the use of the low-cost gyroscope to compensate for an angular error is considered by many researchers. Conventional Kalman filtering methods that fuse the odometry with the gyroscope may produce infeasible solution because the robot parameters are estimated regardless of their physical constraints. In this paper, we propose a constrained Kalman filtering method that applies general constrained optimization technique to the estimation of the robot parameters. The state observability is improved by the additional state variables and the accuracy is also improved by the nonapproximated Kalman filter design. Experimental results show the proposed method effectively compensates for the odometry error and yields feasible parameter estimation at the same time

An effective human-robot interaction is essential for wide penetration of mobile home robots into the market. The ability to track human in the home environment is one of fundamental functions of home robots. In this paper, we introduce a prototype of Samsung home robot (SHR) developed by Samsung Advanced Institute of Technology (SAIT) and present a scheme of tracking user movement utilizing a structured light range sensor. The structured light range sensor is considered to be cost-effective and robust for a wide range of environment; however, its low resolution of range data tends to extract wrong features for the human tracking, such as table legs, trash cans, door-edges, etc. To compromise this weakness, we have developed an algorithm that excludes those features that do not come from human legs. Efficacy and robustness of this proposed system were examined through experiments in the real home environment.

Two evolutionary programming (EP) methods are proposed for handling nonlinear constrained optimization problems. The first, a hybrid EP, is useful when addressing heavily constrained optimization problems both in terms of computational efficiency and solution accuracy. But this method offers an exact solution only if both the mathematical form of the objective function to be minimized/maximized and its gradient are known. The second method, a two-phase EP (TPEP) removes these restrictions. The first phase uses the standard EP, while an EP formulation of the augmented Lagrangian method is employed in the second phase. Through the use of Lagrange multipliers and by gradually placing emphasis on violated constraints in the objective function whenever the best solution does not fulfill the constraints, the trial solutions are driven to the optimal point where all constraints are satisfied. Simulations indicate that the TPEP achieves an exact global solution without gradient information, with less computation time than the other optimization methods studied here, for general constrained optimization problems.

In this paper, a time-varying two-phase (TVTP) optimization neural network is proposed based on the two-phase neural network and the time-varying programming neural network. The proposed TVTP algorithm gives exact feasible solutions with a finite penalty parameter when the problem is a constrained time-varying optimization. It can be applied to system identification and control where it has some constraints on weights in the learning of the neural network. To demonstrate its effectiveness and applicability, the proposed algorithm is applied to the learning of a neo-fuzzy neuron model.