LiDAR directly acquires 3D point measurements of physical surfaces, making it well-suited for interpreting objects in tasks such as object reconstruction and semantic mapping. However, each object is represented by a sparse point set, and the observations are inherently partial due to self-occlusion, which poses significant challenges for object registration. The uncertainty of the modeled surface depends on the local point density, and outliers from non-overlapping areas are often involved in the registration process. To address these issues, we propose a stochastic object registration method that models surface uncertainty using a Gaussian process and an overlap determination scheme based on object detector outputs. Both the overlap determination scheme and the stochastic object registration method are verified through Monte Carlo simulations on numerous real-world urban object point clouds, demonstrating performance improvements of 52% and 12%, respectively, compared to the baseline methods. To validate the full pipeline in real-world conditions, the proposed registration method is integrated with PointPillars, achieving an 8% accuracy improvement over the baseline method and demonstrating a 98.8% convergence ratio in LiDAR-based vehicle registration.

In this study, we propose an object shape alignment method through a robust optimization scheme for 6-degrees-of-freedom (DOF) object pose compensation. Although the pose estimation of the 3D object by the camera has been rapidly improved in recent years with the development of deep learning, the estimate still contains errors due to several factors. To compensate for this, we perform a shape alignment between the 2D segmentation of the object and the projection of the 3D object in the image plane. To avoid convergence to a local minimum in nonlinear optimization, we separate the pose into translation and rotation. This approach derives the optimization of a linear form in terms of a translation with reduced computational cost. For the rotation, the parallel optimization is performed with multiple initial values, reflecting to the uncertainty of an initial value. We formulate an invariant extended Kalman filter (EKF)-based object-visual simultaneous localization and mapping (SLAM) with a camera-object relative pose as the measurement model. To verify the performance of the proposed algorithm, we present the improved results of camera-object relative pose accuracy and localization and mapping accuracy in the several sequences of YCB-video dataset.

As the use of smartwatches continues to grow, improving the accuracy of tracking for smartwatch users has become increasingly important. Smartwatches are worn on the wrist, which means the sensors may detect free and independent hand movements while walking. This creates challenges for pedestrian dead reckoning (PDR) using the inertial sensors embedded in smartwatches because the data may lose information on walking characteristics. Therefore, it is essential to identify the user’s walking state first, whether moving or stationary, and then conduct precise step detection, which is a critical factor in reducing errors in PDR. In this study, we propose an adaptive step detection algorithm tailored to four categorized user activity states: walking, running, hand movements while standing, and hand movements while walking. We conducted human activity recognition (HAR) with these four categories to identify the user’s state and decide the appropriate method for step detection. To enable implementation on mobile devices, we propose a lightweight deep learning-based HAR network utilizing a grouped convolutional layer. The proposed network reduces the number of trainable parameters by approximately one-third compared to conventional CNN structures while achieving $\mathbf{9 6. 4 5 \%}$ accuracy in activity classification. For adaptive step detection, we utilize gyroscope data for normal walking, accelerometer data for running, and walking frequency obtained via fast Fourier transform (FFT) from normal walking data for segments of walking interspersed with hand movements. By applying these tailored step detection methods to each walking scenario, we enhance step detection accuracy, thereby improving smartwatch user position and distance estimation in overall PDR.

With recent advances in artificial intelligence, deep learning networks with various structures are being applied to target detection. However, in the infrared small target detection (IRSTD) field, an appropriate network structure is still required to identify blurry and low-contrast targets accurately. Conventional U-net algorithms use skip connection and attention module because information loss occurs as the convolution layer becomes deeper. However, the problem of not being able to recognize the entire image due to the limitations of convolution is fatal in IRSTD. To overcome these limitations, we design an encoder that divides the image into multiple small pieces and stacks them hierarchically to extract features. Therefore, the proposed network has a hybrid encoder structure that combines a convolution-based multiscale encoder to extract local information and a hierarchical-based multipatch encoder to extract global information by running in parallel. To effectively fuse the hierarchical information obtained from each layer, the loss function, which plays an important role in learning, has also been changed to suit the hybrid encoder. The proposed algorithm, named hierarchical multipatch feature and attention multiscale feature fusion U-net (HMAMFU-net), can guarantee effective target detection performance with two datasets: NUDT-SIRST and NUAA-SIRST, and performance analysis is conducted with other state-of-the-art algorithms. PyTorch implementation is available athttps://github.com/skylih87/HMAMFU-net.

Event cameras are asynchronous neuromorphic vision sensors with high temporal resolution and no motion blur, offering advantages over standard frame-based cameras especially in high-speed motions and high dynamic range conditions. However, event cameras are unable to capture the overall context of the scene, and produce different events for the same scenery depending on the direction of the motion, creating a challenge in data association. Standard camera, on the other hand, provides frames at a fixed rate that are independent of the motion direction, and are rich in context. In this paper, we present a robust feature tracking method that employs 8-DOF warping model in minimizing the difference between brightness increment patches from events and frames, exploiting the complementary nature of the two data types. Unlike previous works, the proposed method enables tracking of features under complex motions accompanying distortions. Extensive quantitative evaluation over publicly available datasets was performed where our method shows an improvement over state-of-the-art methods in robustness with greatly prolonged feature age and in accuracy for challenging scenarios.

In this paper, we analyze the effect of nonorthogonality in a dual-axis gimbal constituting a rotational inertial navigation system (RINS) on the navigation performance. The RINS alleviates navigation errors caused by sensor errors through periodic attitude modulations. Hence, it is robust to the GNSS-denied environments. However, the rotations themselves cause new error factors, where the most common one being the non-orthogonality between the gimbal rotation axes. Therefore, in this paper, the navigation error additionally generated by the non-orthogonality of the dual-axis gimbal during the rotational motion of the RINS is analyzed. The results show that the non-orthogonality directly affects the N-axis attitude error, and hence consequentially the E-axis velocity and position error as well.

In order to improve detection performance in a U-net-based IR small target detection (IRSTD) algorithm, it is crucial to fuse low-level and high-level features. Conventional algorithms perform feature fusion by adding a convolution layer to the skip pathway of the U-net and by connecting the skip connection densely. However, with the added convolution operation, the number of parameters of the network increases, hence the inference time increases accordingly. Therefore, in this letter, a UNet3+ based full-scale skip connection U-net is used as a base network to lower the computational cost by fusing the feature with a small number of parameters. Moreover, we propose an effective encoder and decoder structure for improved IRSTD performance. A residual attention block is applied to each layer of the encoder for effective feature extraction. As for the decoder, a residual attention block is applied to the feature fusion section to effectively fuse the hierarchical information obtained from each layer. In addition, learning is performed through full-scale deep supervision to reflect all the information obtained from each layer. The proposed algorithm, coined Attention Multiscale feature Fusion U-net (AMFU-net), can hence guarantee effective target detection performance and a lightweight structure (mIoU: 0.7512, FPS: 86.1). Pytorch implementation is available at: github.com/cwon789/AMFU-net.

This paper addresses the challenges associated with interpreting synthetic aperture radar (SAR) images, which provide limited visual information compared to optical images. We propose a new method for generating virtual data and a SAR-to-optical image translation neural network to recognize targets in SAR images. The creation of virtual data involves classifying it into targets and backgrounds. The target data generates a virtual optical image using a 3D model, while the background data is synthesized by combining the target with the existing SAR image. For image translation using the virtual dataset, we design a modified dense nested U-net that converts images for target recognition. By incorporating the proposed translation network into the YOLO v4 detection algorithm, we verify the impact of virtual optical images on target recognition. The experimental results demonstrate that our proposed method outperforms the conventional approach, which relies solely on SAR target image data for learning.

In infrared search and tracking (IRST) systems, small target detection is challenging because IR imaging lacks feature information and has a low signal-to-noise ratio. The recently studied small IR target detection methods have achieved high detection performance without considering execution time. We propose a fast and robust single-frame IR small target detection algorithm while maintaining excellent detection performance. The augmented infrared intensity map based on the standard deviation speeds up small target detection and improves detection accuracy. Density-based clustering helps to detect the shape of objects and makes it easy to identify centroid points. By incorporating these two approaches, the proposed method has a novel approach to the small target detection algorithm. We have self-built 300 images with various scenes and experimented with comparing other methods. Experimental results demonstrate that the proposed method is suitable for real-time detection and effective even when the target size is as small as 2 pixels.

In this paper, unscented Kalman filter (UKF) based on maximum correntropy criterion (MCC) instead of minimum mean square error (MMSE) criterion, and it is applied to tightly coupled integration of inertial navigation system (INS) and ultra wide-band (UWB). UWB can measure distance with an accuracy of less than 30cm in line-of-sight environment, but provides distance measurement with various types of non-Gaussian uncertainty noise in non-line-of-sight environment. In this case, if the INS/UWB system is configured with the existing MMSE-based filter, a large error occurs. To solve this problem, in this paper, UKF is designed based on MCC. Through simulation analysis, it is confirmed that the proposed filter has robust characteristics against UWB uncertainty and enables stable INS/UWB integration.

In the presence of fast motion, point clouds obtained from mechanical spinning LiDAR can be easily distorted due to the slow scanning speed of the LiDAR. Existing LiDAR-only odometry algorithms generally ignore this distortion or compensate by linearly interpolating the estimated relative motion between scans. However, when there are abrupt and nonlinear motion changes, the linear interpolation method poorly compensates for the distortions, which can cause significant drift in motion estimates. In this work, we present a LiDAR-only odometry algorithm that estimates motion by slicing LiDAR scans into shorter times to compensate more agilely for point cloud distortions. Observations from only one small scan slice inevitably lack spatial uniqueness, so the multimodal problem needs to be addressed. For LiDAR-only odometry with small scan slices, we introduce the ensemble Kalman filter, a kind of Monte Carlo-based Bayesian filter. The proposed method makes it possible to perform odometry with only a very narrow field of view (FoV), and the robustness to point cloud distortion is improved. We demonstrate the effectiveness of the proposed method through Monte Carlo simulations and several tests with fast-moving scenarios. The experimental results prove the possibility of odometry with a very narrow FoV of down to 10 degrees and robustness against motion distortion.

In this paper, we propose a novel object-based visual-inertial navigation system fully embedded in a matrix Lie group and built upon the invariant Kalman filtering theory. Specifically, we focus on relative pose measurements of objects and derive an error equation at the associated tangent space. We prove that the observability property does not suffer from the filter inconsistency and nonlinear error terms are identically zero at the object initialization. A thorough Monte-Carlo simulation reveals that our approach yields consistent estimates and is very robust to a large initial state uncertainty. Further-more, we demonstrate a real-world application to the KITTI dataset with a deep neural network-based 3D object detector. Experimental results report that noises on pose measurements follow a Gaussian-like density matching our assumption. The proposed method improves the localization and object global mapping accuracy by probabilistically accounting for inertial readings and object pose uncertainties at multiple views.

In this article, we propose a new methodology to fuse visual-inertial measurements for land vehicles in a challenging urban environment in which a GNSS signal is not available nor reliable. Motivated by a degenerate case caused by a large bias of a MEMS IMU, we redesign a system model of visual-inertial odometry in a framework of extended Kalman filter. In particular, the system model is propagated through a reduced inertial sensor system composed of a 3-axis gyroscope, a 2-axis accelerometer, and a single-axis odometer. An analytical observability derivation reveals unobservable bases of our estimator, and these directions are resolved by using intermittent position measurements from a GNSS receiver. Furthermore, we inspect the uncertainties of the state vector in a Monte-Carlo simulation that agrees with our theoretical results. The proposed method is validated through the KITTI benchmark dataset and an extensive field testing showing a position drift as 1.25% in tunnels on average and a mean position error of 2.81m in the street canyon over a 6.7km driving.

In this article, we propose a visual-inertial navigation system that directly minimizes a photometric error without an explicit data-association. We focus on the photometric error parametrized by pose and structure parameters that is highly nonconvex due to the nonlinearity of image intensity. The key idea is to introduce anoptimal intensity gradientthat accounts for a projective uncertainty of a pixel. Ensembles sampled from the state uncertainty contribute to the proposed gradient and yield a correct update direction even in a bad initialization point. We present two sets of experiments to demonstrate the strengths of our framework. First, a thorough Monte Carlo simulation in a virtual trajectory is designed to reveal robustness to large initial uncertainty. Second, we show that the proposed framework can achieve superior estimation accuracy with efficient computation time over state-of-the-art visual-inertial fusion methods in a real-world UAV flight test, where most scenes are composed of a featureless floor.

In this paper, we propose a novel filtering-based method that fuses events from a dynamic vision sensor (DVS), images, and inertial measurements to estimate camera poses. A DVS is a bio-inspired sensor that generates events triggered by brightness changes. It can cover the drawbacks of a conventional camera by virtual of its independent pixels and high dynamic range. Specifically, we focus on optical flow obtained from both a stream of events and intensity images in which the former is much like a differential quantity, whereas the latter is a pixel difference in a much longer time interval than events. This nature characteristic motivates us to model optical flow estimated from events directly, but feature tracks for images in the filter design. An inequality constraint is considered in our method since the inverse scene-depth is larger than zero by its definition. Furthermore, we evaluate our proposed method in the benchmark DVS dataset and a dataset collected by the authors. The results reveal that the presented algorithm has reduced the position error by 49.9% on average and comparable accuracy only using events when compared to the state-of-the-art filtering-based estimator.

In this paper, we present a hybrid visual inertial navigation algorithm for an autonomous and intelligent vehicle that combines the multi-state constraint Kalman filter (MSCKF) with the nonlinear visual-inertial graph optimization. The MSCKF is a well-known visual inertial odometry (VIO) method that performs the fusion between an inertial measurement unit (IMU) and the image measurements within a sliding window. The MSCKF computes the re-projection errors from the camera measurements and the states in the sliding window. During this process, the structure-only estimation is performed without exploiting the full information over the window, like the relative interstate motion constraints and their uncertainties. The key contribution of this paper is combination of the filtering and non-linear optimization method for VIO, and the design of a novel measurement model that exploits all of the measurements and information available within the sliding window. The local visual-inertial optimization is performed using pre-integrated IMU measurements and camera measurements. It infers the probabilistically optimal relative pose constraints. These local optimal constraints are used to estimate the global states under the MSCKF framework. The proposed local-optimal-multi-state constraint Kalman filter is validated using a simulation data set, as well as publicly available real-world data sets generated from real-world urban driving experiments.

In this study, a simple but effective spoofing detection method using a global positioning system (GPS) directional antenna is proposed which exploits the difference between the estimated direction‐of‐arrival (DOA) and the measured DOA from a GPS almanac and ephemeris data. The receiving signal's DOA is estimated by using a single antenna power measurement‐based complex extended Kalman filter (EKF) which is a complex valued state space based estimation technique. Furthermore, an adaptive logic is applied to the complex EKF to reduce the effect of the measurement disturbance. To maintain the validity of the proposed algorithm, it is assumed that the spoofer is aware of the target's location, but that its DOA is not perfectly the same as that of the authentic GPS signal. In addition, the orientation of the directional antenna is known by using an attitude and heading reference system that is attached to the antenna, and its antenna radiation pattern is also known. The proposed detection method is evaluated using a theoretical analysis and simulations. It is finally confirmed that the proposed algorithm can detect a spoofing signal according to the different direction angles of the spoofing signal, and especially those with low DOAs.

Since synthetic aperture radar (SAR) images contain severe noise, it is important to extract noise excluded feature when recognizing the target in the SAR image. Therefore, previous SAR automatic target recognition (ATR) methods use separate preprocessing process or pose information of SAR to reduce the influence of noise. However, since noise characteristics of SAR images are different from image to image, recognition accuracy cannot be guaranteed if the preprocessing process is conducted improperly or there is no pose information. For this reason, we propose multiple feature-based convolutional neural networks (MFCNNs) recognizing the target of the SAR image without using a separate preprocessing process or pose information. MFCNN consists of three steps. First, extract strong features of the target with more effect of noise and smoothed features with lesser effect of noise. Second, aggregate extracted features with complementary relationships into a single column vector. Last, fully connected networks recognize the target using aggregated features. We used moving and stationary target acquisition and recognition SAR public data set for simulations and confirmed that the proposed method can recognize the target of the SAR image more accurately than previous SAR ATR methods without preprocessing process and pose information.

In this paper, we proposed Advanced Heuristic Drift Elimination (AHDE) which can remove azimuth drift error in indoor environments. In Pedestrian Dead Reckoning (PDR) system, azimuth error is one of the main factors that cause estimated position error. In order to reduce azimuth error, several methods are used. Heuristic Drift Elimination (HDE) algorithm proposed by Johann Borenstein shows great strength in indoor environments. HDE assumes that generally walls and corridors are straight and either parallel or orthogonal to each other in man-made building. They called the typical directions of walls and corridors as the dominant directions. HDE is corrected if the computed azimuth angle matches the closest dominant direction. HDE also has limitation when the pedestrian walks in various directions because HDE can cause a new azimuth error by matching the closed dominant direction. To overcome these limitations, we propose AHDE which is based on INS-EKF-ZUPT (IEZ) by using foot-mounted IMU. The algorithm consists with the following two steps. First, it determines whether a pedestrian is walking straight forward or not. If a pedestrian is not walking straight forward, the algorithm estimates the biases of accelerometers and gyroscopes by Zero velocity UPdaTe (ZUPT) method. However if the pedestrian is walking straight forward, the algorithm determines whether the pedestrian is walking along the dominant direction or not. When it is determined that pedestrian is walking along the dominant direction, the algorithm corrects the computed azimuth angle to the closest dominant direction. When it is determined that the pedestrian is not walking along the dominant direction but walking straight with no change in azimuth, AHDE applies a correction to the gyro output which contains the bias error. Experimental results show that the accuracy of AHDE is improved compared to HDE and the algorithm is a powerful method which can reduce the azimuth error in complex motion.

Target localization, whose goal is to estimate the location of an unknown target, is one of the key issues in applications of wireless sensor networks (WSNs). With recent advances in fabrication technology, deployments of large-scale WSNs have become economically feasible. However, there exist issues such as limited communication and the curse of dimensionality in applying machine-learning algorithms such as support vector regression (SVR) on large-scale WSNs. Here, in order to overcome such issues, we propose an ensemble implementation of SVR for the problem of target localization. The convergence property of the localization algorithm using the ensemble SVR is verified, and the robustness of the proposed scheme against measurement noise is analyzed. Furthermore, experimental results confirm that the estimation performance of the proposed method is more accurate and robust to measurement noise than the conventional SVR predictor.