For continuous systems modeled by dynamical equations such as ODEs and SDEs, Bellman's principle of optimality takes the form of the Hamilton-Jacobi-Bellman (HJB) equation, which provides the theoretical target of reinforcement learning (RL). Although recent advances in RL successfully leverage this formulation, the existing methods typically assume the underlying dynamics are known a priori because they need explicit access to the drift and diffusion coefficients to update the value function following the HJB equation. We address this inherent limitation of HJB-based RL; we propose a model-free approach still targeting the HJB equation and the corresponding temporal difference method. We prove exponential stability of the induced continuous-time dynamics, and we empirically demonstrate the resulting advantages over transition–kernel–based formulations. The proposed formulation paves the way toward bridging stochastic control and model-free reinforcement learning.

Hybrid modeling is an approach that integrates data-driven models with physics-based models to compensate for discrepancies between model predictions and real-world behaviors, enabling us to obtain accurate dynamical system models. Previous studies have pointed out that in the training phase, applying constraints on the data-driven parts, called regularizers, is necessary when the physics models include unidentified parameters. However, the appropriate formulation of such regularizers remains unclear. In this work, we conducted experiments to investigate effective ways of introducing regularizers. We found that we can strike the balance between parameter estimation and state prediction by utilizing correlational information between physics and whole models for regularization. Furthermore, we extended the hybrid modeling approach to stochastic differential equations (SDEs), proposing a novel SDE learning algorithm using the probability distributions for the regularizers. Our experimental results showed that the proposed approach improved parameter estimation accuracy and enabled us to acquire correct stochastic dynamics.

Fault diagnosis is still a challenging task especially for unseen faults, which could happen in the systems. Zero-sample fault diagnosis alleviates this issue by utilizing information from seen faults and attributes defined by domain knowledge. However, existing methods suffer from irrelevant features in the original space while existing supervised feature selection methods yield unsatisfactory performance due to discrepancy caused by domain difference. In this article, we propose a novel feature selection method for zero-sample fault diagnosis, called concrete partial autoencoder. The concrete partial autoencoder selects features beneficial for both seen and unseen faults through striking a balance between classification accuracy and reconstruction errors of selected features. The concrete partial autoencoder utilizes categorical reparameterization to efficiently solve the feature selection problem. The evaluation results on the Tennessee Eastman Process show that the proposed method improves classification accuracy and robustness against irrelevant features at zero-sample fault diagnosis.

Remaining useful life (RUL) estimation is a crucial enabler of predictive maintenance. Since an adequate amount of labeled data is unavailable due to a low frequency of failures, semisupervised RUL estimation methods have been proposed to utilize unlabeled data collected in maintenance intervals to improve performance. However, those methods have limited capability since they only partially utilize unlabeled data during training. This article proposes a novel semisupervised method for RUL estimation in which both pretext and auxiliary tasks aim to estimate relative RULs. It draws more unlabeled samples from shorter RULs with a geometric progression and applies different strategies for pairing unlabeled samples at different training stages. The proposed method is evaluated on NASA C-MAPSS dataset and Backblaze HDD dataset. The results show that the proposed method significantly improves the performance with scarce labeled data for training.

Several methods of semantic segmentation using light detection and ranging (LiDAR) sensors have been proposed for the recognition of surrounding objects by autonomous driving cars. LiDAR is a sensor that compensates for the weaknesses of other sensors, such as cameras or radar systems, and semantic segmentation assigns a class label to each point in the LiDAR point cloud. Recently, real-time semantic segmentation methods that are capable of processing LiDAR point clouds at frame rates have been proposed. Real-time semantic segmentation is essential for the autonomous driving system because it can output class labels for LiDAR point clouds at high speeds. However, this segmentation method suffers from a delay equal to processing time. To address this challenge, we propose a novel method that combines SalsaNext [1], a method of real-time LiDAR semantic segmentation, and semantic forecasting, which predicts the results of future semantic segmentation. We quantitatively evaluate our method using the Semantic-KITTI dataset, which comprises point cloud data acquired from the LiDAR sensor in the real world, and compare the latency and accuracy of our method with other semantic segmentation methods. Consequently, our method is found to be capable of operating in real-time and with low-latency, and it can achieve a performance similar to that of previously reported real-time semantic segmentation methods.

Three-dimensional (3D) human pose estimation is one of the most basic tasks for human-interacting robots. Especially in weightless environments such as the International Space Station (ISS), wherein objects may move with a higher degree of freedom compared to on the ground, a camera with a wider field of view (FOV) is crucial in improving the probability of capturing surrounding humans. To this end, we propose a learning-based 3D human pose estimation (3D-HPE) using a fisheye camera targeted at weightless environments. One impediment is that 3D-HPE trained on existing datasets are incapable of addressing the adverse effects of strong fisheye distortion and weightlessness as existing human detection and pose estimation datasets are recorded on the ground using typical rectilinear cameras. To overcome this difficulty, we integrate virtual camera projection to perform a detected human-centered undistortion from fisheye to rectilinear images. We also include upside-down augmentation during training to improve the performance toward weightlessness. Our results show that these two techniques successfully mitigate the adverse effects of weightlessness and fisheye distortion.

This paper proposes a method of searching for missing people in mountains by UAVs (Unmanned Aerial Vehicles) based on beacon signals. This method alternately updates the distribution of estimated target position and unknown parameters by particle filtering, and determines the next best observation location based on the idea of the uncertainty sampling. We will show how this method can be integrated with both of the range-based and range-free localization algorithms, and demonstrate the efficiency of it in a simulation study.

Dynamic mode decomposition (DMD) is a method to extract coherent modes from nonlinear dynamical systems. In this paper, we propose an extension of DMD, sparse nonnegative DMD, which generates a nonlinear and sparse modal representation of dynamics. In particular, this makes DMD more suitable for video processing. We reformulate DMD as a block-multiconvex optimization problem to impose constraints and regularizations directly on the structures of the estimated dynamic modes. We introduce the results of experiments with synthetic data and a surveillance video dataset and show that sparse nonnegative DMD can extract part-based dynamic modes from video streams.

Dynamic mode decomposition (DMD) is a data-driven method for calculating a modal representation of a nonlinear dynamical system, and it has been utilized in various fields of science and engineering. In this paper, we propose Bayesian DMD, which provides a principled way to transfer the advantages of the Bayesian formulation into DMD. To this end, we first develop a probabilistic model corresponding to DMD, and then, provide the Gibbs sampler for the posterior inference in Bayesian DMD. Moreover, as a specific example, we discuss the case of using a sparsity-promoting prior for an automatic determination of the number of dynamic modes. We investigate the empirical performance of Bayesian DMD using synthetic and real-world datasets.

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynamics, which is not always possible since we may not have any a priori knowledge about them. In this paper, we propose a fully data-driven method for Koopman spectral analysis based on the principle of learning Koopman invariant subspaces from observed data. To this end, we propose minimization of the residual sum of squares of linear least-squares regression to estimate a set of functions that transforms data into a form in which the linear regression fits well. We introduce an implementation with neural networks and evaluate performance empirically using nonlinear dynamical systems and applications.

In an asteroid exploration and sample return mission, accurate estimation of the shape and motion of the target asteroid is essential for selecting a touchdown site and navigating a spacecraft during touchdown operation. In this work, we present an automatic estimation method for the shape and motion of an asteroid, which is planned to be tested in future exploration missions including Japanese Hayabusa-2 [1]. Our task is to estimate the shape and rotation axis of the asteroid, as well as positions of the spacecraft from optical images. The proposed method is based on the expectation conditional-maximization (ECM) framework that consists of an auxiliary particle filter and nonlinear optimization techniques. One of our technical contributions is the estimation of the direction of rotation axis of the asteroid from monocular camera images, which are taken by the moving spacecraft. We conducted two experiments with synthetic data and an asteroid mock-up to show the validity of the proposed method and to present the numerical accuracy.

Over the last decade, face alignment researches have been advancing rapidly and have vast applications related to face recognition, pose estimation, and human-robot interaction. Face alignment is typically performed in a two-stage fashion by alternatively using local landmark detector and global shape regularizer. While both landmark detector and shape regularizer have achieved impressive progress recently, shape initialization with mean shape remains a critical issue. In this paper, we present a unique sparse initialization method that is inspired by the sparse model. Experiment results with two datasets selected from the widely used Multi-PIE Database show that our method outperforms conventional initialization techniques. In addition to its capability to handle test data with high shape variability and potential occlusion, our method has merit of simplicity and can be easily integrated with other face alignment approaches.

Most literatures have been relying on image processing approaches such as skin detection and depth thresholding for hand detection. These techniques are restricted by strong assumptions and normally possess low robustness in actual applications. In this paper, we focus on an appearance approach and propose a new feature extraction method based on sparse pixel-pair wise intensity comparisons for hand detection. Our method can be viewed as a generalized BRIEF descriptor and can be easily adopted for other object detection or recognition tasks. We perform extensive experiments and prove that our method achieves comparable results with normal, noisy, and occluded hand images in term of both test accuracy and ROC. The main contributions of our work are threefold: 1) We introduce a new and simple feature extraction method that is robust against image noise, cluttered backgrounds, and partial occlusion. 2) Combined with AdaBoost, we show that the new feature descriptor is effective for hand detection. 3) The new feature descriptor has been rigorously compared with existing feature descriptors with a new hand database that has very challenging image backgrounds.

Anomaly detection from sensor data is an important data mining application for efficient and secure operation of complicated systems. In this study, we propose a novel anomaly detection method for multivariate time-series to capture relationships of variables and time-domain correlations simultaneously, without assuming any generative models of signals. The supposed framework in this study is a semi-supervised anomaly detection where we seek unusual parts of test data compared with reference data. The proposed method is based on feature extraction with sparse representation and relationship learning with dimensionality reduction. Our idea comes from the similarity between a sparse feature matrix extracted from multivariate time-series and a term-document matrix. We conducted experiments with synthetic and simulated data, and confirmed that the proposed method successfully detected anomalies in multivariate time-series signals. Especially, it demonstrated superior performance with anomalies in which only relationships of time-series patterns are changed from reference data (multivariate anomalies).

Coupled nonnegative matrix factorization (CNMF) unmixing is proposed for the fusion of low-spatial-resolution hyperspectral and high-spatial-resolution multispectral data to produce fused data with high spatial and spectral resolutions. Both hyperspectral and multispectral data are alternately unmixed into end member and abundance matrices by the CNMF algorithm based on a linear spectral mixture model. Sensor observation models that relate the two data are built into the initialization matrix of each NMF unmixing procedure. This algorithm is physically straightforward and easy to implement owing to its simple update rules. Simulations with various image data sets demonstrate that the CNMF algorithm can produce high-quality fused data both in terms of spatial and spectral domains, which contributes to the accurate identification and classification of materials observed at a high spatial resolution.

Coupled non-negative matrix factorization (CNMF) is introduced for hyperspectral and multispectral data fusion. The CNMF fused data have little spectral distortion while enhancing spatial resolution of all hyperspectral band images owing to its unmixing based algorithm. CNMF is applied to the synthetic dataset generated from real airborne hyperspectral data taken over pasture area. The spectral quality of fused data is evaluated by the classification accuracy of pasture types. The experiment result shows that CNMF enables accurate identification and classification of observed materials at fine spatial resolution.

Learning dynamical systems is one of the important problems in many fields. In this paper, we present an algorithm for learning non-linear dynamical systems which works by aligning local linear models, based on a probabilistic formulation of subspace identification. Because the procedure for constructing a state sequence in subspace identification can be interpreted as the CCA between past and future observation sequences, we can derive a latent variable representation for this problem. Therefore, as in a similar manner to the recent works on learning a mixture of probabilistic models, we obtain a framework for constructing a state space by aligning local linear coordinates. This leads to a prominent algorithm for learning non-linear dynamical systems. Finally, we apply our method to motion capture data and show how our algorithm works well.

This paper proposes an alternative solution to a mapping problem in two different cases; when bearing measurements to features (landmarks) and odometry are measured and when bearing and range measurements to features are measured. Our approach named

In this paper, we introduce an alternative solution to the Bearing-only Mapping problem in which a mobile robot builds a map of features (landmarks) using only relative bearing measurements to them and odometry information. Our approach named BOM-STMDS (Bearing-Only Mapping by Sequential Triangulation and Multi-Dimensional Scaling) first tries to estimate relative distances among the features, then finds two-dimensional coordinates of all features by using multi-dimensional scaling (MDS) and its enhancements. BOMSTMDS is different from the conventional BOSLAM methods based on Bayesian filtering in that robot self-localization is not mandatory. Another remarkable property is that BOM-STMDS is able to utilize prior information about relative distances among features efficiently. In the experiment, the performance of BOM-STMDS is shown to be competitive with a conventional EKF-based BOSLAM method.

A new method of shape modeling for robotic applications is proposed. We utilize visual ID-tags to acquire local tangent planes of objects, and generate approximate polyhedral models of the environment. We developed an optimization algorithm for estimating tag configurations and a method of forming polyhedral models of objects. Our approach enables easy modeling using a single camera without pose information. The shape models are superimposed on the real environment, and they are used in many robotic tasks. We examined the performance of the modeling method, and demonstrated the effectiveness of the models in robotic applications.

In this paper, we propose series of algorithms for detecting change points in time-series data based on subspace identification, meaning a geometric approach for estimating linear state-space models behind time-series data. Our algorithms are derived from the principle that the subspace spanned by the columns of an observability matrix and the one spanned by the subsequences of time-series data are approximately equivalent. In this paper, we derive a batch-type algorithm applicable to ordinary time-series data, i.e. consisting of only output series, and then introduce the online version of the algorithm and the extension to be available with input-output time-series data. We illustrate the effectiveness of our algorithms with comparative experiments using some artificial and real datasets.

This paper proposes a new map building framework for mobile robot named Localization-Free Mapping by Dimensionality Reduction (LFMDR). In this framework, the robot map building is interpreted as a problem of reconstructing the 2-D coordinates of objects so that they maximally preserve the local proximity of the objects in the space of robot's observation history. Not only traditional linear PCA but also recent manifold learning techniques can be used for solving this problem. In contrast to the SLAM framework, LFMDR framework does not require localization procedures nor explicit measurement and motion models. In the latter part of this paper, we will demonstrate "visibility-only" and "bearing-only" localization-free mappings which are derived by applying LFMDR framework to the visibility and bearing measurements respectively.

We propose a robot control system utilizing semantic information behind robot-plans. Robots read the semantic information annotated to control programs, and utilize it for proper situation-recognition and rational action-generation. We try to develop a framework based on reusable and sharable representations among robots and humans. To this end, we construct an ontology of task world defining a systematized vocabulary for task models and knowledge processing. We introduce several important ideas for modeling the task world. We demonstrate an example of execution monitoring in a simulated blocks world task. Semantic information serves as a guide for rational and reasonable inference, planning, and diagnosis

In this paper, we present a subspace method for learning nonlinear dynamical systems based on stochastic realization, in which state vectors are chosen using kernel canonical correlation analysis, and then state-space systems are identified through regression with the state vectors. We construct the theoretical underpinning and derive a concrete algorithm for nonlinear identification. The obtained algorithm needs no iterative optimization procedure and can be implemented on the basis of fast and reliable numerical schemes. The simulation result shows that our algorithm can express dynamics with a high degree of accuracy.

Development of advanced anomaly detection and failure diagnosis technologies for spacecraft is a quite significant issue in the space industry, because the space environment is harsh, distant and uncertain. While several modern approaches based on qualitative reasoning, expert systems, and probabilistic reasoning have been developed recently for this purpose, any of them has a common difficulty in obtaining accurate and complete a priori knowledge on the space systems from human experts. A reasonable alternative to this conventional anomaly detection method is to reuse a vast amount of telemetry data which is multi-dimensional time-series continuously produced from a number of system components in the spacecraft.This paper proposes a novel "knowledge-free" anomaly detection method for spacecraft based on Kernel Feature Space and directional distribution, which constructs a system behavior model from the past normal telemetry data from a set of telemetry data in normal operation and monitors the current system status by checking incoming data with the model.In this method, we regard anomaly phenomena as unexpected changes of causal associations in the spacecraft system, and hypothesize that the significant causal associations inside the system will appear in the form of principal component directions in a high-dimensional non-linear feature space which is constructed by a kernel function and a set of data.We have confirmed the effectiveness of the proposed anomaly detection method by applying it to the telemetry data obtained from a simulator of an orbital transfer vehicle designed to make a rendezvous maneuver with the International Space Station.

Autonomous map construction is one of the most fundamental and significant issues in intelligent mobile robot research. While a variety of map construction methods have been proposed, most require some quantitative measurements of the environment and a mechanism of precise self-localization. This paper proposes a novel map construction method using only qualitative information about "how often two objects are observed simultaneously." This method is based on heuristics--"closely located objects are likely to be seen simultaneously more often than distant objects" and a well-known multivariate data analysis technique-multidimensional scaling. A significant feature of this method is that it requires neither quantitative sensor measurements nor information about the robot's own position. Simulation and experimental results demonstrated that this method is sufficiently practical for capturing a qualitative spatial relationship among identifiable landmark objects rapidly.

This paper proposes a new map construction method based on rough information of "how often two objects are observed simultaneously". This method is founded on the combination of a simple and general heuristics "closely located objects are likely to be seen simultaneously more often than distant objects", and a well-known technique in the multivariate data analysis - multidimensional scaling. A significant feature of this method is that it requires little quantitative models nor precise sensor information, unlike conventional map construction methods. Simulation and experiment results suggest that this method is sufficiently practical for grasping a topological configuration of identifiable landmarks quickly.

When an autonomous robot is to learn its behavior, whether an appropriate state space is available or not is a critical issue for the flexibility and efficiency of the learning process. What is problematic is that it is usually very difficult to prepare such an ideal state space manually beforehand. We propose a new state space "reconstruction" method. With this, behavior-based robots can autonomously "rebuild" their state spaces after they accumulate behavior experience using initial state spaces. This reconstruction approach is more advantageous than the conventional state space construction methods or incremental state partitioning methods in that it achieves both the efficiency in the learning process and the optimality of the resultant behavior performance.