Coarse-Grained Reconfigurable Architectures (CGRAs) have been the subject of extensive research due to their balance between performance, energy efficiency, and flexibility. CGRAs must be capable of executing a dataflow graph (DFG), which depends on a compiler producing quality valid mappings with feasible running time performance and portable mapping DFGs on different CGRA architectures. Machine learning-based compilers have shown promising results by presenting high quality and performance but offer limited portability. Moreover, some approaches do not explore efficient placement methods or do not demonstrate whether scaling to more challenging, less connected architectures. This paper presents SmartMap, an architecture-agnostic framework that uses an actor-critic reinforcement learning method applied to a Monte-Carlo Tree Search (MCTS) to learn how to map a DFG onto a CGRA. This framework offers full portability using a state-action representation layer in the policy network instead of a probability distribution over actions. SmartMap uses a graph traversal placement method to provide scalability and improve the efficiency of MCTS by enabling more efficient exploration during the search. Our results show that SmartMap has 2.81x more mapping capacity, a 16.82x speed-up in compilation time, and consumes fewer resources compared to the state-of-the-art.

Large foundation models have shown strong open-world generalization to complex problems in vision and language, but similar levels of generalization have yet to be achieved in robotics. One fundamental challenge is the lack of robotic data, which are typically obtained through expensive on-robot operation. A promising remedy is to leverage cheaper, *off-domain* data such as action-free videos, hand-drawn sketches, or simulation data. In this work, we posit that *hierarchical* vision-language-action (VLA) models can be more effective in utilizing off-domain data than standard monolithic VLA models that directly finetune vision-language models (VLMs) to predict actions. In particular, we study a class of hierarchical VLA models, where the high-level VLM is finetuned to produce a coarse 2D path indicating the desired robot end-effector trajectory given an RGB image and a task description. The intermediate 2D path prediction is then served as guidance to the low-level, 3D-aware control policy capable of precise manipulation. Doing so alleviates the high-level VLM from fine-grained action prediction, while reducing the low-level policy's burden on complex task-level reasoning. We show that, with the hierarchical design, the high-level VLM can transfer across significant domain gaps between the off-domain finetuning data and real-robot testing scenarios, including differences in embodiments, dynamics, visual appearances, and task semantics, etc. In the real-robot experiments, we observe an average of 20% improvement in success rate across seven different axes of generalization over OpenVLA, representing a 50% relative gain. Visual results are provided at: [https://hamster-robot.github.io/](https://hamster-robot.github.io/)

We present a novel formulation of ergodic trajectory optimization that can be specified over general domains using kernel maximum mean discrepancy. Ergodic trajectory optimization is an effective approach that generates coverage paths for problems related to robotic inspection, information gathering problems, and search and rescue. These optimization schemes compel the robot to spend time in a region proportional to the expected utility of visiting that region. Current methods for ergodic trajectory optimization rely on domain-specific knowledge, e.g., a defined utility map, and well-defined spatial basis functions to produce ergodic trajectories. Here, we present a generalization of ergodic trajectory optimization based on maximum mean discrepancy that requires only samples from the search domain. We demonstrate the ability of our approach to produce coverage trajectories on a variety of problem domains including robotic inspection of objects with differential kinematics constraints and on Lie groups without having access to domain specific knowledge. Furthermore, we show favorable computational scaling compared to existing state-of-the-art methods for ergodic trajectory optimization with a trade-off between domain specific knowledge and computational scaling, thus extending the versatility of ergodic coverage on a wider application domain.

Acquiring prior knowledge of trajectory distributions in specific environments can significantly expedite the optimisation process in robot motion planning. Leveraging successful past plans and utilising trajectory generative models as priors offers a clear advantage. Previous studies have proposed various methods to harness these priors, such as using prior samples for initialisation or incorporating the prior distribution into trajectory optimisation through inference. Recently, diffusion models have demonstrated effectiveness in encoding multi-modal data in high-dimensional settings. In this study, we introduce a methodology that integrates Stein Variational Gradient Descent (SVGD) with Gaussian Process Motion Planning (GPMP), leveraging diffusion models as multi-modal priors. This approach combines the advantages of deep generative model and Bayesian inference to reduce the computation time required to approximate the posterior distribution of trajectories, particularly when adapting to new, unseen environments. In addition, we incorporate path signatures into our method to enhance the diversity of the posterior distribution, thereby improving the optimality of trajectories in multi-modal settings. To validate our approach, we conduct comparative assessments against multiple baseline methods across various scenarios, including 2D planar robots and robotic manipulators.

Effective robotic exploration in continuous domains requires planning trajectories that maximize coverage over a predefined region. A recent development, Stein Variational Ergodic Search (SVES), proposed parallel ergodic exploration (a key approach within the field of robotic exploration), via Stein variational inference that computes a set of candidate trajectories approximating the posterior distribution over the solution space trajectories. While this approach leverages GPU parallelism well, the trajectories in the set might not be distinct enough, leading to a suboptimal set. In this paper, we propose two key methods to diversify the solution set of this approach. First, we leverage the signature kernel within the SVES framework, introducing a pathwise, sequence-sensitive interaction that preserves the Markovian structure of the trajectories and naturally spreads paths across distinct regions of the search space. Second, we propose a derivative-free evolution-strategy interpretation of SVES that exploits batched, GPU-friendly fitness evaluations and can be paired with approximate gradients whenever analytic gradients of the kernel are unavailable or computationally intractable. The resulting method both retains SVES’s advantages while diversifying the solution set and extending its reach to black-box objectives. Across planar forest search, 3D quadrotor coverage, and model-predictive control benchmarks, our approach consistently reduces ergodic cost and produces markedly richer trajectory sets than SVES without significant extra tuning effort.

In numerical integration, Bayesian quadrature (BQ) excels at producing estimates with quantified uncertainties, particularly in sparse data settings. However, its computational scalability and kernel learning capabilities have lagged behind modern advances in Gaussian process research. To bridge this gap, we recast the BQ posterior integral as a convolution operation, which enables efficient computation via fast Fourier transform of low-rank matrices. We introduce two new methods enabled by recasting BQ as a convolution: fast Fourier Bayesian quadrature and sparse spectrum Bayesian quadrature. These methods enhance the computational scalability of BQ and expand kernel flexibility, enabling the use of \textit{any} stationary kernel in the BQ setting. We empirically validate the efficacy of our approach through a range of integration tasks, substantiating the benefits of the proposed methodology.

We integrate neural networks into kinodynamic motion planning and present the Learning for KinoDynamic Tree Expansion (L4KDE) method. Tree-based planning approaches, such as rapidly exploring random tree (RRT), are the dominant approach to finding globally optimal plans in continuous state-space motion planning. Central to these approaches is tree expansion, the procedure in which new nodes are added to an ever-expanding tree. We study the kinodynamic variants of tree-based planning, where we have known system dynamics and kinematic constraints. In the interest of quickly selecting nodes to connect newly sampled coordinates, existing methods typically cannot optimise the finding of nodes that have a low cost to transition to sampled coordinates. Instead, they use metrics like Euclidean distance between coordinates as a heuristic for selecting candidate nodes to connect to the search tree. We propose L4KDE to address this issue. L4KDE uses a neural network to predict transition costs between queried states, which can be efficiently computed in batch, providing much higher quality estimates of transition cost compared to commonly used heuristics while maintaining almost-surely asymptotic optimality guarantee. We empirically demonstrate the significant performance improvement provided by L4KDE on a variety of challenging system dynamics,with the ability to generalise across different instances of the same model class and in conjunction with a suite of modern tree-based motion planners.

Probabilistic Movement Primitives (ProMPs) and their variants are powerful methods for enabling robots to learn complex tasks from human demonstrations, where motion trajectories are represented as stochastic processes with Gaussian assumptions. However, despite their computational efficiency, these methods have limited expressiveness in capturing the diversity found in human demonstrations, which are typically characterized by the multi-modality of motions. For example, when picking up an object partially obscured by an obstacle, some individuals may opt to go to the right, while others may choose the left side of the object. In this paper, we introduce Stein Movement Primitives (SMPs), a novel approach to probabilistic movement primitives. We formulate motion primitive adaptation as a non-parametric probabilistic inference using Stein Variational Gradient Descent (SVGD), thus avoiding any explicit posterior distribution assumptions and enabling the direct representation of the multi-modality in human demonstrations. We illustrate how our method can adapt robot motion to different scenarios while maintaining high similarity to the original demonstrations, even when the demonstrations are multi-modal. Experimentally, we demonstrate our approach to several domain adaptation problems using the LASA dataset and with a real robotic arm.

In large-scale regression problems, random Fourier features (RFFs) have significantly enhanced the computational scalability and flexibility of Gaussian processes (GPs) by defining kernels through their spectral density, from which a finite set of Monte Carlo samples can be used to form an approximate low-rank GP. However, the efficacy of RFFs in kernel approximation and Bayesian kernel learning depends on the ability to tractably sample the kernel spectral measure and the quality of the generated samples. We introduce Stein random features (SRF), leveraging Stein variational gradient descent, which can be used to both generate high-quality RFF samples of known spectral densities as well as flexibly and efficiently approximate traditionally non-analytical spectral measure posteriors. SRFs require only the evaluation of log-probability gradients to perform both kernel approximation and Bayesian kernel learning that results in superior performance over traditional approaches. We empirically validate the effectiveness of SRFs by comparing them to baselines on kernel approximation and well-known GP regression problems.

Robotic grasping of 3D deformable objects is critical for real-world applications such as food handling and robotic surgery. Unlike rigid and articulated objects, 3D deformable objects have infinite degrees of freedom. Fully defining their state requires 3D deformation and stress fields, which are exceptionally difficult to analytically compute or experimentally measure. Thus, evaluating grasp candidates for grasp planning typically requires accurate, but slow 3D finite element method (FEM) simulation. Sampling-based grasp planning is often impractical, as it requires evaluation of a large number of grasp candidates. Gradient-based grasp planning can be more efficient, but requires a differentiable model to synthesize optimal grasps from initial candidates. Differentiable FEM simulators may fill this role, but are typically no faster than standard FEM. In this work, we propose learning a predictive graph neural network (GNN), DefGraspNets, to act as our differentiable model. We train DefGraspNets to predict 3D stress and deformation fields based on FEM-based grasp simulations. DefGraspNets not only runs up to 1500x faster than the FEM simulator, but also enables fast gradient-based grasp optimization over 3D stress and deformation metrics. We design DefGraspNets to align with real-world grasp planning practices and demonstrate generalization across multiple test sets, including real-world experiments.

This paper explores the problem of collision-free motion generation for manipulators by formulating it as a global motion optimization problem. We develop a parallel optimization technique to solve this problem and demonstrate its effectiveness on massively parallel GPUs. We show that combining simple optimization techniques with many parallel seeds leads to solving difficult motion generation problems within 53ms on average, 62x faster than SOTA trajectory optimization methods. We achieve SOTA performance by combining L-BFGS step direction estimation with a novel parallel noisy line search scheme and a particle-based optimization solver. To further aid trajectory optimization, we develop a parallel geometric planner that is atleast 28x faster than SOTA RRTConnect implementations. We also introduce a collision-free IK solver that can solve over 9000 queries/s. We are releasing our GPU accelerated library CuRobo that contains core components for robot motion generation. Additional details are available at sites.google.com/nvidia.com/curobo.

Motion generation seeks to produce safe and feasible robot motion from start to goal. Various tools at different levels of granularity have been developed. On one extreme, sampling-based motion planners focus on completeness - a solution, if it exists, would eventually be found. However, produced paths are often of low quality, and contain superfluous motion. On the other, reactive methods optimise the immediate cost to obtain the next controls, producing smooth and legible motion that can quickly adapt to perturbations, uncertainties, and changes in the environment. However, reactive methods are highly local, and often produce motion that become trapped in non-convex regions of the environment. This paper contributes, Geometric Fabric Command Sequences, a method that lies in the middle ground. It can produce globally optimal motion that is smooth and intuitive, while being also reactive. We model motion via a reactive Geometric Fabric policy that ingests a sequence of attractor states, or commands, and then apply global optimisation over the space of commands. We postulate that solutions for different problems and scenes are highly transferable when conditioned on environmental features. Therefore, an implicit generative model is trained on solutions from optimisation and environment features in a self-supervised manner. That is, faced with multiple motion generation problems, the learning and optimisation are contained within the same loop: the optimisation generates labels for learning, while the learning improves the optimisation for the next problem, which in turn provides higher quality labels. We empirically validate our method in both simulation and on a real-world 6-DOF JACO arm.

Advances in differentiable numerical integrators have enabled the use of gradient descent techniques to learn ordinary differential equations (ODEs), where a flexible function approximator (often a neural network) is used to estimate the system dynamics, given as a time derivative. However, these integrators can be unsatisfactorily slow and unstable when learning systems of ODEs from long sequences. We propose to learn an ODE of interest from data by viewing its dynamics as a vector field related to another base vector field via a diffeomorphism (i.e., a differentiable bijection), represented by an invertible neural network (INN). By learning both the INN and the dynamics of the base ODE, we provide an avenue to offload some of the complexity in modelling the dynamics directly on to the INN. Consequently, by restricting the base ODE to be amenable to integration, we can speed up and improve the robustness of integrating trajectories from the learned system. We demonstrate the efficacy of our method in training and evaluating benchmark ODE systems, as well as within continuous-depth neural networks models. We show that our approach attains speed-ups of up to two orders of magnitude when integrating learned ODEs.

We propose an adaptive optimisation approach for tuning stochastic model predictive control (MPC) hyper-parameters while jointly estimating probability distributions of the transition model parameters based on performance rewards. In particular, we develop a Bayesian optimisation (BO) algorithm with a heteroscedastic noise model to deal with varying noise across the MPC hyper-parameter and dynamics model parameter spaces. Typical homoscedastic noise models are unrealistic for tuning MPC since stochastic controllers are inherently noisy, and the level of noise is affected by their hyper-parameter settings. We evaluate the proposed optimisation algorithm in simulated control and robotics tasks where we jointly infer control and dynamics parameters. Experimental results demonstrate that our approach leads to higher cumulative rewards and more stable controllers.

Reproducing real world dynamics in simulation is critical for the development of new control and perception methods. This task typically involves the estimation of simu-lation parameter distributions from observed rollouts through an inverse inference problem characterized by multi-modality and skewed distributions. We address this challenging problem through a novel Bayesian inference approach that approximates a posterior distribution over simulation parameters given real sensor measurements. By extending the commonly used Gaus-sian likelihood model for trajectories via the multiple-shooting formulation, our gradient-based particle inference algorithm, Stein Variational Gradient Descent, is able to identify highly nonlinear, underactuated systems. We leverage GPU code gen-eration and differentiable simulation to evaluate the likelihood and its gradient for many particles in parallel. Our algorithm infers nonparametric distributions over simulation parame-ters more accurately than comparable baselines and handles constraints over parameters efficiently through gradient-based optimization. We evaluate estimation performance on several physical experiments. On an underactuated mechanism where a 7-DOF robot arm excites an object with an unknown mass configuration, we demonstrate how the inference technique can identify symmetries between the parameters and provide highly accurate predictions. Website: https://uscresl.github.io/prob-diff-sim

In an environment where a manipulator needs to execute multiple consecutive tasks, the act of object manoeuvre will change the underlying configuration space, affecting all subsequent tasks. Previously free configurations might now be occupied by the manoeuvred objects, and previously occupied space might now open up new paths. We propose Lazy Tree-based Replanner (LTR *)-a novel hybrid planner that inherits the rapid planning nature of existing anytime incremental sampling-based planners. At the same time, it allows subsequent tasks to leverage prior experience via a lazy experience graph. Previous experience is summarised in a lazy graph structure, and LTR * is formulated to be robust and beneficial regard-less of the extent of changes in the workspace. Our hybrid approach attains a faster speed in obtaining an initial solution than existing roadmap-based planners and often with a lower cost in trajectory length. Subsequent tasks can utilise the lazy experience graph to speed up finding a solution and take advant-age of the optimised graph to minimise the cost objective. We provide proofs of probabilistic completeness and almost-surely asymptotic optimal guarantees. Experimentally, we show that in repeated pick-and-place tasks, L T R * attains a high gain in performance when planning for subsequent tasks.

Deep reinforcement learning (RL) is a promising approach to solving complex robotics problems. However, the process of learning through trial-and-error interactions is often highly time-consuming, despite recent advancements in RL algorithms. Additionally, the success of RL is critically dependent on how well the reward-shaping function suits the task, which is also time-consuming to design. As agents trained on a variety of robotics problems continue to proliferate, the ability to reuse their valuable learning for new domains becomes increasingly significant. In this paper, we propose a post-hoc technique for policy fusion using Optimal Transport theory as a robust means of consolidating the knowledge of multiple agents that have been trained on distinct scenarios. We further demonstrate that this provides an improved weights initialisation of the neural network policy for learning new tasks, requiring less time and computational resources than either retraining the parent policies or training a new policy from scratch. Ultimately, our results on diverse agents commonly used in deep RL show that specialised knowledge can be unified into a “Renaissance agent”, allowing for quicker learning of new skills.

Bayesian optimisation (BO) algorithms have shown remarkable success in applications involving expensive black-box functions. Traditionally BO has been set as a sequential decision-making process which estimates the utility of query points via an acquisition function and a prior over functions, such as a Gaussian process. Recently, however, a reformulation of BO via density-ratio estimation (BORE) allowed reinterpreting the acquisition function as a probabilistic binary classifier, removing the need for an explicit prior over functions and increasing scalability. In this paper, we present a theoretical analysis of BORE's regret and an extension of the algorithm with improved uncertainty estimates. We also show that BORE can be naturally extended to a batch optimisation setting by recasting the problem as approximate Bayesian inference. The resulting algorithms come equipped with theoretical performance guarantees and are assessed against other batch and sequential BO baselines in a series of experiments.

Bayesian probabilistic integration, or Bayesian quadrature (BQ), has arisen as a popular means of numerical integral estimation with quantified uncertainty for problems where computational cost limits data availability. BQ leverages flexible Gaussian processes (GPs) to model an integrand which can be subsequently analytically integrated through properties of Gaussian distributions. However, BQ is inherently limited by the fact that the method relies on the use of a strict set of kernels for use in the GP model of the integrand, reducing the flexibility of the method in modeling varied integrand types. In this paper, we present spectral Bayesian quadrature, a form of Bayesian quadrature that allows for the use of any shift-invariant kernel in the integrand GP model while still maintaining the analytical tractability of the integral posterior, increasing the flexibility of BQ methods to address varied problem settings. Additionally our method enables integration with respect to a uniform expectation, effectively computing definite integrals of challenging integrands. We derive the theory and error bounds for this model, as well as demonstrate GBQ’s improved accuracy, flexibility, and data efficiency, compared to traditional BQ and other numerical integration methods, on a variety of quadrature problems.

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are computed from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI). The need to ensure analytical tractability of the predictive often poses limitations that can hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the link between class-probability estimation and density-ratio estimation, and the lesser-known link between density-ratios and EI. By circumventing the tractability constraints, this reformulation provides numerous advantages, not least in terms of expressiveness, versatility, and scalability.

Deep learning-based object pose estimators are often unreliable and overconfident especially when the input image is outside the training domain, for instance, with sim2real transfer. Efficient and robust uncertainty quantification (UQ) in pose estimators is critically needed in many robotic tasks. In this work, we propose a simple, efficient, and plug-and-play UQ method for 6-DoF object pose estimation. We ensemble 2–3 pre-trained models with different neural network architectures and/or training data sources, and compute their average pair-wise disagreement against one another to obtain the uncertainty quantification. We propose four disagreement metrics, including a learned metric, and show that the average distance (ADD) is the best learning-free metric and it is only slightly worse than the learned metric, which requires labeled target data. Our method has several advantages compared to the prior art: 1) our method does not require any modification of the training process or the model inputs; and 2) it needs only one forward pass for each model. We evaluate the proposed UQ method on three tasks where our uncertainty quantification yields much stronger correlations with pose estimation errors than the baselines. Moreover, in a real robot grasping task, our method increases the grasping success rate from 35% to 90%. Video and code are available at https://sites.google.com/view/fastuq.

Critical for the coexistence of humans and robots in dynamic environments is the capability for agents to understand each other’s actions, and anticipate their movements. This paper presents Stochastic Process Anticipatory Navigation (SPAN), a framework that enables nonholonomic robots to navigate in environments with crowds, while anticipating and accounting for the motion patterns of pedestrians. To this end, we learn a predictive model to predict continuous-time stochastic processes to model future movement of pedestrians. Anticipated pedestrian positions are used to conduct chance constrained collision-checking, and are incorporated into a time-to-collision control problem. An occupancy map is also integrated to allow for probabilistic collision-checking with static obstacles. We demonstrate the capability of SPAN in crowded simulation environments, as well as with a real-world pedestrian dataset.

Sampling-based motion planning is the predominant paradigm in many real-world robotic applications, but its performance is immensely dependent on the quality of the samples. The majority of traditional planners are inefficient as they use uninformative sampling distributions instead of exploiting structures and patterns in the problem to guide better sampling strategies. Moreover, most current learning-based planners are susceptible to posterior collapse or mode collapse due to the sparsity and highly varying nature of C-Space and motion plan configurations. This work introduces a conditional normalising flow-based distribution learned through previous experiences, which improves existing methods’ sampling scheme. Our distribution can be conditioned on the current problem instance to provide informative prior to sample configurations within promising regions. When we train our sampler with an expert planner, the resulting distribution is often near-optimal, and the planner can find a solution faster, with less invalid samples and less initial cost. The normalising flow-based distribution uses simple invertible transformations that are very computationally efficient, and our optimisation formulation explicitly avoids mode collapse in contrast to other existing learning-based sampler. Finally, we provide a formulation and theoretical foundation to sample from the distribution efficiently. Experimentally we demonstrate utilising the flow-based distribution in a sampling-based motion planner allows a solution to be found faster, with fewer samples and better overall runtime performance.

Being able to safely operate for extended periods of time in dynamic environments is a critical capability for autonomous systems. This generally involves the prediction and understanding of motion patterns of dynamic entities, such as vehicles and people, in the surroundings. Many motion prediction methods in the literature implicitly account for environmental factors by learning on observed motion in a fixed environment, and are designed to make predictions in the same environment. In this paper, we address the problem of generating likely motion trajectories for novel environments, represented as occupancy grid maps, where motion has not been observed. We introduce the Occupancy-Conditional Trajectory Network (OTNet) framework, capable of transferring the previously observed motion patterns in known environments to new environments. OTNet provides a functional representation for motion trajectories and utilises neural networks to learn occupancy-conditional distributions over the function parameters. We empirically demonstrate our method’s ability to generate complex multi-modal trajectory patterns in both simulated and real-world environments.

This work addresses the problem of predicting the motion trajectories of dynamic objects in the environment. Recent advances in predicting motion patterns often rely on machine learning techniques to extrapolate motion patterns from observed trajectories, with no mechanism to directly incorporate known rules. We propose a novel framework, which combines probabilistic learning and constrained trajectory optimisation. The learning component of our framework provides a distribution over future motion trajectories conditioned on observed past coordinates. This distribution is then used as a prior to a constrained optimisation problem which enforces chance constraints on the trajectory distribution. This results in constraint-compliant trajectory distributions which closely resemble the prior. In particular, we focus our investigation on collision constraints, such that extrapolated future trajectory distributions conform to the environment structure. We empirically demonstrate on real-world and simulated datasets the ability of our framework to learn complex probabilistic motion trajectories for motion data, while directly enforcing constraints to improve generalisability, producing more robust and higher quality trajectory distributions.

We consider Bayesian inference problems where the likelihood function is either expensive to evaluate or only available via noisy estimates. This setting encompasses application scenarios involving, for example, large datasets or models whose likelihood evaluations require expensive simulations. We formulate this problem within a Bayesian optimisation framework over a space of probability distributions and derive an upper confidence bound (UCB) algorithm to propose non-parametric distribution candidates. The algorithm is designed to minimise regret, which is defined as the Kullback-Leibler divergence with respect to the true posterior in this case. Equipped with a Gaussian process surrogate model, we show that the resulting UCB algorithm achieves asymptotically no regret. The method can be easily implemented as a batch Bayesian optimisation algorithm whose point evaluations are selected via Markov chain Monte Carlo. Experimental results demonstrate the method’s performance on inference problems.

Accurate simulation of complex physical systems enables the development, testing, and certification of control strategies before they are deployed into the real systems. As simulators become more advanced, the analytical tractability of the differential equations and associated numerical solvers incorporated in the simulations diminishes, making them difficult to analyse. A potential solution is the use of probabilistic inference to assess the uncertainty of the simulation parameters given real observations of the system. Unfortunately the likelihood function required for inference is generally expensive to compute or totally intractable. In this paper we propose to leverage the power of modern simulators and recent techniques in Bayesian statistics for likelihood-free inference to design a control framework that is efficient and robust with respect to the uncertainty over simulation parameters. The posterior distribution over simulation parameters is propagated through a potentially non-analytical model of the system with the unscented transform, and a variant of the information theoretical model predictive control. This approach provides a more efficient way to evaluate trajectory roll outs than Monte Carlo sampling, reducing the online computation burden. Experiments show that the controller proposed attained superior performance and robustness on classical control and robotics tasks when compared to models not accounting for the uncertainty over model parameters.

Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. On the other hand, reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sampleinefficient and brittle. In this work, we combine the strengths of model-based methods with the flexibility of learning-based methods to obtain a general method that is able to overcome inaccuracies in the robotics perception/actuation pipeline, while requiring minimal interactions with the environment. This is achieved by leveraging uncertainty estimates to divide the space in regions where the given model-based policy is reliable, and regions where it may have flaws or not be well defined. In these uncertain regions, we show that a locally learned-policy can be used directly with raw sensory inputs. We test our algorithm, Guided Uncertainty-Aware Policy Optimization (GUAPO), on a real-world robot performing peg insertion. Videos are available at: https://sites.google.com/view/guapo-rl.

Accurate uncertainty estimation associated with the pose transformation between two 3D point clouds is critical for autonomous navigation, grasping, and data fusion. Iterative closest point (ICP) is widely used to estimate the transformation between point cloud pairs by iteratively performing data association and motion estimation. Despite its success and popularity, ICP is effectively a deterministic algorithm, and attempts to reformulate it in a probabilistic manner generally do not capture all sources of uncertainty, such as data association errors and sensor noise. This leads to overconfident transformation estimates, potentially compromising the robustness of systems relying on them. In this paper we propose a novel method to estimate pose uncertainty in ICP with a Markov Chain Monte Carlo (MCMC) algorithm. Our method combines recent developments in optimization for scalable Bayesian sampling such as stochastic gradient Langevin dynamics (SGLD) to infer a full posterior distribution of the pose transformation between two point clouds. We evaluate our method, called Bayesian ICP, in experiments using 3D Kinect data demonstrating that our method is capable of both quickly and accuractely estimating pose uncertainty, taking into account data association uncertainty as reflected by the shape of the objects.

Learning from offline task demonstrations is a problem of great interest in robotics. For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task. However, leveraging a fixed batch of data can be problematic for larger datasets and longer-horizon tasks with greater variations. The data can exhibit substantial diversity and consist of suboptimal solution approaches. In this paper, we propose Implicit Reinforcement without Interaction at Scale (IRIS), a novel framework for learning from large-scale demonstration datasets. IRIS factorizes the control problem into a goal-conditioned low-level controller that imitates short demonstration sequences and a high-level goal selection mechanism that sets goals for the low-level and selectively combines parts of suboptimal solutions leading to more successful task completions. We evaluate IRIS across three datasets, including the RoboTurk Cans dataset collected by humans via crowdsourcing, and show that performant policies can be learned from purely offline learning. Additional results at https://sites.google.com/stanford.edu/iris/.

Granular media (e.g., cereal grains, plastic resin pellets, and pills) are ubiquitous in robotics-integrated industries, such as agriculture, manufacturing, and pharmaceutical development. This prevalence mandates the accurate and efficient simulation of these materials. This work presents a software and hardware framework that automatically calibrates a fast physics simulator to accurately simulate granular materials by inferring material properties from real-world depth images of granular formations (i.e., piles and rings). Specifically, coefficients of sliding friction, rolling friction, and restitution of grains are estimated from summary statistics of grain formations using likelihood-free Bayesian inference. The calibrated simulator accurately predicts unseen granular formations in both simulation and experiment; furthermore, simulator predictions are shown to generalize to more complex tasks, including using a robot to pour grains into a bowl, as well as to create a desired pattern of piles and rings.

Recent advancements in Bayesian likelihood-free inference enables a probabilistic treatment for the problem of estimating simulation parameters and their uncertainty given sequences of observations. Domain randomization can be performed much more effectively when a posterior distribution provides the correct uncertainty over parameters in a simulated environment. In this paper, we study the integration of simulation parameter inference with both model-free reinforcement learning and model-based control in a novel sequential algorithm that alternates between learning a better estimation of parameters and improving the controller. This approach exploits the interdependence between the two problems to generate computational efficiencies and improved reliability when a black-box simulator is available. Experimental results suggest that both control strategies have better performance when compared to traditional domain randomization methods.

We establish a general form of explicit, input-dependent, measure-valued warpings for learning nonstationary kernels. While stationary kernels are uniquitous and simple to use, they struggle to adapt to functions that vary in smoothness with respect to the input. The proposed learning algorithm warps inputs as conditional Gaussian measures that control the smoothness of a standard stationary kernel. This construction allows us to capture non-stationary patterns in the data and provides intuitive inductive bias. The resulting method is based on sparse spectrum Gaussian processes, enabling closed-form solutions, and is extensible to a stacked construction to capture more complex patterns. The method is extensively validated alongside related algorithms on synthetic and real world datasets. We demonstrate a remarkable efficiency in the number of parameters of the warping functions in learning problems with both small and large data regimes.

We consider the problem of sequentially optimising the conditional expectation of an objective function, with both the conditional distribution and the objective function assumed to be fixed but unknown. Assuming that the objective function belongs to a reproducing kernel Hilbert space (RKHS), we provide a novel upper confidence bound (UCB) based algorithm CME-UCB via estimation of the conditional mean embeddings (CME), and derive its regret bound. Along the way, we derive novel approximation guarantees for the CME estimates. Finally, experiments are carried out in a synthetic example and in a likelihood-free inference application that highlight the useful insights of the proposed method.

In likelihood-free settings where likelihood evaluations are intractable, approximate Bayesian computation (ABC) addresses the formidable inference task to discover plausible parameters of simulation programs that explain the observations. However, they demand large quantities of simulation calls. Critically, hyperparameters that determine measures of simulation discrepancy crucially balance inference accuracy and sample efficiency, yet are difficult to tune. In this paper, we present kernel embedding likelihood-free inference (KELFI), a holistic framework that automatically learns model hyperparameters to improve inference accuracy given limited simulation budget. By leveraging likelihood smoothness with conditional mean embeddings, we nonparametrically approximate likelihoods and posteriors as surrogate densities and sample from closed-form posterior mean embeddings, whose hyperparameters are learned under its approximate marginal likelihood. Our modular framework demonstrates improved accuracy and efficiency on challenging inference problems in ecology.

Bayesian optimisation (BO) has been a successful approach to optimise functions which are expensive to evaluate and whose observations are noisy. Classical BO algorithms, however, do not account for errors about the location where observations are taken, which is a common issue in problems with physical components. In these cases, the estimation of the actual query location is also subject to uncertainty. In this context, we propose an upper confidence bound (UCB) algorithm for BO problems where both the outcome of a query and the true query location are uncertain. The algorithm employs a Gaussian process model that takes probability distributions as inputs. Theoretical results are provided for both the proposed algorithm and a conventional UCB approach within the uncertain-inputs setting. Finally, we evaluate each method’s performance experimentally, comparing them to other input noise aware BO approaches on simulated scenarios involving synthetic and real data.

Kernel methods have been successfully used in various domains to model nonlinear patterns. However, the structure of the kernels is typically handcrafted for each dataset based on the experience of the data analyst. In this paper, we present a novel technique to learn kernels that best fit the data. We exploit the measure-theoretic view of a shift-invariant kernel given by the Bochner’s theorem, and automatically learn the measure in terms of a parameterized quantile function. This flexible black box quantile function, evaluated on Quasi-Monte Carlo samples, builds up quasi-random Fourier feature maps that can approximate arbitrary kernels. The proposed method is not only general enough to be used in any kernel machine, but can also be combined with other kernel design techniques. We learn expressive kernels on a variety of datasets, verifying the methods ability to automatically discover complex patterns without being guided by human expert knowledge.

Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inference task. We formalize deconditional kernel mean embeddings as a solution to this inverse problem, and show that it can be naturally viewed as a nonparametric Bayes' rule. Critically, we introduce the notion of task transformed Gaussian processes and establish deconditional kernel means as their posterior predictive mean. This connection provides Bayesian interpretations and uncertainty estimates for deconditional kernel mean embeddings, explains their regularization hyperparameters, and reveals a marginal likelihood for kernel hyperparameter learning. These revelations further enable practical applications such as likelihood-free inference and learning sparse representations for big data.

Path planners are generally categorised as either trajectory optimisers or sampling-based planners. The latter is the predominant planning paradigm for occupancy maps. Most trajectory optimisers require a fully defined artificial potential field for planning and cannot incorporate updates from a partially observed model such as an occupancy map. A stochastic trajectory optimiser capable of planning over occupancy map was presented in [1]. However, its scalability is limited by the cubic complexity of the Gaussian process path representation. In this work, we introduce a novel highly expressive path representation based on kernel approximation to perform trajectory optimisation over occupancy maps. This approach reduces the computational complexity to a fixed cost that only depends on the number of features. We show that stochastic sampling is crucial for planning in occupancy maps and present comparisons to other state-of-the-art planning methods, using simulated and real occupancy data. These experiments demonstrate the significant reduction in runtime, resulting in performance comparable to or better than sampling-based methods.

This paper addresses the problem of learning instantaneous occupancy levels of dynamic environments and predicting future occupancy levels. Due to the complexity of most real environments, such as urban streets or crowded areas, the efficient and robust incorporation of temporal dependencies into otherwise static occupancy models remains a challenge. We propose a method to capture the uncertainty of moving objects and incorporate this uncertainty information into a continuous occupancy map represented in a rich high-dimensional feature space. This data-efficient model not only allows us to learn the occupancy states incrementally, but also makes predictions about what the future occupancy states will be. Experiments performed using 2D and 3D laser data collected from crowded unstructured outdoor environments show that the proposed methodology can accurately predict occupancy states for areas of around 1000 m2at 10 Hz, making the proposed framework ideal for online applications under real-time constraints.

Sampling efficiency in a highly constrained environment has long been a major challenge for sampling-based planners. In this work, we propose Rapidly-exploring Random disjointed-Trees*(RRdT*), an incremental optimal multi-query planner. RRdT*uses multiple disjointed-trees to exploit local-connectivity of spaces via Markov Chain random sampling, which utilises neighbourhood information derived from previous successful and failed samples. To balance local exploitation, RRdT*actively explore unseen global spaces when local-connectivity exploitation is unsuccessful. The active trade-off between local exploitation and global exploration is formulated as a multi-armed bandit problem. We argue that the active balancing of global exploration and local exploitation is the key to improving sample efficient in sampling-based motion planners. We provide rigorous proofs of completeness and optimal convergence for this novel approach. Furthermore, we demonstrate experimentally the effectiveness of RRdT*'s locally exploring trees in granting improved visibility for planning. Consequently, RRdT*outperforms existing state-of-the-art incremental planners, especially in highly constrained environments.

Sensors producing 3D point clouds such as 3D laser scanners and RGB-D cameras are widely used in robotics, be it for autonomous driving or manipulation. Aligning point clouds produced by these sensors is a vital component in such applications to perform tasks such as model registration, pose estimation, and SLAM. Iterative closest point (ICP) is the most widely used method for this task, due to its simplicity and efficiency. In this paper we propose a novel method which solves the optimisation problem posed by ICP using stochastic gradient descent (SGD). Using SGD allows us to improve the convergence speed of ICP without sacrificing solution quality. Experiments using Kinect as well as Velodyne data show that, our proposed method is faster than existing methods, while obtaining solutions comparable to standard ICP. An additional benefit is robustness to parameters when processing data from different sensors.

Mapping the occupancy of an environment is central for robot autonomy. Traditional occupancy grid maps discretise the environment into independent cells, neglecting important spatial correlations, and are unable to capture the continuous nature of the real world. With these drawbacks of grid maps in mind, Hilbert Maps (HM) and more recently Bayesian Hilbert Maps (BHMs), were introduced as a continuous representation of the environment. In this paper we propose a method to merge Bayesian Hilbert Maps built by a team of robots in a decentralised manner. The training of BHMs requires the inversion of a large covariance matrix, incurring cubic complexity. We introduce an approximation, Fast Bayesian Hilbert Maps (Fast-BHM), which reduces the time complexity to below quadratic. Such speed-ups allow the building and merging of Bayesian Hilbert Map models to be practical, opening the door for multi-robot Hilbert Map systems which can be much faster and more robust than an individual robot. By merging several individual Fast-BHMs in a decentralised manner we obtain a unified model of the environment which is itself a Fast-BHM. We conduct experiments to show that global Fast-BHM models do not deteriorate after repeated merging and training. We then empirically demonstrate, due to its the compact representation, fused Fast-BHMs outperform fusion methods involving discretising continuous representations, when the amount of information communicated is limited.

Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects' choices and their learning abilities. We demonstrate the benefits of this approach using a new dataset drawn from patients with either unipolar (n = 34) or bipolar (n = 33) depression and matched healthy controls (n = 34) making decisions on a two-armed bandit task. The results indicate that this new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects' choices. We show that the model can be interpreted using off-policy simulations and thereby provides a novel clustering of subjects' learning processes-something that often eludes traditional approaches to modelling and behavioural analysis.

This paper introduces a novel methodology for 3D template matching that is scalable to higher-dimensional spaces and larger kernel sizes. It uses the Hilbert Maps framework to model raw pointcloud information as a continuous occupancy function, and we derive a closed-form solution to the convolution operation that takes place directly in the Reproducing Kernel Hilbert Space defining these functions. The result is a third function modeling activation values, that can be queried at arbitrary resolutions with logarithmic complexity, and by iteratively searching for high similarity areas we can determine matching candidates. Experimental results show substantial speed gains over standard discrete convolution techniques, such as sliding window and fast Fourier transform, along with a significant decrease in memory requirements, without accuracy loss. This efficiency allows the proposed methodology to be used in areas where discrete convolution is currently infeasible. As a practical example we explore the key problem in robotics of global localization, in which a vehicle must be positioned on a map using only its current sensor information, and provide comparisons with other state-of-the-art techniques in terms of computational speed and accuracy.

Mapping the occupancy level of an environment is important for a robot to navigate in unknown and unstructured environments. To this end, continuous occupancy mapping techniques which express the probability of a location as a function are used. In this work, we provide a theoretical analysis to compare and contrast the two major branches of Bayesian continuous occupancy mapping techniques---Gaussian process occupancy maps and Bayesian Hilbert maps---considering the fact that both utilize kernel functions to operate in a rich high-dimensional implicit feature space and use variational inference to learn parameters. Then, we extend the recent Bayesian Hilbert maps framework which is so far only used for stationary robots, to map large environments with moving robots. Finally, we propose convolution of kernels as a powerful tool to improve different aspects of continuous occupancy mapping. Our claims are also experimentally validated with both simulated and real-world datasets.

Gaussian Processes (GPs) provide an extremely powerful mechanism to model a variety of problems but incur an O(N3) complexity in the number of data samples. Common approximation methods rely on what are often termed inducing points but still typically incur an O(NM2) complexity in the data and corresponding inducing points. Using Random Fourier Feature (RFF) maps, we overcome this by transforming the problem into a Bayesian Linear Regression formulation upon which we apply a Bayesian Variational treatment that also allows learning the corresponding kernel hyperparameters, likelihood and noise parameters. In this paper we introduce an alternative method using Fourier series to obtain spectral representations of common kernels, in particular for periodic warpings, which surprisingly have a convergent, non-random form using special functions, requiring fewer spectral features to approximate their corresponding kernel to high accuracy. Using this, we can fuse the Random Fourier Feature spectral representations of common kernels with their periodic counterparts to show how they can more effectively and expressively learn patterns in time-series for both interpolation and extrapolation. This method combines robustness, scalability and equally importantly, interpretability through a symbolic declarative grammar that is both functionally and humanly intuitive — a property that is crucial for explainable decision making. Using probabilistic programming and Variational Inference we are able to efficiently optimise over these rich functional representations. We show significantly improved Gram matrix approximation errors, and also demonstrate the method in several time-series problems comparing other commonly used approaches such as recurrent neural networks.

Recent advances in embedded technology have enabled more pervasive machine learning. One of the common applications in this field is Egocentric Activity Recognition (EAR), where users wearing a device such as a smartphone or smartglasses are able to receive feedback from the embedded device. Recent research on activity recognition has mainly focused on improving accuracy by using resource intensive techniques such as multi-stream deep networks. Although this approach has provided state-of-the-art results, in most cases it neglects the natural resource constraints (e.g. battery) of wearable devices. We develop a Reinforcement Learning model-free method to learn energy-aware policies that maximize the use of low-energy cost predictors while keeping competitive accuracy levels. Our results show that a policy trained on an egocentric dataset is able use the synergy between motion and vision sensors to effectively tradeoff energy expenditure and accuracy on smartglasses operating in realistic, real-world conditions.

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout the process execution. In these cases, strategies to optimise control policies for individual stages of the process are not applicable, and instead the whole policy needs to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. With that in mind, we propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.

Reinforcement learning has been very successful at learning control policies for robotic agents in order to perform various tasks, such as driving around a track, navigating a maze, and bipedal locomotion. One significant drawback of reinforcement learning methods is that they require a large number of data points in order to learn good policies, a trait known as poor data efficiency or poor sample efficiency. One approach for improving sample efficiency is supervised pre-training of policies to directly clone the behavior of an expert, but this suffers from poor generalization far from the training data. We propose to improve this by using Gaussian dropout networks with a regularization term based on variational inference in the pre-training step. We show that this initializes policy parameters to significantly better values than standard supervised learning or random initialization, thus greatly reducing sample complexity compared with state-of-the-art methods, and enabling an RL algorithm to learn optimal policies for high-dimensional continuous control problems in a practical time frame.

Decision making under uncertainty is a challenging task, especially when dealing with complex robotics scenarios. The Partially Observable Markov Decision Process (POMDP) framework, designed to solve this problem, was subject to much work lately. Most POMDP solvers, however, focus on planning in discrete state, action and/or observations spaces, which does not truly reflect the complexity of most real world problems. This paper addresses the issue by devising a method for solving POMDPs with continuous state, action and observations spaces. The proposed planner, Continuous Belief Tree Search (CBTS), uses Bayesian Optimisation (BO) to dynamically sample promising actions while constructing a belief tree. This dynamic sampling allows for richer action selection than offline action discretisation. CBTS is complemented by a novel trajectory generation technique, relying on the theory of Reproducing Kernel Hilbert Spaces (RKHS), yielding trajectories amenable for robotics applications. The resulting trajectory planner kCBTS outperforms other continuous planners on space modelling and robot parking problems.

Robots often have to deal with the challenges of operating in dynamic and sometimes unpredictable environments. Although an occupancy map of the environment is sufficient for navigation of a mobile robot or manipulation tasks with a robotic arm in static environments, robots operating in dynamic environments demand richer information to improve robustness, efficiency, and safety. For instance, in path planning, it is important to know the direction of motion of dynamic objects at various locations of the environment for safer navigation or human-robot interaction. In this paper, we introduce directional statistics into robotic mapping to model circular data. Primarily, in collateral to occupancy grid maps, we propose directional grid maps to represent the location-wide long-term angular motion of the environment. Being highly representative, this defines a probability measure-field over the longitude-latitude space rather than a scalar-field or a vector-field. Withal, we further demonstrate how the same theory can be used to model angular variations in the spatial domain, temporal domain, and spatiotemporal domain. We carried out a series of experiments to validate the proposed models using a variety of robots having different sensors such as RGB cameras and LiDARs on simulated and real-world settings in both indoor and outdoor environments.

Neuroscience studies of human decision-making abilities commonly involve subjects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subject's choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the model's internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

Map-aided localization methods have been employed for vehicle localization to overcome the limitations of global navigation satellite system (GNSS) devices. In this solution, sensor information is matched to the environment map to determine the vehicle position. Occupancy grid maps (OGMs) have been adapted for map-aided localization. However, there are known drawbacks of OGM, such as the environment discretization, the assumption of independence between grid cells, and the need for dense measurements. In recent years, Gaussian process occupancy map (GPOM) was developed to suppress some of the OGM limitations. GPOM enables the computation of the likelihood of occupancy at any location, even if not directly observed by the sensor, thus representing the environment in a continuous manner. Taking into account the superiority of GPOM over OGM, we devise a novel vehicle localization technique for urban environments. This solution enables more accurate localization due to the use of a representation that better models the real environment. The development of the proposed method is based on Monte Carlo localization, which is a popular map-aided localization method. Two road features commonly found in urban cities were chosen to build the maps: road curbs and road markings. Specifically, the proposed localization method relies on a GPOM constructed with curb data and an OGM built with road marking data. Experiments were performed in real urban environments. Maps were intentionally generated using sparse light detection and ranging (LIDAR) data to verify the localization in non-observed areas. The localization system was evaluated by comparing the results with a high precision GNSS device.

This paper addresses the task of unsupervised feature learning for three-dimensional occupancy mapping, as a way to segment higher-level structures based on raw unorganized point cloud data. In particular, we focus on detecting planar surfaces, which are common in most structured or semi-structured environments. This segmentation is then used to minimize the amount of parameters necessary to properly create a 3D occupancy model of the surveyed space, thus increasing computational speed and decreasing memory requirements. As the 3D modeling tool, an extension to Hilbert Maps was selected, since it naturally uses a feature-based representation of the environment to achieve real-time performance. Experiments conducted in simulated and real large-scale datasets show a substantial gain in performance, while decreasing the amount of stored information by orders of magnitude without sacrificing accuracy.

Malicious software (malware) has been extensively used for illegal activity and new malware variants are discovered at an alarmingly high rate. The ability to group malware variants into families with similar characteristics makes possible to create mitigation strategies that work for a whole class of programs. In this paper, we present a malware family classification approach using a deep neural network based on the ResNet-50 architecture. Malware samples are represented as byteplot grayscale images and a deep neural network is trained freezing the convolutional layers of ResNet-50 pre-trained on the ImageNet dataset and adapting the last layer to malware family classification. The experimental results on a dataset comprising 9,339 samples from 25 different families showed that our approach can effectively be used to classify malware families with an accuracy of 98.62%.

Scene understanding is a crucial requirement for robot navigation. Conditional Random Fields (CRF) are commonly used to solve the scene labelling problem since they represent contextual information efficiently and provide efficient inference methods. However, when a robot navigates through an unknown environment, it is often necessary to adjust the parameters of the CRF online to maintain the same level of accuracy under changes no predicted during the training phase. Online parameter learning can be challenging since ground truth information is not available for newly encountered scenes. To address this issue, this paper proposes a stochastic gradient descent (SGD) method to learn the parameters of a constrained CRF (cCRF) in an online fashion. By leveraging the information from laser scans and image data the complexity of the labelling problem can be significantly reduced. The parameters are estimated by optimising a novel loss function that takes into account highly confident labels as a reference while eliminating the need for manual labelling. These labels are obtained purely based on the information from camera and laser sensors, in a self-supervised manner. Sensor data is pre-processed using methods such as convolutional nets, discriminant analysis, and Euclidean distance based clustering to extract reference labels. We show that this online parameter learning is robust to changes in the data distribution by selecting the learning rate appropriately. Experimental results are presented on the KITTI data set demonstrating the benefits of online CRF training.

Safe path planning is a crucial component in autonomous robotics. The many approaches to find a collision free path can be categorically divided into trajectory optimizers and sampling-based methods. When planning using occupancy maps, the sampling-based approach is the prevalent method. The main drawback of such techniques is that the reasoning about the expected cost of a plan is limited to the search heuristic used by each method. We introduce a novel planning method based on trajectory optimization to plan safe and efficient paths in continuous occupancy maps. We extend the expressiveness of the state-of-the-art functional gradient optimization methods by devising a stochastic gradient update rule to optimize a path represented as a Gaussian process. This approach avoids the need to commit to a specific resolution of the path representation, whether spatial or parametric. We utilize a continuous occupancy map representation in order to define our optimization objective, which enables fast computation of occupancy gradients. We show that this approach is essential in order to ensure convergence to the optimal path, and present results and comparisons to other planning methods in both simulation and with real laser data. The experiments demonstrate the benefits of using this technique when planning for safe and efficient paths in continuous occupancy maps.

Bayesian Optimization has gained much popularity lately, as a global optimization technique for functions that are expensive to evaluate or unknown a priori. While classical BO focuses on where to gather an observation next, it does not take into account practical constraints for a robotic system such as where it is physically possible to gather samples from, nor the sequential nature of the problem while executing a trajectory. In field robotics and other real-life situations, physical and trajectory constraints are inherent problems. This paper addresses these issues by formulating Bayesian Optimization for continuous trajectories within a Partially observable Markov Decision Process (POMDP) framework. The resulting POMDP is solved using Monte-Carlo Tree Search (MCTS), which we adapt to using a reward function balancing exploration and exploitation. Experiments on monitoring a spatial phenomenon with a UAV illustrate how our BO-POMDP algorithm outperforms competing techniques.

Understanding the dynamics of urban environments is crucial for path planning and safe navigation. However, the dynamics might be extremely complex making learning the environment an unfathomable task. Within the methods available for learning dynamic environments, dynamic Gaussian process occupancy maps (DGPOM) are very attractive because they can produce spatially-continuous occupancy maps taking into account neighborhood information, and provide probabilistic estimates, naturally inferring the uncertainty of predictions. Despite these properties, they are extremely slow, especially in dynamic mapping where the parameters of the map have to be updated as new data arrive from range sensors such as LiDARs. In this work, we leverage recent advancements in stochastic variational inference (SVI) to quickly learn dynamic areas in an online fashion. Further, we propose an information-driven technique to “intelligently” select inducing points required for SVI without relying on any object trackers which essentially improves computational time as well as robustness. These long-term occupancy maps entertain all attractive properties of DGPOM while the learning process is significantly faster, yet accurate. Our experiments with both simulation and real robot data on road intersections show a significant improvement in speed while maintaining a comparable or better accuracy.

Visual-Inertial SLAM methods have become a very important technology for several applications in robotics. This kind of approach usually is composed by sensors as rate gyros, accelerometers and monocular cameras. Magnetometers and GPS modules generally used for outdoors are absent in the SLAM system observation, since the magnetometer measurements deteriorate in the presence of ferromagnetic materials and the GPS module signals are unavailable indoors or in urban environments. In order to make use of all these sensors, we propose Markovian jump linear systems (MJLS) to model the modes of operation of the navigation system based on available sensors and their reliability. An extended Kalman filter for MJLS fuses the sensor data and estimates the motion using the best mode of operation for each particular time instant. Experimental results are presented to show the effectiveness of the proposed method, in situations that would pose a challenge for standard data fusion techniques.

Understanding and predicting how influenza propagates is vital to reduce its impact. In this paper we develop a nonparametric model based on Gaussian process (GP) regression to capture the complex spatial and temporal dependencies present in the data. A stochastic variational inference approach was adopted to address scalability. Rather than modeling the problem as a time-series as in many studies, we capture the space-time dependencies by combining different kernels. A kernel averaging technique which converts spatially-diffused point processes to an area process is proposed to model geographical distribution. Additionally, to accurately model the variable behavior of the time-series, the GP kernel is further modified to account for non-stationarity and seasonality. Experimental results on two datasets of state-wide US weekly flu-counts consisting of 19,698 and 89,474 data points, ranging over several years, illustrate the robustness of the model as a tool for further epidemiological investigations.

Online learning has become crucial to many problems in machine learning. As more data is collected sequentially, quickly adapting to changes in the data distribution can offer several competitive advantages such as avoiding loss of prior knowledge and more efficient learning. However, adaptation to changes in the data distribution (also known as covariate shift) needs to be performed without compromising past knowledge already built in into the model to cope with voluminous and dynamic data. In this paper, we propose an online stacked Denoising Autoencoder whose structure is adapted through reinforcement learning. Our algorithm forces the network to exploit and explore favourable architectures employing an estimated utility function that maximises the accuracy of an unseen validation sequence. Different actions, such as Pool, Increment and Merge are available to modify the structure of the network. As we observe through a series of experiments, our approach is more responsive, robust, and principled than its counterparts for non-stationary as well as stationary data distributions. Experimental results indicate that our algorithm performs better at preserving gained prior knowledge and responding to changes in the data distribution.

This paper explores a pragmatic approach to multiple object tracking where the main focus is to associate objects efficiently for online and realtime applications. To this end, detection quality is identified as a key factor influencing tracking performance, where changing the detector can improve tracking by up to 18.9%. Despite only using a rudimentary combination of familiar techniques such as the Kalman Filter and Hungarian algorithm for the tracking components, this approach achieves an accuracy comparable to state-of-the-art online trackers. Furthermore, due to the simplicity of our tracking method, the tracker updates at a rate of 260 Hz which is over 20x faster than other state-of-the-art trackers.

This paper presents a self-supervised approach for learning to associate object detections in a video sequence as often required in tracking-by-detection systems. In this paper we focus on learning an affinity model to estimate the data association cost, which can adapt to different situations by exploiting the sequential nature of video data. We also propose a framework for gathering additional training samples at test time with high variation in visual appearance, naturally inherent in large temporal windows. Reinforcing the model with these difficult samples greatly improves the affinity model compared to standard similarity measures such as cosine similarity. We experimentally demonstrate the efficacy of the resulting affinity model on several multiple object tracking (MOT) benchmark sequences. Using the affinity model alone places this approach in the top 25 state-of-the-art trackers with an average rank of 21.3 across 11 test sequences and an overall multiple object tracking accuracy (MOTA) of 17%. This is considerable as our simple approach only uses the appearance of the detected regions in contrast to other techniques with global optimisation or complex motion models.

In this work, we introduce a novel method for two-dimensional occupancy mapping using Gaussian processes. We address mapping as the task of classifying the robot's environment between free and occupied regions. The biggest challenge when using Gaussian processes for this task is the size of the input datasets. We tackle this problem by introducing a novel kernel, able to use as input data aggregated into two-dimensional cells. Using this kernel, we achieve comparable performance to previous Gaussian process occupancy mapping techniques in a fraction of the time taken by them. The approach can also be used to convert popular occupancy grids into continuous Gaussian process occupancy maps.

The mapping of agricultural crops by capturing images obtained with UAVs enables fast environmental monitoring and diagnosis in large areas. Airborne monitoring in agriculture can a substantially impacts on the identification of diseases and produce accurate information on affected areas. The problem can be formulated as a classification task on aerial images with significant opportunities to impact other fields. This paper presents an active learning method through route planning for improvements in the knowledge on visited areas and minimization uncertainties about the classification of diseases in crops. Binary Logistic Regression and Gaussian Process were used for the detection of pathologies and map interpolation, respectively. A Bayesian optimization strategy is also proposed for the planning of an informative trajectory, which resulted in a maximized search for affected areas in an initially unknown environment.

Detecting changing traffic conditions is of primal importance for the safety of autonomous cars navigating in urban environments. Among the traffic situations that require more attention and careful planning, road junctions are the most significant. This work presents an empirical study of the application of well known machine learning techniques to create a robust method for road junction detection. Features are extracted from 3D pointclouds corresponding to single frames of data collected by a laser rangefinder. Three well known classifiers-support vector machines, adaptive boosting and artificial neural networks-are used to classify them into “junctions” or “roads”. The best performing classifier is used in the next stage, where structured classifiers-hidden Markov models and conditional random fields-are used to incorporate contextual information, in an attempt to improve the performance of the method. We tested and compared these approaches on datasets from two different 3D laser scanners, and in two different countries, Germany and Brazil.

Robots typically possess sensors of different modalities, such as colour cameras, inertial measurement units, and 3D laser scanners. Often, solving a particular problem becomes easier when more than one modality is used. However, while there are undeniable benefits to combine sensors of different modalities the process tends to be complicated. Segmenting scenes observed by the robot into a discrete set of classes is a central requirement for autonomy as understanding the scene is the first step to reason about future situations. Scene segmentation is commonly performed using either image data or 3D point cloud data. In computer vision many successful methods for scene segmentation are based on conditional random fields (CRF) where the maximum a posteriori (MAP) solution to the segmentation can be obtained by inference. In this paper we devise a new CRF inference method for scene segmentation that incorporates global constraints, enforcing the sets of nodes are assigned the same class label. To do this efficiently, the CRF is formulated as a relaxed quadratic program whose MAP solution is found using a gradient-based optimisation approach. The proposed method is evaluated on images and 3D point cloud data gathered in urban environments where image data provides the appearance features needed by the CRF, while the 3D point cloud data provides global spatial constraints over sets of nodes. Comparisons with belief propagation, conventional quadratic programming relaxation, and higher order potential CRF show the benefits of the proposed method.

3D scene reconstruction involves the volumetric modeling of space, and it is a fundamental step in a wide variety of robotic applications, including grasping, obstacle avoidance, path planning, mapping and many others. Nowadays, sensors are able to quickly collect vast amounts of data, and the challenge has become one of storing and processing all this information in a timely manner, especially if real-time performance is required. Recently, a novel technique for the stochastic learning of discriminative models through continuous occupancy maps was proposed: Hilbert Maps [18], that is able to represent the input space at an arbitrary resolution while capturing statistical relationships between measurements. The original framework was proposed for 2D environments, and here we extend it to higher-dimensional spaces, addressing some of the challenges brought by the curse of dimensionality. Namely, we propose a method for the automatic selection of feature coordinate locations, and introduce the concept of localized automatic relevance determination (LARD) to the Hilbert Maps framework, in which different dimensions in the projected Hilbert space operate within independent length-scale values. The proposed technique was tested against other state-of-the-art 3D scene reconstruction tools in three different datasets: a simulated indoors environment, RIEGL laser scans and dense LSD-SLAM pointclouds. The results testify to the proposed framework's ability to model complex structures and correctly interpolate over unobserved areas of the input space while achieving real-time training and querying performances.

We consider the problem of building continuous occupancy representations in dynamic environments for robotics applications. The problem has hardly been discussed previously due to the complexity of patterns in urban environments, which have both spatial and temporal dependencies. We address the problem as learning a kernel classifier on an efficient feature space. The key novelty of our approach is the incorporation of variations in the time domain into the spatial domain. We propose a method to propagate motion uncertainty into the kernel using a hierarchical model. The main benefit of this approach is that it can directly predict the occupancy state of the map in the future from past observations, being a valuable tool for robot trajectory planning under uncertainty. Our approach preserves the main computational benefits of static Hilbert maps — using stochastic gradient descent for fast optimization of model parameters and incremental updates as new data are captured. Experiments conducted in road intersections of an urban environment demonstrated that spatio-temporal Hilbert maps can accurately model changes in the map while outperforming other techniques on various aspects.

Quantile regression deals with the problem of computing robust estimators when the conditional mean and standard deviation of the predicted function are inadequate to capture its variability. The technique has an extensive list of applications, including health sciences, ecology and finance. In this work we present a non-parametric method of inferring quantiles and derive a novel Variational Bayesian (VB) approximation to the marginal likelihood, leading to an elegant Expectation Maximisation algorithm for learning the model. Our method is nonparametric, has strong convergence guarantees, and can deal with nonsymmetric quantiles seamlessly. We compare the method to other parametric and non-parametric Bayesian techniques, and alternative approximations based on expectation propagation demonstrating the benefits of our framework in toy problems and real datasets.

We introduce a novel method for the continuous online prediction of particulate matter in the air (more specifically, PM10 and PM2.5) given sparse sensor information. A nonparametric model is developed using Gaussian Processes, which eschews the need for an explicit formulation of internal -- and usually very complex -- dependencies between meteorological variables. Instead, it uses historical data to extrapolate pollutant values both spatially (in areas with no sensor information) and temporally (the near future). Each prediction also contains a respective variance, indicating its uncertainty level and thus allowing a probabilistic treatment of results. A novel training methodology (Structural Cross-Validation) is presented, which preserves the spatio-temporal structure of available data during the hyperparameter optimization process. Tests were conducted using a real-time feed from a sensor network in an area of roughly 50x80 km, alongside comparisons with other techniques for air pollution prediction. The promising results motivated the development of a smartphone applicative and a website, currently in use to increase the efficiency of air quality monitoring and control in the area.

One of the challenges in precision agriculture is the detection of diseased crops in agricultural environments. This paper presents a methodology to detect the Ceratocystis wilt disease in Eucalyptus crops. An unmanned aerial vehicle is used to obtain high-resolution RGB images of a predefined area. The methodology enables the extraction of visual features from image regions and uses several supervised machine learning (ML) techniques to classify regions into three classes: ground, healthy and diseased plants. Several learning techniques were compared using data obtained from a commercial Eucalyptus plantation. Experimental results show that the GP learning model is more reliable than the other learning methods for accurately identifying diseased trees.

We tackle the problem of multi-task learning with copula process. Multivariable prediction in spatial and spatial-temporal processes such as natural resource estimation and pollution monitoring have been typically addressed using techniques based on Gaussian processes and co-Kriging. While the Gaussian prior assumption is convenient from analytical and computational perspectives, nature is dominated by non-Gaussian likelihoods. Copula processes are an elegant and flexible solution to handle various non-Gaussian likelihoods by capturing the dependence structure of random variables with cumulative distribution functions rather than their marginals. We show how multi-task learning for copula processes can be used to improve multivariable prediction for problems where the simple Gaussianity prior assumption does not hold. Then, we present a transductive approximation for multi-task learning and derive analytical expressions for the copula process model. The approach is evaluated and compared to other techniques in one artificial dataset and two publicly available datasets for natural resource estimation and concrete slump prediction.

In this paper, we propose a dense motion segmentation method for human daily activity recognition from a wearable device - "Smart Glasses". The glasses are embedded with a camera, which allows the system to automatically recognise the wearer's activities from a first-person perspective. This application can be broadly applied to patients, elderly, safety workers, e-health monitoring, or anyone requiring cognitive assistance or guidance on their activities of daily living (ADLs). We validate our system in challenging real-world scenarios, and compare two feature extraction approaches: averaged optical flow and a combined dense motion segmentation approach. We classify them using LogitBoost (on Decision Stumps) and Support Vector Machine (SVM). We also suggest the optimal settings of the classifiers through cross-validation over our ADLs database. The results show that the optical flow with average pooling has a good performance when classifying general locomotive activities. The results also indicate the benefits that dense motion segmentation features can have on reliably classify activities involving a moving object, such as hands. We achieve an overall accuracy of up to 69.76% on 12 ADLs using local classifiers, and with a Hidden Markov Model (HMM) process this accuracy improves to up to 89.59%.

This paper presents a method for the continuous segmentation of dynamic objects using only a vehicle mounted monocular camera without any prior knowledge of the object's appearance. Prior work in online static/dynamic segmentation [1] is extended to identify multiple instances of dynamic objects by introducing an unsupervised motion clustering step. These clusters are then used to update a multi-class classifier within a self-supervised framework. In contrast to many tracking-by-detection based methods, our system is able to detect dynamic objects without any prior knowledge of their visual appearance shape or location. Furthermore, the classifier is used to propagate labels of the same object in previous frames, which facilitates the continuous tracking of individual objects based on motion. The proposed system is evaluated using recall and false alarm metrics in addition to a new multi-instance labelled dataset to measure the performance of segmenting multiple instances of objects.

Environmental monitoring with mobile robots requires solving the informative path planning problem. A key challenge is how to compute a continuous path over space and time that will allow a robot to best sample the environment for an initially unknown phenomenon. To address this problem we devise a layered Bayesian Optimisation approach that uses two Gaussian Processes, one to model the phenomenon and the other to model the quality of selected paths. By using different acquisition functions over both models we tackle the exploration-exploitation trade off in a principled manner. Our method optimises sampling over continuous paths and allows us to find trajectories that maximise the reward over the path. We test our method on a large scale experiment for modelling ozone concentration in the US, and on a mobile robot modelling the changes in luminosity. Comparisons are presented against information based criteria and point-based strategies demonstrating the benefits of our method.

A key challenge for long-term autonomy is to enable a robot to automatically model properties of the environment while actively searching for better decisions to accomplish its task. This amounts to the problem of exploration-exploitation in the context of active perception. This paper addresses active perception and presents a technique to incrementally model the roughness of the terrain a robot navigates on while actively searching for waypoints that reduce the overall vibration experienced during travel. The approach employs Gaussian processes in conjunction with Bayesian optimisation for decision making. The algorithms are executed in real-time on the robot while it explores the environment. We present experiments with an outdoor vehicle navigating over several types of terrains demonstrating the properties and effectiveness of the approach.

We propose a novel pervasive system to recognise human daily activities from a wearable device. The system is designed in a form of reading glasses, named `Smart Glasses', integrating a 3-axis accelerometer and a first-person view camera. Our aim is to classify user's activities of daily living (ADLs) based on both vision and head motion data. This ego-activity recognition system not only allows caretakers to track on a specific person (such as patient or elderly people), but also has the potential to remind/warn people with cognitive impairments of hazardous situations. We present the following contributions in this paper: a feature extraction method from accelerometer and video; a classification algorithm integrating both locomotive (body motions) and stationary activities (without or with small motions); a novel multi-scale dynamic graphical model structure for structured classification over time. We collect, train and validate our system on a large dataset containing 20 hours of ADLs data, including 12 daily activities under different environmental settings. Our method improves the classification performance (F-Score) of conventional approaches from 43.32%(video features) and 66.02%(acceleration features) by an average of 20-40% to 84.45%, with an overall accuracy of 90.04% in realistic ADLs.

We address the problem of automatically segmenting dynamic objects in an urban environment from a moving camera without manual labelling, in an online, self-supervised learning manner. We use input images obtained from a single uncalibrated camera placed on top of a moving vehicle, extracting and matching pairs of sparse features that represent the optical flow information between frames. This optical flow information is initially divided into two classes, static or dynamic, where the static class represents features that comply to the constraints provided by the camera motion and the dynamic class represents the ones that do not. This initial classification is used to incrementally train a Gaussian Process (GP) classifier to segment dynamic objects in new images. The hyperparameters of the GP covariance function are optimized online during navigation, and the available self-supervised dataset is updated as new relevant data is added and redundant data is removed, resulting in a near-constant computing time even after long periods of navigation. The output is a vector containing the probability that each pixel in the image belongs to either the static or dynamic class (ranging from 0 to 1), along with the corresponding uncertainty estimate of the classification. Experiments conducted in an urban environment, with cars and pedestrians as dynamic objects and no prior knowledge or additional sensors, show promising results even when the vehicle is moving at considerable speeds (up to 50 km/h). This scenario produces a large quantity of featureless regions and false matches that is very challenging for conventional approaches. Results obtained using a portable camera device also testify to our algorithm's ability to generalize over different environments and configurations without any fine-tuning of parameters.

We present a novel estimation algorithm for filtering and regression with a number of advantages over existing methods. The algorithm has wide application in robotics as no assumptions are made about the underlying distributions, it can represent non-Gaussian multi-modal posteriors, and learn arbitrary non-linear models from noisy data. Our method is a generalisation of the Kernel Bayes' Rule that produces multi-modal posterior estimates represented as Gaussian mixtures. The algorithm learns non-linear state transition and observation models from data and represents all distributions internally as elements in a reproducing kernel Hilbert space. Inference occurs in the Hilbert space and can be performed recursively. When an estimate of the posterior distribution is required, we apply a quadratic programming pre-image method to determine the Gaussian mixture components of the posterior representation. We demonstrate our algorithm with two filtering experiments and one regression experiment; a multi-modal tracking simulation, a real tracking problem involving a miniature slot-car with an attached inertial measurement unit, and a regression problem of estimating the velocity field of a set of pedestrian paths for robot path-planning. Our algorithm compares favourably with the Gaussian process in the regression case, and a particle filter with learned process and observation models (the “GP-BayesFilter” particle filter).

Current robotic systems carry many diverse sensors such as laser scanners, cameras and inertial measurement units just to name a few. Typically such data is fused by engineering a feature that weights the different sensors against each other in perception tasks. However, in a long-term autonomy setting the sensor readings may change drastically over time which makes a manual feature design impractical. A method that can automatically combine features of different data sources would be highly desirable for adaptation to different environments. In this paper, we propose a novel clustering method, coined Layered Affinity Propagation, for automatic clustering of observations that only requires the definition of features on individual data sources. How to combine these features to obtain a good clustering solution is left to the algorithm, removing the need to create and tune a complicated feature encompassing all sources. We evaluate the proposed method on data containing two very common sensor modalities, images and range information. In a first experiment we show the capability of the method to perform scene segmentation on Kinect data. A second experiment shows how this novel method handles the task of clustering segmented colour and depth data obtained from a Velodyne and camera in an urban environment.

One of the primary aspects of sustainable development involves accurate understanding and modeling of environmental phenomena. Many of these phenomena exhibit variations in both space and time and it is imperative to develop a deeper understanding of techniques that can model space-time dynamics accurately. In this paper we propose NOSTILL-GP - NOn-stationary Space TIme variable Latent Length scale GP, a generic non-stationary, spatio-temporal Gaussian Process (GP) model. We present several strategies, for efficient training of our model, necessary for real-world applicability. Extensive empirical validation is performed using three real-world environmental monitoring datasets, with diverse dynamics across space and time. Results from the experiments clearly demonstrate general applicability and effectiveness of our approach for applications in environmental monitoring.

This paper investigates the possibility of recognising individual persons from their walking gait using three-dimensional 'skeleton' data from an inexpensive consumer-level sensor, the Microsoft 'Kinect'. In an experimental pilot study it is shown that the K-means algorithm - as a candidate unsupervised clustering algorithm - is able to cluster gait samples from four persons with a nett accuracy of 43.6%.

This paper proposes a novel activity recognition approach from video data obtained with a wearable camera. The objective is to recognise the user's activities from a tiny front-facing camera embedded in his/her glasses. Our system allows carers to remotely access the current status of a specified person, which can be broadly applied to those living with disabilities including the elderly who require cognitive assistance or guidance for daily activities. We collected, trained and tested our system on videos collected from different environmental settings. Sequences of four basic activities (drinking, walking, going upstairs and downstairs) are tested and evaluated in challenging real-world scenarios. An optical flow procedure is used as our primary feature extraction method, from which we downsize, reformat and classify sequence of activities using k-Nearest Neighbour algorithm (k-NN), LogitBoost (on Decision Stumps) and Support Vector Machine (SVM). We suggest the optimal settings of these classifiers through cross-validations and achieve an accuracy of 54.2% to 71.9%. Further smoothing using Hidden Markov Model (HMM) improves the result to 68.5%-82.1%.

This paper introduces a novel framework for estimating the motion of a robotic car from image information, a scenario widely known as visual odometry. Most current monocular visual odometry algorithms rely on a calibrated camera model and recover relative rotation and translation by tracking image features and applying geometrical constraints. This approach has some drawbacks: translation is recovered up to a scale, it requires camera calibration which can be tricky under certain conditions, and uncertainty estimates are not directly obtained. We propose an alternative approach that involves the use of semi-parametric statistical models as means to recover scale, infer camera parameters and provide uncertainty estimates given a training dataset. As opposed to conventional non-parametric machine learning procedures, where standard models for egomotion would be neglected, we present a novel framework in which the existing parametric models and powerful non-parametric Bayesian learning procedures are combined. We devise a multiple output Gaussian Process (GP) procedure, named Coupled GP, that uses a parametric model as the mean function and a non-stationary covariance function to map image features directly into vehicle motion. Additionally, this procedure is also able to infer joint uncertainty estimates (full covariance matrices) for rotation and translation. Experiments performed using data collected from a single camera under challenging conditions show that this technique outperforms traditional methods in trajectories of several kilometers.

We present an approach to automatically learn the visual appearance of an environment in terms of object classes. The procedure is totally unsupervised, incremental, and can be executed in real time. The traversability property of an unseen object is also learnt without human supervision by the interaction between the robot and the environment. An incremental version of affinity propagation, a state-of-the-art clustering procedure, is used to cluster image patches into groups of similar visual appearance. For each of these clusters, we obtain the probability of representing an obstacle through the interaction of the robot with the environment. This information then allows the robot to navigate safely through the environment based solely on visual information. Experimental results show that our method extracts meaningful clusters from the images and learns the appearance of objects efficiently. We show that the approach generalises well to both indoor and outdoor environments and that the amount of learning reduces as the robot explores the environment. This is a fundamental property for autonomous adaptation and long-term autonomy.

Automated rock recognition is a key step for building a fully autonomous mine. When characterizing rock types from drill performance data, the main challenge is that there is not an obvious one-to-one correspondence between the two. In this paper, a hybrid rock recognition approach is proposed which combines Gaussian Process (GP) regression with clustering. Drill performance data is also known as Measurement While Drilling (MWD) data and a rock hardness measure - Adjusted Penetration Rate (APR) is extracted using the raw data in discrete drill holes. GP regression is then applied to create a more dense APR distribution, followed by clustering which produces discrete class labels. No initial labelling is needed. Comparisons are made with alternative measures of rock hardness from MWD data as well as state-of-the-art GP classification. Experimental results from an actual mine site show the effectiveness of our proposed approach.

Environmental Monitoring (EM) is typically performed using sensor networks that collect measurements in predefined static locations. The possibility of having one or more autonomous robots to perform this task increases versatility and reduces the number of necessary sensor nodes to cover the same area. However, several problems arise when making use of autonomous moving robots for EM. The main challenges are how to build an accurate spatial-temporal model while choosing locations for measuring the phenomenon. This paper addresses the problem by using Bayesian Optimisation for choosing sensing locations, and presents a new utility function that takes into account the distance travelled by a moving robot. The proposed methodology is tested in simulation and in a real environment. Compared to existing strategies, our approach exhibits slightly better accuracy in terms of RMSE error and considerably reduces the total distance travelled by the robot.

We address the problem of building a continuous occupancy representation of the environment with ranging sensors. Observations from such sensors provide two types of information: a line segment or a beam indicating no returns along them (free-space); a point or return at the end of the segment representing an occupied surface. To model these two types of observations in a principled statistical manner, we propose a novel methodology based on integral kernels. We show that integral kernels can be directly incorporated into a Gaussian process classification (GPC) framework to provide a continuous non-parametric Bayesian estimation of occupancy. Directly handling line segment and point observations avoids the need to discretise segments into points, reducing the computational cost of GPC inference and learning. We present experiments on 2D and 3D datasets demonstrating the benefits of the approach.

Gesture recognition is an important aspect of interpersonal social interaction. Developing a similar capacity in a robot will improve human-robot interaction. Various unsupervised clustering methods applied to clustering a set of dynamic human arm gestures are compared. Unsupervised clustering is important in gesture recognition as it imposes no a priori bound on the set of gestures. Results are compared using v-measure, a metric that allows differential weighting between clustering homogeneity and completeness. Experiments show that the best clustering method depends on the desired balance between homogeneity and completeness.

This paper addresses the problem of using visual information to estimate vehicle motion (a.k.a. visual odometry) from a machine learning perspective. The vast majority of current visual odometry algorithms are heavily based on geometry, using a calibrated camera model to recover relative translation (up to scale) and rotation by tracking image features over time. Our method eliminates the need for a parametric model by jointly learning how image structure and vehicle dynamics affect camera motion. This is achieved with a Gaussian Process extension, called Coupled GP, which is trained in a supervised manner to infer the underlying function mapping optical flow to relative translation and rotation. Matched image features parameters are used as inputs and linear and angular velocities are the outputs in our non-linear multi-task regression problem. We show here that it is possible, using a single uncalibrated camera and establishing a first-order temporal dependency between frames, to jointly estimate not only a full 6 DoF motion (along with a full covariance matrix) but also relative scale, a non-trivial problem in monocular configurations. Experiments were performed with imagery collected with an unmanned aerial vehicle (UAV) flying over a deserted area at speeds of 100-120 km/h and altitudes of 80-100 m, a scenario that constitutes a challenge for traditional visual odometry estimators.

Observing human motion patterns is informative for social robots that share the environment with people. This paper presents a methodology to allow a robot to navigate in a complex environment by observing pedestrian positional traces. A continuous probabilistic function is determined using Gaussian process learning and used to infer the direction a robot should take in different parts of the environment. The approach learns and filters noise in the data producing a smooth underlying function that yields more natural movements. Our method combines prior conventional planning strategies with most probable trajectories followed by people in a principled statistical manner, and adapts itself online as more observations become available. The use of learning methods are automatic and require minimal tuning as compared to potential fields or spline function regression. This approach is demonstrated testing in cluttered office and open forum environments using laser and vision sensing modalities. It yields paths that are similar to the expected human behaviour without any a priori knowledge of the environment or explicit programming.

This paper presents an automated approach for the classification of vegetation in natural environments based on high resolution aerial imagery acquired by a low flying Unmanned Aerial Vehicle (UAV). Standard colour and texture descriptors are extracted on a frame by frame basis to build a representation of appearance, which is probabilistically classified by a novel multi-class generalisation of the Gaussian Process (GP) developed for this work. A GP approach was selected for probabilistic outputs, and the ability to automatically determine the relevance of each input dimension to each of the C classes in the problem. When learning hyperparameters from N training examples, the new formulation scales at O(N ), rather than O(CN3) for the standard one-vs-all approach. The novel classification framework is trained and validated on a set of manual labels, and then queried to visualise a map of vegetation type under the UAV flight path. Mapping results are presented for a region of farmland in Northern Queensland, Australia that is infested with two invasive introduced tree species.

Obtaining a comprehensive model of large and complex terrain typically entails the use of both multiple sensory modalities and multiple data sets. This paper demonstrates the use of dependent Gaussian processes for data fusion in the context of large scale terrain modeling. Specifically, this paper derives and demonstrates the use of a non-stationary kernel (Neural Network) in this context. Experiments performed on multiple large scale (spanning about 5 sq km) 3D terrain data sets obtained from multiple sensory modalities (GPS surveys and laser scans) demonstrate the approach to data fusion and provide a preliminary demonstration of the superior modeling capability of Gaussian processes based on this kernel.

Autonomous operation of blast hole drill rigs requires monitoring of drilling parameters known as "Measurement While Drilling" (MWD) data. From these data, rock properties can be inferred. A supervised classification scheme is usually used to map MWD data inputs to rock type outputs given some labeled training data. However, the geology has no definite ground truth that can allow a reliable labeling of the training data, nor is there a clear input-output pair connection between the MWD data and the rock types. In this paper, an adaptive unsupervised approach is proposed to estimate the rock types in a data driven way by minimizing the entropy gradient of the characterizing measure "Optimized Adjusted Penetration Rate" (OAPR). Neither data labeling nor fixed model parameters are required because of the data driven nature of the algorithm. Experimental results illustrate the effectiveness of our solution.

This paper introduces a new approach to adoptively learn the dynamics of a robotic system. The methodology is based on maximizing the information gain from new observations while modeling the dynamics with a Multiple Output Gaussian Process (MOGP). High-dimensional state action spaces with unknown dependencies between inputs and outputs can be highly computationally expensive to learn. Gaussian process modeling is a Bayesian technique that naturally overcomes one of the most difficult problems in machine learning known as over-fitting. This makes it very appealing for on-line problems where testing multiple hypothesis is difficult. The computational cost of the learning task is reduced by having a smaller dataset of informative training points. Therefore we introduce a learning strategy capable of determining the most informative training set for the MOGP. This method can be implemented for learning the behavior of dynamic systems where due to their complexity and disturbances are infeasible to be analytically defined. The benefits of our approach are verified in two experiments: learning the dynamics of a cart pole system in simulation and the dynamics of a robotic blimp.

Multi-task learning remains a difficult yet important problem in machine learning. In Gaussian processes the main challenge is the definition of valid kernels (covariance functions) able to capture the relationships between different tasks. This paper presents a novel methodology to construct valid multi-task covariance functions (Mercer kernels) for Gaussian processes allowing for a combination of kernels with different forms. The method is based on Fourier analysis and is general for arbitrary stationary covariance functions. Analytical solutions for cross covariance terms between popular forms are provided including Matérn, squared exponential and sparse covariance functions. Experiments are conducted with both artificial and real datasets demonstrating the benefits of the approach. 1

This paper introduces a simple yet powerful data transformation strategy for kernel machines. Instead of adapting the parameters of the kernel function w.r.t. the given data (as in conventional methods), we adjust both the kernel hyper-parameters and the given data itself. Using this approach, the input data is transformed to be more representative of the assumptions encoded in the kernel function. A novel complex mapping is proposed to nonlinearly adjust the data. Optimization of the data transformation parameters is performed in two different manners. Firstly, the complex data mapping parameters and kernel hyper-parameters are selected separately, with the former guided by frequency metrics and the latter under the Bayesian framework. Next, the complex data mapping parameters and kernel hyper-parameters are optimized simultaneously in a Bayesian formulation by creating a new category of "integrated kernel" with the complex data mapping embedded. Experiments using Gaussian Process learning have shown that both methods improve the learning accuracy in either classification or regression tasks, with the complex mapping embedded kernel approach outperforming the separate complex mapping one.

This paper describes a method of incorporating sensor and localisation uncertainty into contextual occupancy maps to provide for robust mapping. This paper builds on a recently proposed application of the Gaussian process (GP) to occupancy mapping. An extension of GPs is employed which incorporates uncertain inputs into the covariance function. In turn, this allows statistically consistent, multi-resolution maps to be constructed which exploit the spatial inference properties of GPs while correctly accounting for sensor and localisation errors. Experiments are described, with both synthetic and real data, which show the benefits of complete uncertainty modeling and how contextual occupancy maps may be constructed by fusing data from different sensors on different robots in a common probabilistic representation.

The need for efficient monitoring of spatio-temporal dynamics in large environmental surveillance applications motivates the use of robotic sensors to achieve sufficient spatial and temporal coverage. A common approach in machine learning to model spatial dynamics is to use the nonparametric Bayesian framework known as Gaussian Processes (GPs) (c.f., [1]) which are fully specified by a mean and a covariance function. However, defining suitable covariance functions that are able to appropriately model complex space-time dependencies in the environment is a challenging task. In this paper, we develop a generic approach for constructing several classes of covariance functions for spatio-temporal GP modeling. The GP models are then extended to perform efficient path planning in continuous space while maximizing the information gain. Extensive empirical evaluation for the different classes of covariance functions using real world sensing datasets is discussed, including experiments on a tethered robotic system - Networked Info Mechanical System (NIMS).

This paper proposes a novel method for computing robot motion uncertainty from ranging sensor data. The method utilises the recently proposed CRF-Matching procedure which matches laser scans based on shape descriptors. Motion estimates are computed in a probabilistic framework by performing inference on a probabilistic graphical model. We propose an efficient sampling procedure for obtaining probable association hypothesis of the probabilistic graphical model. The hypothesis are used to generate estimates on the uncertainty of translational and rotational movements of the robot. Experiments demonstrate the benefits of the approach on simulated data sets and on laser scans from an urban environment. The approach is also combined with the well-established delayed-state information filter for a large-scale outdoor simultaneous localisation and mapping task.

This paper presents a novel approach to data fusion for stochastic processes that model spatial data. It addresses the problem of data fusion in the context of large scale terrain modeling for a mobile robot. Building a model of large scale and complex terrain that can adequately handle uncertainty and incompleteness in a statistically sound manner is a very challenging problem. To obtain a comprehensive model of such terrain, typically, multiple sensory modalities as well as multiple data sets are required. This work uses Gaussian processes to model large scale terrain. The model naturally provides a multi-resolution representation of space, incorporates and handles uncertainties appropriately and copes with incompleteness of sensory information. Gaussian process regression techniques are applied to estimate and interpolate (to fill gaps in unknown areas) elevation information across the field. In this work, the GP modeling approach is extended to fuse multiple, multi-modal data sets to obtain a best estimate of the elevation given the individual data sets. The individual data sets are treated as different noisy samples of the same underlying terrain. Experiments performed on sparse GPS based survey data and dense laser scanner data taken at mine-sites are reported.

This paper presents a novel framework for integrating fundamental tasks in robotic navigation through a statistical inference procedure. A probabilistic model that jointly reasons about scan-matching, moving object detection and their motion estimation is developed. Scan-matching and moving object detection are two important problems for full autonomy of robotic systems in complex dynamic environments. Popular techniques for solving these problems usually address each task in turn disregarding important dependencies. The model developed here jointly reasons about these tasks by performing inference in a probabilistic graphical model. It allows different but related problems to be expressed in a single framework. The experiments demonstrate that jointly reasoning results in better estimates for both tasks compared to solving the tasks individually.

A crucial component of an autonomous mine is the ability to infer rock types from mechanical measurements of a drill rig. The major difficulty lies in that there is not a clear one to one correspondence between the mechanical measurements and the rock type due to the mechanical noise as well as the variety of the rock geology. This paper proposes a novel wavelet feature space projection approach to robustly classify rock types from drilling data with Gaussian Process classification. Instead of applying Gaussian Process classifier directly to the given measurement pieces, a group of wavelet features are extracted from the neighboring region of a specific data point. Gaussian Process classification is then carried out on the new extracted wavelet features. By putting neighboring data points into consideration rather than dealing with each data point individually, the underlying pattern can be better captured and more robust to noise and data variations. Experimental results on synthetic data as well as varied real world drilling data have shown the effectiveness of our approach.

Place recognition is a challenging task in any SLAM system. Algorithms based on visual appearance are becoming popular to detect locations already visited, also known as loop closures, because cameras are easily available and provide rich scene detail. These algorithms typically result in pairs of images considered depicting the same location. To avoid mismatches, most of them rely on epipolar geometry to check spatial consistency. In this paper we present an alternative system that makes use of stereo vision and combines two complementary techniques: bag-of-words to detect loop closing candidate images, and conditional random fields to discard those which are not geometrically consistent. We evaluate this system in public indoor and outdoor datasets from the Rawseeds project, with hundred-metre long trajectories. Our system achieves more robust results than using spatial consistency based on epipolar geometry.

Terrain modeling remains a challenging yet key component for the deployment of ground robots to the field. The difficulty arrives from the variability of terrain shapes, sparseness of the data, and high degree uncertainty often encountered in large, unstructured environments. This paper presents significant advances to data fusion for stochastic processes modeling spatial data, demonstrated in large-scale terrain modeling tasks. We explore dependent Gaussian processes to provide a multi-resolution representation of space and associated uncertainties, while integrating sensors from different modalities. Experiments performed on multiple multi-modal datasets (3D laser scans and GPS) demonstrate the approach for terrains of about 5 km2.

Measurement-while-drilling (MWD) data recorded from drill rigs can provide a valuable estimation of the type and strength of the rocks being drilled. Typical MWD sensors include bit pressure, rotation pressure, pull-down pressure, pull-down rate, and head speed. This letter presents an empirical comparison of the statistical performance, ease of implementation, and computational efficiency associated with three machine-learning techniques. A recently proposed method, boosting, is compared with two well-established methods, neural networks and fuzzy logic, used as benchmarks. MWD data were acquired from blast holes at an iron ore mine in Western Australia. The boreholes intersected a number of rock types including shale, iron ore, and banded iron formation. Boosting and neural networks presented the best performance overall. However, from the viewpoint of implementation simplicity and computational load, boosting outperformed the other two methods.

Analysis of drill performance data provides a powerful method for estimating subsurface geology. While there have been studies relating such measurement-while-drilling (MWD) parameters to rock properties, none of them has attempted to model context, that is, to associate local measurements with measurements obtained in neighbouring regions. This paper proposes a novel approach to infer geology from drill measurements by incorporating spatial relationships through a Conditional Random Field (CRF) framework. A boosting algorithm is used as a local classifier mapping drill measurements to corresponding geological categories. The CRF then uses this local information in conjunction with neighbouring measurements to jointly reason about their categories. Model parameters are learned from training data by maximizing the pseudo-likelihood. The probability distribution of classified borehole sections is calculated using belief propagation. We present experimental results of applying the method to MWD data collected from a semi-autonomous drill rig at an iron ore mine in Western Australia.

Despite significant developments in the simultaneous localisation and mapping (SLAM) problem, loop closure detection is still challenging in large scale unstructured environments. Current solutions rely on heuristics that lack generalisation properties, in particular when range sensors are the only source of information about the robot's surrounding environment. This paper presents a machine learning approach for the loop closure detection problem using range sensors. A binary classifier based on boosting is used to detect loop closures. The algorithm performs robustly, even under potential occlusions and significant changes in rotation and translation. We developed a number of features, extracted from range data, that are invariant to rotation. Additionally, we present a general framework for scan-matching SLAM in outdoor environments. Experimental results in large scale urban environments show the robustness of the approach, with a detection rate of 85% and a false alarm rate of only 1%. The proposed algorithm can be computed in real-time and achieves competitive performance with no manual specification of thresholds given the features.

In this paper we introduce a new statistical modeling technique for building occupancy maps. The problem of mapping is addressed as a classification task where the robot's environment is classified into regions of occupancy and unoccupancy. Our model provides both a continuous representation of the robot's surroundings and an associated predictive variance. This is obtained by employing a Gaussian process as a non-parametric Bayesian learning technique to exploit the fact that real-world environments inherently possess structure. This structure introduces a correlation between points on the map which is not accounted for by many common mapping techniques such as occupancy grids. Using a trained neural network covariance function to model the highly non-stationary datasets, it is possible to generate accurate representations of large environments at resolutions which suit the desired applications while also providing inferences into occluded regions, between beams, and beyond the range of the sensor, even with relatively few sensor readings. We demonstrate the benefits of our approach in a simulated data set with known ground-truth, and in an outdoor urban environment covering an area of 120,000 m2.

This paper addresses the problem of large scale terrain modeling for a mobile robot. Building a model of large scale terrain data that can adequately handle uncertainty and incompleteness in a statistically sound way is a very challenging problem. This work proposes the use of Gaussian processes as models of large scale terrain. The proposed model naturally provides a multi-resolution representation of space, incorporates and handles uncertainties aptly and copes with incompleteness of sensory information. Gaussian process regression techniques are applied to estimate and interpolate (to fill gaps in unknown areas) elevation information across the field. The estimates obtained are the best linear unbiased estimates for the data under consideration. A single non-stationary (neural network) Gaussian process is shown to be powerful enough to model large and complex terrain, handling issues relating to discontinuous data effectively. A local approximation methodology based on KD-trees is also proposed in order to ensure local smoothness and yet preserve the characteristic features of rich and complex terrain data. The use of the local approximation technique based on KD-trees further addresses concerns relating to the scalability of the proposed approach for large data sets. Experiments performed on sparse GPS based survey data as well as dense laser scanner data taken at different mine-sites are reported in support of these claims.

Moving objects are present in many robotic applications. An accurate detection and motion estimation of these objects can be crucial for the success and safety of the robot and people surrounding it. This paper presents a new probabilistic framework for clustering dependent or relational data, applied to the problem of motion clustering and estimation. While conventional techniques such as scan differencing perform well in many cases, they usually assume that a good pose estimate is available and fail when points belonging to dynamic objects show some overlap in consecutive readings. The technique proposed, CRF-Clustering, by explicitly reasoning about the underlying motion of the object, is able to deal with poor initial motion estimate and overlapping points. Moreover, it is able to consider the dependencies between neighbor points in the scans to reduce the noise in the clustering assignment. The model parameters can be estimated from labeled data in a statistically sound learning procedure. Experiments show that CRF-Clustering is able to detect moving objects, cluster them and estimate their motion.

This paper addresses the long-standing problem of feature representation in the natural world for autonomous navigation systems. The proposed representation combines Isomap, which is a nonlinear manifold learning algorithm, with expectation maximization, which is a statistical learning scheme. The representation is computed off-line and results in a compact, nonlinear, non-Gaussian sensor likelihood model. This model can be easily integrated into estimation algorithms for navigation and tracking. The compactness of the model makes it especially attractive for deployment in decentralized sensor networks. Real sensory data from unstructured terrestrial and underwater environments are used to demonstrate the versatility of the computed likelihood model. The experimental results show that this approach can provide consistent models of natural environments to facilitate complex visual tracking and data-association problems.

In this paper, simultaneous localisation and mapping (SLAM) is combined with landmark recognition to close large loops in unstructured, outdoor environments. Camera and laser information are fused to recognise and create appearance models for landmarks. The representation is obtained through a non-linear probabilistic regression model encoding a neighbourhood preserving dimensionality reduction. A new data association algorithm is proposed where landmarks are associated based on both position and appearance. The resulting system is more robust and able to recover from possible misassociations. Experiments demonstrate the benefits of this approach in challenging problems involving mapping with large loop closings in irregular terrain, and with dynamic objects.

This paper presents a general framework for multi-sensor object recognition through a discriminative probabilistic approach modelling spatial and temporal correlations. The algorithm is developed in the context of Conditional Random Fields (CRFs) trained with virtual evidence boosting. The resulting system is able to integrate arbitrary sensor information and incorporate features extracted from the data. The spatial relationships captured by are further integrated into a smoothing algorithm to improve recognition over time. We demonstrate the benefits of modelling spatial and temporal relationships for the problem of detecting cars using laser and vision data in outdoor environments.

In this paper, we combined the application of a non-linear dimensionality reduction technique, isomap, with expectation maximisation in graphical probabilistic models for learning and classification of hyperspectral image. Hyperspectral image spectroscopy gives much greater information content per pixel on the image than a normal colour image. This should greatly help with the autonomous identification of natural and man-made objects in unfamiliar terrains for robotic vehicles. However, the large information content of such data makes interpretation of hyperspectral images time-consuming and user-intensive. Isomap is used to find the underlying manifold of the training data. This low dimensional representation of the hyperspectral data facilitates the learning of a mixture of linear models representation similar to a mixture of factor analysers, the joint probability distributions of the model can be calculated offline. The learnt model is then applied to the hyperspectral image at run-time and data classification can be performed. We also show the comparison with results from standard techniques

This paper presents a unified framework for online visual perception. The twin problems of visual feature extraction and representation are explicitly addressed. Simple paradigms for supervised and unsupervised feature extraction are presented to represent the extremes in visual perception system design. Visual feature representation is addressed through a combination of isomap, a non-linear dimensionality reduction algorithm, and expectation maximization (EM), a statistical learning scheme. A joint probability distribution of this representation is computed offline based on existing training data. Example applications based on real visual data from heterogenous, unstructured environments demonstrate the versatility of the generative models

This paper presents an algorithm for recognition and segmentation of natural features in unstructured environments. By providing a Bayesian solution for the density estimation problem, the algorithm needs significantly less training data than conventional techniques and is applicable to different environments. The algorithm is based on colour and wavelet convolution of image patches to model the information contained in natural features. Dimensionality reduction techniques are applied to map data points to a lower dimensional space where Bayesian density estimation is computed. Experiments were performed in underwater, aerial and terrestrial domains demonstrating the accuracy and generalisation properties of the algorithm for recognition and segmentation. Comparisons with conventional density estimation techniques are provided to illustrate the benefits of the new approach

In this paper, we use the Bayesian Structural EM algorithm as a classification method to learn and interpret hyperspectral sensor data in robotic planetary missions. Hyperspectral image spectroscopy is an emerging technique for geological investigations from airborne or orbital sensors. Many spacecraft carry spectroscopic equipment as wavelengths outside the visible light in the electromagnetic spectrum give much greater information about an object. The algorithm presented combines the standard Expectation Maximisation (EM), which optimises parameters, with structure search for model selection. We use the Bayesian Information Criterion (BIC) score to learn the network struc ture. The procedure only converges to a local maxima, thus requiring a good initial graph structure. Two initial structures are used: the Naïve Bayes, and the Tree-Augmented-Naïve Bayes structures. Our preliminary experiments show that the former results in a structure that can correctly determine the presence and types of minerals with merely 13% accuracy while the latter results in a structure that has approximately 94% accuracy.

This paper presents a robust stochastic framework for the incorporation of visual observations into conventional estimation, data fusion, navigation and control algorithms. The representation combines Isomap, a non-linear dimensionality reduction algorithm, with expectation maximization, a statistical learning scheme. The joint probability distribution of this representation is computed offline based on existing training data. The training phase of the algorithm results in a nonlinear and non-Gaussian likelihood model of natural features conditioned on the underlying visual states. This generative model can be used online to instantiate likelihoods corresponding to observed visual features in real-time. The instantiated likelihoods are expressed as a Gaussian mixture model and are conveniently integrated within existing non-linear filtering algorithms. Example applications based on real visual data from heterogenous, unstructured environments demonstrate the versatility of the generative models.