In this paper, we present a lab setup to measure accurate 3D gaze positions of a person during unconstrained navigation and interaction in an environment with known geometry. We evaluate the accuracy of distance measurements up to 5.7m and compare them with baseline vergence-point estimation of the Tobii Pro Glasses 3.

Accurate human activity and trajectory prediction are crucial for ensuring safe and reliable human-robot interactions in dynamic environments, such as industrial settings, with mobile robots. Datasets with fine-grained action labels for moving people in industrial environments with mobile robots are scarce, as most existing datasets focus on social navigation in public spaces. This paper introduces the THÖR-MAGNI Act dataset, a substantial extension of the THÖR-MAGNI dataset, which captures participant movements alongside robots in diverse semantic and spatial contexts. THOR-MAGNI Act provides 8.3 hours of manually labeled participant actions derived from egocentric videos recorded via eye-tracking glasses. These actions, aligned with the provided THOR-MAGNI motion cues, follow a long-tailed distribution with diversified acceleration, velocity, and navigation distance profiles. We demonstrate the utility of THÖR-MAGNI Act for two tasks: action-conditioned trajectory prediction and joint action and trajectory prediction. We propose two efficient transformer-based models that outperform the baselines to address these tasks. These results underscore the potential of THOR-MAGNI Act to develop predictive models for enhanced human-robot interaction in complex environments.

To achieve natural and intuitive interaction with people, HRI frameworks combine a wide array of methods for human perception, intention communication, human-aware navigation and collaborative action. In practice, when encountering unpredictable behavior of people or unexpected states of the environment, these frameworks may lack the ability to dynamically recognize such states, adapt and recover to resume the interaction. Large Language Models (LLMs), owing to their advanced reasoning capabilities and context retention, present a promising solution for enhancing robot adaptability. This potential, however, may not directly translate to improved interaction metrics. This paper considers a representative interaction with an industrial robot involving approach, instruction, and object manipulation, implemented in two conditions: (1) fully scripted and (2) including LLM-enhanced responses. We use gaze tracking and questionnaires to measure the participants' task efficiency, engagement, and robot perception. The results indicate higher SUbjective ratings for the LLM condition, but objective metrics show that the scripted condition performs comparably, particularly in efficiency and focus during simple tasks. We also note that the scripted condition may have an edge over LLM-enhanced responses in terms of response latency and energy consumption, especially for trivial and repetitive interactions.

Large Language Models (LLMs) excel at generating contextually relevant text but lack logical reasoning abilities. They rely on statistical patterns rather than logical inference, making them unreliable for structured decision-making. Integrating LLMs with task planning can address this limitation by combining their natural language understanding with the precise, goal-oriented reasoning of planners. This paper introduces ViPlan, a hybrid system that leverages Vision Language Models (VLMs) to extract high-level semantic information from visual and textual inputs while integrating classical planners for logical reasoning. ViPlan utilizes VLMs to generate syntactically correct and semantically meaningful PDDL problem files from images and natural language instructions, which are then processed by a task planner to generate an executable plan. The entire process is embedded within a behavior tree framework, enhancing efficiency, reactivity, replanning, modularity, and flexibility. The generation and planning capabilities of ViPlan are empirically evaluated with simulated and real-world experiments.

Simultaneous Localization and Mapping (SLAM) allows mobile robots to navigate without external positioning systems or pre-existing maps. Radar is emerging as a valuable sensing tool, especially in vision-obstructed environments, as it is less affected by particles than lidars or cameras. Modern 4D imaging radars provide three-dimensional geometric information and relative velocity measurements, but they bring challenges, such as a small field of view and sparse, noisy point clouds. Detecting loop closures in SLAM is critical for reducing trajectory drift and maintaining map accuracy. However, the directional nature of 4D radar data makes identifying loop closures, especially from reverse viewpoints, difficult due to limited scan overlap. This article explores using 4D radar for loop closure in SLAM, focusing on similar and opposing viewpoints. We generate submaps for a denser environment representation and use introspective measures to reject false detections in feature-degenerate environments. Our experiments show accurate loop closure detection in geometrically diverse settings for both similar and opposing viewpoints, improving trajectory estimation with up to 82% improvement in ATE and rejecting false positives in self-similar environments.

Gas source localization in complex environments is critical for applications such as environmental monitoring, industrial safety, and disaster response. Traditional methods often struggle with the challenges posed by a lack of environmental topography integration, especially when interactions between wind and obstacles distort gas dispersion patterns. In this paper, we propose a deep learning-based approach, which leverages spatial context and environmental mapping to enhance gas source localization. By integrating Simultaneous Localization and Mapping (SLAM) with a U-Net-based model, our method predicts the likelihood of gas source locations by analyzing gas sensor data, wind flow, and topography of the environment represented by a 2D occupancy map. We demonstrate the efficacy of our approach using a wheeled robot equipped with a photoionization detector, a LIDAR, and an anemometer, in various scenarios with dynamic wind fields and multiple obstacles. The results show that our approach can robustly locate gas sources, even in challenging environments with fluctuating wind directions, outperforming conventional methods by utilizing topography contextual information. This study underscores the importance of topographical context in gas source localization and offers a flexible and robust solution for real-world applications. Data and code are publicly available.

Maps of dynamics are effective representations of motion patterns learned from prior observations, with recent research demonstrating their ability to enhance various downstream tasks such as human-aware robot navigation, long-term human motion prediction, and robot localization. Current advancements have primarily concentrated on methods for learning maps of human flow in environments where the flow is static, i.e., not assumed to change over time. In this paper we propose an online update method of the CLiFF-map (an advanced map of dynamics type that models motion patterns as velocity and orientation mixtures) to actively detect and adapt to human flow changes. As new observations are collected, our goal is to update a CLiFF-map to effectively and accurately integrate them, while retaining relevant historic motion patterns. The proposed online update method maintains a probabilistic representation in each observed location, updating parameters by continuously tracking sufficient statistics. In experiments using both synthetic and real-world datasets, we show that our method is able to maintain accurate representations of human motion dynamics, contributing to high performance flow-compliant planning downstream tasks, while being orders of magnitude faster than the comparable baselines.

The rapid development of Large Language Models (LLMs) creates an exciting potential for flexible, general knowledge-driven Human-Robot Interaction (HRI) systems for assistive robots. Existing HRI systems demonstrate great progress in interpreting and following user instructions, action generation, and robot task solving. On the other hand, bi-directional, multimodal, and context-aware support of the user in collaborative tasks still remains an open challenge. In this paper, we present a gaze- and speech-informed interface to the assistive robot, which is able to perceive the working environment from multiple vision inputs and support the dynamic user in their tasks. Our system is designed to be modular and transferable to adapt to diverse tasks and robots, and it is real-time capable due to the language-based interaction state representation and fast on-board perception modules. Its development was supported by multiple public dissemination events, contributing important considerations for improved robustness and user experience. Furthermore, in a lab study, we compare the performance and user ratings of our system with those of a traditional scripted HRI pipeline. Our findings indicate that an LLM-based approach enhances adaptability and marginally improves user engagement and task execution metrics but may produce redundant output, while a scripted pipeline is well suited for more straightforward tasks.

Neural implicit surface representations are currently receiving a lot of interest as a means to achieve high-fidelity surface reconstruction at a low memory cost, compared to traditional explicit representations. However, state-of-the-art methods still struggle with excessive memory usage and non-smooth surfaces. This is particularly problematic in large-scale applications with sparse inputs, as is common in robotics use cases. To address these issues, we first introduce a sparse structure, tri-quadtrees, which represents the environment using learnable features stored in three planar quadtree projections. Secondly, we concatenate the learnable features with a Fourier feature positional encoding. The combined features are then decoded into signed distance values through a small multilayer perceptron. We demonstrate that this approach facilitates smoother reconstruction with a higher completion ratio with fewer holes. Compared to two recent baselines, one implicit and one explicit, our approach requires only 10%–50% as much memory, while achieving competitive quality. The code is released on https://github.com/ljjTYJR/3QFP.

Long-term human motion prediction (LHMP) is essential for safely operating autonomous robots and vehicles in populated environments. It is fundamental for various applications, including motion planning, tracking, human-robot interaction and safety monitoring. However, accurate prediction of human trajectories is challenging due to complex factors, including, for example, social norms and environmental conditions. The influence of such factors can be captured through Maps of Dynamics (MoDs), which encode spatial motion patterns learned from (possibly scattered and partial) past observations of motion in the environment and which can be used for data-efficient, interpretable motion prediction (MoD-LHMP). To address the limitations of prior work, especially regarding accuracy and sensitivity to anomalies in long-term prediction, we propose the Laminar Component Enhanced LHMP approach (LaCE-LHMP). Our approach is inspired by data-driven airflow modelling, which estimates laminar and turbulent flow components and uses predominantly the laminar components to make flow predictions. Based on the hypothesis that human trajectory patterns also manifest laminar flow (that represents predictable motion) and turbulent flow components (that reflect more unpredictable and arbitrary motion), LaCE-LHMP extracts the laminar patterns in human dynamics and uses them for human motion prediction. We demonstrate the superior prediction performance of LaCE-LHMP through benchmark comparisons with state-of-the-art LHMP methods, offering an unconventional perspective and a more intuitive understanding of human movement patterns.

We propose a dense RGBD SLAM system based on 3D Gaussian Splatting that provides metrically accurate pose tracking and visually realistic reconstruction. To this end, we first propose a Gaussian densification strategy based on the rendering loss to map unobserved areas and refine reobserved areas. Second, we introduce extra regularization parameters to alleviate the "forgetting" problem during contiunous mapping, where parameters tend to overfit the latest frame and result in decreasing rendering quality for previous frames. Both mapping and tracking are performed with Gaussian parameters by minimizing re-rendering loss in a differentiable way. Compared to recent neural and concurrently developed Gaussian splatting RGBD SLAM baselines, our method achieves state-of-the-art results on the synthetic dataset Replica and competitive results on the real-world dataset TUM. The code is released on https://github.com/ljjTYJR/HF-SLAM.

The human gaze is an important cue to signal intention, attention, distraction, and the regions of interest in the immediate surroundings. Gaze tracking can transform how robots perceive, understand, and react to people, enabling new modes of robot control, interaction, and collaboration. In this paper, we use gaze tracking data from a rich dataset of human motion (THÖR-MAGNI) to investigate the coordination between gaze direction and head rotation of humans engaged in various indoor activities involving navigation, interaction with objects, and collaboration with a mobile robot. In particular, we study the spread and central bias of fixations in diverse activities and examine the correlation between gaze direction and head rotation. We introduce various human motion metrics to enhance the understanding of gaze behavior in dynamic interactions. Finally, we apply semantic object labeling to decompose the gaze distribution into activity-relevant regions.

Object detection is crucial in diverse autonomous systems like surveillance, autonomous driving, and driver assistance, ensuring safety by recognizing pedestrians, vehicles, traffic lights, and signs. However, adverse weather conditions such as snow, fog, and rain pose a challenge, affecting detection accuracy and risking accidents and damage. This clearly demonstrates the need for robust object detection solutions that work in all weather conditions. We employed three strategies to enhance deep learning-based object detection in adverse weather: training on real-world all-weather images, training on images with synthetic augmented weather noise, and integrating object detection with adverse weather image denoising. The synthetic weather noise is generated using analytical methods, GAN networks, and style-transfer networks. We compared the performance of these strategies by training object detection models using real-world all-weather images from the BDD100K dataset and for assessment employed unseen real-world adverse weather images. Adverse weather denoising methods were evaluated by denoising real-world adverse weather images and the results of object detection on denoised and original noisy images were compared. We found that the model trained using all-weather real-world images performed best, while the strategy of doing object detection on denoised images performed worst.

Human motion prediction is important for mobile service robots and intelligent vehicles to operate safely and smoothly around people. The more accurate predictions are, particularly over extended periods of time, the better a system can, e.g., assess collision risks and plan ahead. In this paper, we propose to exploit maps of dynamics (MoDs, a class of general representations of place-dependent spatial motion patterns, learned from prior observations) for long-term human motion prediction (LHMP). We present a new MoD-informed human motion prediction approach, named CLiFF-LHMP, which is data efficient, explainable, and insensitive to errors from an upstream tracking system. Our approach uses CLiFF -map, a specific MoD trained with human motion data recorded in the same environment. We bias a constant velocity prediction with samples from the CLiFF-map to generate multi-modal trajectory predictions. In two public datasets we show that this algorithm outperforms the state of the art for predictions over very extended periods of time, achieving 45 % more accurate prediction performance at 50s compared to the baseline.

Robots are increasingly used in shared environments with humans, making effective communication a necessity for successful human-robot interaction. In our work, we study a crucial component: active communication of robot intent. Here, we present an anthropomorphic solution where a humanoid robot communicates the intent of its host robot acting as an “Anthropomorphic Robotic Mock Driver” (ARMoD). We evaluate this approach in two experiments in which participants work alongside a mobile robot on various tasks, while the ARMoD communicates a need for human attention, when required, or gives instructions to collaborate on a joint task. The experiments feature two interaction styles of the ARMoD: a verbal-only mode using only speech and a multimodal mode, additionally including robotic gaze and pointing gestures to support communication and register intent in space. Our results show that the multimodal interaction style, including head movements and eye gaze as well as pointing gestures, leads to more natural fixation behavior. Participants naturally identified and fixated longer on the areas relevant for intent communication, and reacted faster to instructions in collaborative tasks. Our research further indicates that the ARMoD intent communication improves engagement and social interaction with mobile robots in workplace settings.

This article presents an accurate, highly efficient, and learning-free method for large-scale odometry estimation using spinning radar, empirically found to generalize well across very diverse environments—outdoors, from urban to woodland, and indoors in warehouses and mines—without changing parameters. Our method integrates motion compensation within a sweep with one-to-many scan registration that minimizes distances between nearby oriented surface points and mitigates outliers with a robust loss function. Extending our previous approach conservative filtering for efficient and accurate radar odometry (CFEAR), we present an in-depth investigation on a wider range of datasets, quantifying the importance of filtering, resolution, registration cost and loss functions, keyframe history, and motion compensation. We present a new solving strategy and configuration that overcomes previous issues with sparsity and bias, and improves our state-of-the-art by 38%, thus, surprisingly, outperforming radar simultaneous localization and mapping (SLAM) and approaching lidar SLAM. The most accurate configuration achieves 1.09% error at 5 Hz on the Oxford benchmark, and the fastest achieves 1.79% error at 160 Hz.

Human motion trajectory prediction, an essential task for autonomous systems in many domains, has been on the rise in recent years. With a multitude of new methods proposed by different communities, the lack of standardized benchmarks and objective comparisons is increasingly becoming a major limitation to assess progress and guide further research. Existing benchmarks are limited in their scope and flexibility to conduct relevant experiments and to account for contextual cues of agents and environments. In this paper we present Atlas, a benchmark to systematically evaluate human motion trajectory prediction algorithms in a unified framework. Atlas offers data preprocessing functions, hyperparameter optimization, comes with popular datasets and has the flexibility to setup and conduct underexplored yet relevant experiments to analyze a method’s accuracy and robustness. In an example application of Atlas, we compare five popular model- and learning-based predictors and find that, when properly applied, early physics-based approaches are still remarkably competitive. Such results confirm the necessity of benchmarks like Atlas.

State of the art mapping algorithms can produce high-quality maps. However, they are still vulnerable to clutter and outliers which can affect map quality and in consequence hinder the performance of a robot, and further map processing for semantic understanding of the environment. This paper presents ROSE, a method for building-level structure detection in robotic maps. ROSE exploits the fact that indoor environments usually contain walls and straight-line elements along a limited set of orientations. Therefore metric maps often have a set of dominant directions. ROSE extracts these directions and uses this information to segment the map into structure and clutter through filtering the map in the frequency domain (an approach substantially underutilised in the mapping applications). Removing the clutter in this way makes wall detection (e.g. using the Hough transform) more robust. Our experiments demonstrate that (1) the application of ROSE for decluttering can substantially improve structural feature retrieval (e.g., walls) in cluttered environments, (2) ROSE can successfully distinguish between clutter and structure in the map even with substantial amount of noise and (3) ROSE can numerically assess the amount of structure in the map.

This paper presents an accurate, highly efficient and learning free method for large-scale radar odometry estimation. By using a simple filtering technique that keeps the strongest returns, we produce a clean radar data representation and reconstruct surface normals for efficient and accurate scan matching. Registration is carried out by minimizing a point-to-line metric and robustness to outliers is achieved using a Huber loss. Drift is additionally reduced by jointly registering the latest scan to a history of keyframes. We found that our odometry pipeline generalize well to different sensor models and datasets without changing a single parameter. We evaluate our method in three widely different environments and demonstrate an improvement over spatially cross validated state-of-the-art with an overall translation error of 1.76% in a public urban radar odometry benchmark, running merely on a single laptop CPU thread at 55 Hz.

Models of human behaviour, such as pedestrian flows, are beneficial for safe and efficient operation of mobile robots. We present a new methodology for benchmarking of pedestrian flow models based on the afforded safety of robot navigation in human-populated environments. While previous evaluations of pedestrian flow models focused on their predictive capabilities, we assess their ability to support safe path planning and scheduling. Using real-world datasets gathered continuously over several weeks, we benchmark state-of-the-art pedestrian flow models, including both time-averaged and time-sensitive models. In the evaluation, we use the learned models to plan robot trajectories and then observe the number of times when the robot gets too close to humans, using a predefined social distance threshold. The experiments show that while traditional evaluation criteria based on model fidelity differ only marginally, the introduced criteria vary significantly depending on the model used, providing a natural interpretation of the expected safety of the system. For the time-averaged flow models, the number of encounters increases linearly with the percentage operating time of the robot, as might be reasonably expected. By contrast, for the time-sensitive models, the number of encounters grows sublinearly with the percentage operating time, by planning to avoid congested areas and times.

In this paper the intersection problem in human-robot systems with respect to noisy information is discussed. The interaction between humans and mobile robots in shared areas requires a high level of safety especially at the intersections of trajectories. We discuss the intersection problem with respect to noisy information on the basis of an analytic geometrical model and its TS fuzzy version. The transmission of a 2-dimensional Gaussian noise signal, in particular information on human and robot orientations, through a non-linear static system and its fuzzy version, will be described. We discuss the problem: Given the parameters of the input distributions, find the parameters of the output distributions.

The interaction between humans and mobile robots in shared areas requires a high level of safety especially at the crossings of the trajectories of humans and robots. We discuss the intersection calculation and its fuzzy version in the context of human-robot navigation with respect to noise information. Based on known parameters of the Gaussian input distributions at the orientations of human and robot the parameters of the output distributions at the intersection are to be found by analytical and fuzzy calculation. Furthermore the inverse task is discussed where the parameters of the output distributions are given and the parameters of the input distributions are searched. For larger standard deviations of the orientation signals we suggest mixed Gaussian models as approximation of nonlinear distributions.

In this paper, we propose a novel sub-pixel motion extraction method, called as Two Dimensional Spatial Keystone Transform (2DS-KST), for the motion detection and estimation from successive noisy Occupancy Grid Maps (OGMs). It extends the KST in radar imaging or motion compensation to 2D real spatial case, based on multiple hypotheses about possible directions of moving obstacles. Simulation results show that 2DS-KST has a good performance on the extraction of sub-pixel motions in very noisy environment, especially for those slowly moving obstacles.

Voxel volumes are simple to implement and lend themselves to many of the tools and algorithms available for 2D images. However, the additional dimension of voxels may be costly to manage in memory when mapping large spaces at high resolutions. While lowering the resolution and using interpolation is common work-around, in the literature we often find that authors either use trilinear interpolation or nearest neighbors and rarely any of the intermediate options. This paper presents a survey of geometric interpolation methods for voxel-based map representations. In particular we study the truncated signed distance field (TSDF) and the impact of using fewer than 8 samples to perform interpolation within a depth-camera pose tracking and mapping scenario. We find that lowering the number of samples fetched to perform the interpolation results in performance similar to the commonly used trilinear interpolation method, but leads to higher frame-rates. We also report that lower bit-depth generally leads to performance degradation, though not as much as may be expected, with voxels containing as few as 3 bits sometimes resulting in adequate estimation of camera trajectories.

How to divide floor plans or navigation maps into semantic representations, such as rooms and corridors, is an important research question in fields such as human-robot interaction, place categorization, or semantic mapping. While most works focus on segmenting robot built maps, those are not the only types of map a robot, or its user, can use. We present a method for segmenting maps from different modalities, focusing on robot built maps and hand-drawn sketch maps, and show better results than state of the art for both types. Our method segments the map by doing a convolution between the distance image of the map and a circular kernel, and grouping pixels of the same value. Segmentation is done by detecting ripple-like patterns where pixel values vary quickly, and merging neighboring regions with similar values. We identify a flaw in the segmentation evaluation metric used in recent works and propose a metric based on Matthews correlation coefficient (MCC). We compare our results to ground-truth segmentations of maps from a publicly available dataset, on which we obtain a better MCC than the state of the art with 0.98 compared to 0.65 for a recent Voronoi-based segmentation method and 0.70 for the DuDe segmentation method. We also provide a dataset of sketches of an indoor environment, with two possible sets of ground truth segmentations, on which our method obtains an MCC of 0.56 against 0.28 for the Voronoi-based segmentation method and 0.30 for DuDe.

Accurate long-term prediction of human motion in populated spaces is an important but difficult task for mobile robots and intelligent vehicles. What makes this task challenging is that human motion is influenced by a large variety of factors including the person's intention, the presence, attributes, actions, social relations and social norms of other surrounding agents, and the geometry and semantics of the environment. In this paper, we consider the problem of computing human motion predictions that account for such factors. We formulate the task as an MDP planning problem with stochastic policies and propose a weighted random walk algorithm in which each agent is locally influenced by social forces from other nearby agents. The novelty of this paper is that we incorporate social grouping information into the prediction process reflecting the soft formation constraints that groups typically impose to their members' motion. We show that our method makes more accurate predictions than three state-of-the-art methods in terms of probabilistic and geometrical performance metrics.

In this paper we address the problem of flow-aware trajectory planning in dynamic environments considering flow model uncertainty. Flow-aware planning aims to plan trajectories that adhere to existing flow motion patterns in the environment, with the goal to make robots more efficient, less intrusive and safer. We use a statistical model called CLiFF-map that can map flow patterns for both continuous media and discrete objects. We propose novel cost and biasing functions for an RRT* planning algorithm, which exploits all the information available in the CLiFF-map model, including uncertainties due to flow variability or partial observability. Qualitatively, a benefit of our approach is that it can also be tuned to yield trajectories with different qualities such as exploratory or cautious, depending on application requirements. Quantitatively, we demonstrate that our approach produces more flow-compliant trajectories, compared to two baselines.

Environment monitoring remains a major challenge for mobile robots, especially in densely cluttered or highly populated dynamic environments, where uncertainties originated from environment and sensor significantly challenge the robot's perception. This paper proposes an effective occupancy filtering method called the dual probability hypothesis density (DPHD) filter, which models uncertain phenomena, such as births, deaths, occlusions, false alarms, and miss detections, by using random finite sets. The key insight of our method lies in the connection of the idea of dynamic occupancy with the concepts of the phase space density in gas kinetic and the PHD in multiple target tracking. By modeling the environment as a mixture of static and dynamic parts, the DPHD filter separates the dynamic part from the static one with a unified filtering process, but has a higher computational efficiency than existing Bayesian Occupancy Filters (BOFs). Moreover, an adaptive newborn function and a detection model considering occlusions are proposed to improve the filtering efficiency further. Finally, a hybrid particle implementation of the DPHD filter is proposed, which uses a box particle filter with constant discrete states and an ordinary particle filter with a time-varying number of particles in a continuous state space to process the static part and the dynamic part, respectively. This filter has a linear complexity with respect to the number of grid cells occupied by dynamic obstacles. Real-world experiments on data collected by a lidar at a busy roundabout demonstrate that our approach can handle monitoring of a highly dynamic environment in real time.

The study of the interaction between autonomous robots and human agents in common working areas is an emerging field of research. Main points thereby are human safety, system stability, performance and optimality of the whole interaction process. Two approaches to deal with human-robot interaction can be distinguished: Long distance prediction which requires the recognition of intentions of other agents, and short distance control which deals with actions and reactions between agents and mutual reactive control of their motions and behaviors. In this context obstacle avoidance plays a prominent role. In this paper long distance prediction is represented by the identification of human intentions to use specific lanes by using fuzzy time clustering of pedestrian tracks. Another issue is the extrapolation of parts of both human and robot trajectories in the presence of scattered/uncertain measurements to guarantee a collision-free robot motion. Short distance control is represented by obstacle avoidance between agents using the method of velocity obstacles and both analytical and fuzzy control methods.

This paper introduces a novel robotic platform for aerial remote gas sensing. Spectroscopic measurement methods for remote sensing of selected gases lend themselves for use on mini-copters, which offer a number of advantages for inspection and surveillance. No direct contact with the target gas is needed and thus the influence of the aerial platform on the measured gas plume can be kept to a minimum. This allows to overcome one of the major issues with gas-sensitive mini-copters. On the other hand, remote gas sensors, most prominently Tunable Diode Laser Absorption Spectroscopy (TDLAS) sensors have been too bulky given the payload and energy restrictions of mini-copters. Here, we introduce and present the Unmanned Aerial Vehicle for Remote Gas Sensing (UAV-REGAS), which combines a novel lightweight TDLAS sensor with a 3-axis aerial stabilization gimbal for aiming on a versatile hexacopter. The proposed system can be deployed in scenarios that cannot be addressed by currently available robots and thus constitutes a significant step forward for the field of Mobile Robot Olfaction (MRO). It enables tomographic reconstruction of gas plumes and a localization of gas sources. We also present first results showing the gas sensing and aiming capabilities under realistic conditions.

We present a mobile robot motion planning approach under kinodynamic constraints that exploits learned perception priors in the form of continuous Gaussian mixture fields. Our Gaussian mixture fields are statistical multi-modal motion models of discrete objects or continuous media in the environment that encode e.g. the dynamics of air or pedestrian flows. We approach this task using a recently proposed circular linear flow field map based on semi-wrapped GMMs whose mixture components guide sampling and rewiring in an RRT* algorithm using a steer function for non-holonomic mobile robots. In our experiments with three alternative baselines, we show that this combination allows the planner to very efficiently generate high-quality solutions in terms of path smoothness, path length as well as natural yet minimum control effort motions through multi-modal representations of Gaussian mixture fields.

Sensor networks have limited capabilities to model complex phenomena occuring between sensing nodes. Mobile robots can be used to close this gap and learn local interpolation models. In this paper, we utilize Echo State Networks in order to learn the calibration and interpolation model between sensor nodes using measurements collected by a mobile robot. The use of Echo State Networks allows to deal with temporal dependencies implicitly, while the spatial mapping with a Gaussian Process estimator exploits the fact that Echo State Networks learn linear combinations of complex temporal dynamics. The resulting Echo State Map elegantly combines spatial and temporal cues into a single representation. We showcase the method in the exposure modeling task of building dust distribution maps for foundries, a challenge which is of great interest to occupational health researchers. Results from simulated data and real world experiments highlight the potential of Echo State Maps. While we focus on particulate matter measurements, the method can be applied for any other environmental variables like temperature or gas concentration.

Place categorisation; i.e., learning to group perception data into categories based on appearance; typically uses supervised learning and either visual or 2D range data. This paper shows place categorisation from 3D data without any training phase. We show that, by leveraging the NDT histogram descriptor to compactly encode 3D point cloud appearance, in combination with standard clustering techniques, it is possible to classify public indoor data sets with accuracy comparable to, and sometimes better than, previous supervised training methods. We also demonstrate the effectiveness of this approach to outdoor data, with an added benefit of being able to hierarchically categorise places into sub-categories based on a user-selected threshold. This technique relieves users of providing relevant training data, and only requires them to adjust the sensitivity to the number of place categories, and provide a semantic label to each category after the process is completed.

Local scan registration approaches commonly only utilize ego-motion estimates (e.g. odometry) as an initial pose guess in an iterative alignment procedure. This paper describes a new method to incorporate ego-motion estimates, including uncertainty, into the objective function of a registration algorithm. The proposed approach is particularly suited for feature-poor and self-similar environments, which typically present challenges to current state of the art registration algorithms. Experimental evaluation shows significant improvements in accuracy when using data acquired by Automatic Guided Vehicles (AGVs) in industrial production and warehouse environments.

The intention of humans and autonomous robots to interact in shared spatial areas is a challenging field of research regarding human safety, system stability and performance of the system's behavior. In this paper the intention recognition between human and robot from the control point of view are addressed and the time schedule of the exchanged signals is discussed. After a description of the kinematic and geometric relations between human and robot a so-called “compass dial” with the relative velocities is presented from which suitable fuzzy control rules are derived. The computation of the collision times at intersections and possible avoidance strategies are further discussed. Computations based on simulated and experimental data show the applicability of the methods presented.

Predicting grasp success is useful for avoiding failures in many robotic applications. Based on reasoning in wrench space, we address the question of how well analytic grasp success prediction works if tactile feedback is incorporated. Tactile information can alleviate contact placement uncertainties and facilitates contact modeling. We introduce a wrench-based classifier and evaluate it on a large set of real grasps. The key finding of this work is that exploiting tactile information allows wrench-based reasoning to perform on a level with existing methods based on learning or simulation. Different from these methods, the suggested approach has no need for training data, requires little modeling effort and is computationally efficient. Furthermore, our method affords task generalization by considering the capabilities of the grasping device and expected disturbance forces/moments in a physically meaningful way.

Creating an accurate model of gas emissions is an important task in monitoring and surveillance applications. A promising solution for a range of real-world applications are gas-sensitive mobile robots with spectroscopy-based remote sensors that are used to create a tomographic reconstruction of the gas distribution. The quality of these reconstructions depends crucially on the chosen sensing geometry. In this paper we address the problem of sensor planning by investigating sensing geometries that minimize reconstruction errors, and then formulate an optimization algorithm that chooses sensing configurations accordingly. The algorithm decouples sensor planning for single high concentration regions (hotspots) and subsequently fuses the individual solutions to a global solution consisting of sensing poses and the shortest path between them. The proposed algorithm compares favorably to a template matching technique in a simple simulation and in a real-world experiment. In the latter, we also compare the proposed sensor planning strategy to the sensing strategy of a human expert and find indications that the quality of the reconstructed map is higher with the proposed algorithm.

The highly percipient nature of human mind in avoiding sensory overload is a crucial factor which gives human vision an advantage over machine vision, the latter has otherwise powerful computational resources at its disposal given today's technology. This stresses the need to focus on methods which extract a concise representation of the environment inorder to approach a complex problem such as visual mapping. This article is an attempt of creating a mapping system, which proposes an architecture that combines task-specific and saliency driven approaches. The proposed method is implemented on a warehouse robot. The proposed solution provide a priority framework which enables an industrial robot to build a concise visual representation of the environment. The method is evaluated on data collected by a RGBD sensor mounted on a fork-lift robot and shows promise for addressing visual mapping problems in industrial environments.

In industrial environments, such as metallurgic facilities, human operators are exposed to harsh conditions where ambient air is often polluted with quartz, dust, lead debris and toxic fumes. Constant exposure to respirable particles can cause irreversible health damages and thus it is of high interest for occupational health experts to monitor the air quality on a regular basis. However, current monitoring procedures are carried out sparsely, with data collected in single day campaigns limited to few measurement locations. In this paper we explore the use and present first experimental results of a novel heterogeneous approach that uses a mobile robot and a network of low cost sensing nodes. The proposed system aims to address the spatial and temporal limitations of current monitoring techniques. The mobile robot, along with standard localization and mapping algorithms, allows to produce short term, spatially dense representations of the environment where dust, gas, ambient temperature and airflow information can be modelled. The sensing nodes on the other hand, can collect temporally dense (and usually spatially sparse) information during long periods of time, allowing in this way to register for example, daily variations in the pollution levels. Using data collected with the proposed system in an steel foundry, we show that a heterogeneous approach provides dense spatio-temporal information that can be used to improve the working conditions in industrial facilities.

Human-robot interaction and robot-robot interaction and cooperation in shared spatial areas is a challenging field of research regarding safety, stability and performance. In this paper the collision avoidance between human and robot by extrapolation of human intentions and a suitable optimization of tracking velocities is discussed. Furthermore for robot-robot interactions in a shared area traffic rules and artificial force potential fields and their optimization by market-based approach are applied for obstacle avoidance. For testing and verification, the navigation strategy is implemented and tested in simulation of more realistic vehicles. Extensive simulation experiments are performed to examine the improvement of the traditional potential field (PF) method by the MBO strategy.

Autonomous navigation in real-world industrial environments is a challenging task in many respects. One of the key open challenges is fast planning and execution of trajectories to reach arbitrary target positions and orientations with high accuracy and precision, while taking into account non-holonomic vehicle constraints. In recent years, lattice-based motion planners have been successfully used to generate kinematically and kinodynamically feasible motions for non-holonomic vehicles. However, the discretized nature of these algorithms induces discontinuities in both state and control space of the obtained trajectories, resulting in a mismatch between the achieved and the target end pose of the vehicle. As endpose accuracy is critical for the successful loading and unloading of cargo in typical industrial applications, automatically planned paths have not be widely adopted in commercial AGV systems. The main contribution of this paper addresses this shortcoming by introducing a path smoothing approach, which builds on the output of a lattice-based motion planner to generate smooth drivable trajectories for non-holonomic industrial vehicles. In real world tests presented in this paper we demonstrate that the proposed approach is fast enough for online use (it computes trajectories faster than they can be driven) and highly accurate. In 100 repetitions we achieve mean end-point pose errors below 0.01 meters in translation and 0.002 radians in orientation. Even the maximum errors are very small: only 0.02 meters in translation and 0.008 radians in orientation.

The problem of gas detection is relevant to many real-world applications, such as leak detection in industrial settings and surveillance. In this paper we address the problem of gas detection in large areas with a mobile robotic platform equipped with a remote gas sensor. We propose a novel method based on convex relaxation for quickly finding an exploration plan that guarantees a complete coverage of the environment. Our method proves to be highly efficient in terms of computational requirements and to provide nearly-optimal solutions. We validate our approach both in simulation and in real environments, thus demonstrating its applicability to real-world problems.

We address the problem of human detection from heavy mobile machinery and robotic equipment operating at industrial working sites. Exploiting the fact that workers are typically obliged to wear high-visibility clothing with reflective markers, we propose a new recognition algorithm that specifically incorporates the highly discriminative features of the safety garments in the detection process. Termed Multi-band Hough Forest, our detector fuses the input from active near-infrared (NIR) and RGB color vision to learn a human appearance model that not only allows us to detect and localize industrial workers, but also to estimate their body orientation. We further propose an efficient pipeline for automated generation of training data with high-quality body part annotations that are used in training to increase detector performance. We report a thorough experimental evaluation on challenging image sequences from a real-world production environment, where persons appear in a variety of upright and non-upright body positions.

The state-of-the-art object pose estimation approaches represent the set of detected poses together with corresponding uncertainty. The inaccurate noisy poses may result in a configuration of overlapping objects especially in cluttered environments. Under a rigid body assumption the inter-penetrations between pairs of objects are geometrically inconsistent. In this paper, we propose the principle of minimum translation search, PROMTS, to find an inter-penetration-free configuration of the initially detected objects. The target application is to automate the task of unloading shipping containers, where a geometrically consistent configuration of objects is required for high level reasoning and manipulation. We find that the proposed approach to resolve geometrical inconsistencies improves the overall pose estimation accuracy. We examine the utility of two selected search methods: A-star and Depth-Limited search. The performance of the search algorithms are tested on data sets generated in simulation and from real-world scenarios. The results show overall improvement of the estimated poses and suggest that depth-limited search presents the best overall performance.

In this work we tackle the problem of planning grasps for an underactuated gripper which enable it to retrieve target objects from a cluttered environment. Furthermore, we investigate how additional manipulation capabilities of the gripping device, provided by active surfaces on the inside of the fingers, can lead to performance improvement in the grasp execution process. To this end, we employ a simple strategy, in which the target object is `pulled-in' towards the palm during grasping which results in firm enveloping grasps. We show the effectiveness of the suggested methods by means of experiments conducted in a real-world scenario.

In this paper we present an inspection robot to produce gas distribution maps and localize gas sources in large outdoor environments. The robot is equipped with a 3D laser range finder and a remote gas sensor that returns integral concentration measurements. We apply principles of tomography to create a spatial gas distribution model from integral gas concentration measurements. The gas distribution algorithm is framed as a convex optimization problem and it models the mean distribution and the fluctuations of gases. This is important since gas dispersion is not an static phenomenon and furthermore, areas of high fluctuation can be correlated with the location of an emitting source. We use a compact surface representation created from the measurements of the 3D laser range finder with a state of the art mapping algorithm to get a very accurate localization and estimation of the path of the laser beams. In addition, a conic model for the beam of the remote gas sensor is introduced. We observe a substantial improvement in the gas source localization capabilities over previous state-of-the-art in our evaluation carried out in an open field environment.

In this paper, we propose an approach for robotic manipulation systems to autonomously reason about their environments under incomplete information. The target application is to automate the task of unloading the content of shipping containers. Our goal is to capture possible support relations between objects in partially known static configurations. We employ support vector machines (SVM) to estimate the probability of a support relation between pairs of detected objects using features extracted from their geometrical properties and 3D sampled points of the scene. The set of probabilistic support relations is then used for reasoning about optimally selecting an object to be unloaded first. The proposed approach has been extensively tested and verified on data sets generated in simulation and from real world configurations.

Industrial environments are rarely static and often their configuration is continuously changing due to the material transfer flow. This is a major challenge for infrastructure free localization systems. In this paper we address this challenge by introducing a localization approach that uses a dual-timescale approach. The proposed approach - Dual-Timescale Normal Distributions Transform Monte Carlo Localization (DT-NDT-MCL) - is a particle filter based localization method, which simultaneously keeps track of the pose using an apriori known static map and a short-term map. The short-term map is continuously updated and uses Normal Distributions Transform Occupancy maps to maintain the current state of the environment. A key novelty of this approach is that it does not have to select an entire timescale map but rather use the best timescale locally. The approach has real-time performance and is evaluated using three datasets with increasing levels of dynamics. We compare our approach against previously proposed NDT-MCL and commonly used SLAM algorithms and show that DT-NDT-MCL outperforms competing algorithms with regards to accuracy in all three test cases.

This paper presents a method for improving the quality of distorted 3D point clouds made from a vehicle equipped with a laser scanner moving over uneven terrain. Existing methods that use 3D point-cloud data (for tasks such as mapping, localisation, and object detection) typically assume that each point cloud is accurate. For autonomous robots moving in rough terrain, it is often the case that the vehicle moves a substantial amount during the acquisition of one point cloud, in which case the data will be distorted. The method proposed in this paper is capable of increasing the accuracy of 3D point clouds, without assuming any specific features of the environment (such as planar walls), without resorting to a “stop-scan-go” approach, and without relying on specialised and expensive hardware. Each new point cloud is matched to the previous using normal-distribution-transform (NDT) registration, after which a mini-loop closure is performed with a local, per-scan, graph-based SLAM method. The proposed method increases the accuracy of both the measured platform trajectory and the point cloud. The method is validated on both real-world and simulated data.

Due to its environmental, economical and safety implications, methane leak detection is a crucial task to address in the biogas production industry. In this paper, we introduce Gasbot, a robotic platform that aims to automatize methane emission monitoring in landfills and biogas production sites. The distinctive characteristic of the Gasbot platform is the use of a Tunable Laser Absorption Spectroscopy (TDLAS) sensor. This sensor provides integral concentration measurements over the path of the laser beam. Existing gas distribution mapping algorithms can only handle local measurements obtained from traditional in-situ chemical sensors. In this paper we also describe an algorithm to generate 3D methane concentration maps from integral concentration and depth measurements. The Gasbot platform has been tested in two different scenarios: an underground corridor, where a pipeline leak was simulated and in a decommissioned landfill site, where an artificial methane emission source was introduced.

Autonomous vehicles operating in real-world industrial environments have to overcome numerous challenges, chief among which is the creation and maintenance of consistent 3D world models. This paper proposes to address the challenges of online real-world mapping by building upon previous work on compact spatial representation and formulating a novel 3D mapping approach - the Normal Distributions Transform Occupancy Map (NDT-OM). The presented algorithm enables accurate real-time 3D mapping in large-scale dynamic environments employing a recursive update strategy. In addition, the proposed approach can seamlessly provide maps at multiple resolutions allowing for fast utilization in high-level functions such as localization or path planning. Compared to previous approaches that use the NDT representation, the proposed NDT-OM formulates an exact and efficient recursive update formulation and models the full occupancy of the map.

In this work we propose a method to effectively remove noise from depth images obtained with a commodity structured light sensor. The proposed approach fuses data into a consistent frame of reference over time, thus utilizing prior depth measurements and viewpoint information in the noise removal process. The effectiveness of the approach is compared to two state of the art, single-frame denoising methods in the context of feature descriptor matching and keypoint detection stability. To make more general statements about the effect of noise removal in these applications, we extend a method for evaluating local image gradient feature descriptors to the domain of 3D shape descriptors. We perform a comparative study of three classes of such descriptors: Normal Aligned Radial Features, Fast Point Feature Histograms and Depth Kernel Descriptors; and evaluate their performance on a real-world industrial application data set. We demonstrate that noise removal enabled by the dense map representation results in major improvements in matching across all classes of descriptors as well as having a substantial positive impact on keypoint detection reliability.

Ego-motion estimation and environment mapping are two recurring problems in the field of robotics. In this work we propose a simple on-line method for tracking the pose of a depth camera in six degrees of freedom and simultaneously maintaining an updated 3D map, represented as a truncated signed distance function. The distance function representation implicitly encodes surfaces in 3D-space and is used directly to define a cost function for accurate registration of new data. The proposed algorithm is highly parallel and achieves good accuracy compared to state of the art methods. It is suitable for reconstructing single household items, workspace environments and small rooms at near real-time rates, making it practical for use on modern CPU hardware.

In this paper we introduce a method for learning motion patterns in dynamic environments. Representations of dynamic environments have recently received an increasing amount of attention in the research community. Understanding dynamic environments is seen as one of the key challenges in order to enable autonomous navigation in real-world scenarios. However, representing the temporal dimension is a challenge yet to be solved. In this paper we introduce a spatial representation, which encapsulates the statistical dynamic behavior observed in the environment. The proposed Conditional Transition Map (CTMap) is a grid-based representation that associates a probability distribution for an object exiting the cell, given its entry direction. The transition parameters are learned from a temporal signal of occupancy on cells by using a local-neighborhood cross-correlation method. In this paper, we introduce the CTMap, the learning approach and present a proof-of-concept method for estimating future paths of dynamic objects, called Conditional Probability Propagation Tree (CPPTree). The evaluation is done using a real-world dataset collected at a busy roundabout.

In this paper, we propose a new approach for automatically building symbolic relational descriptions of static configurations of objects to be manipulated by a robotic system. The main goal of our work is to provide advanced cognitive abilities for such robotic systems to make them more aware of the outcome of their actions. We describe how such symbolic relations are automatically extracted for configurations of box-shaped objects using notions from geometry and static equilibrium in classical mechanics. We also present extensive simulation results as well as some real-world experiments aimed at verifying the output of the proposed approach.

We propose and evaluate a system for detecting and tracking multiple humans wearing high-visibility clothing from vehicles operating in industrial work environments. We use a customized stereo camera setup equipped with IR flash and IR filter to detect the reflective material on the worker's garments and estimate their trajectories in 3D space. An evaluation in two distinct industrial environments with different degrees of complexity demonstrates the approach to be robust and accurate for tracking workers in arbitrary body poses, under occlusion, and under a wide range of different illumination settings.

Industrial applications often impose hard requirements on the precision of autonomous vehicle systems. As a consequence industrial Automatically Guided Vehicle (AGV) systems still use high-cost infrastructure based positioning solutions. In this paper we propose a map based localization method that fulfills the requirements on precision and repeatability, typical for industrial application scenarios. The proposed method - Normal Distributions Transform Monte Carlo Localization (NDT-MCL) is based on a well established probabilistic framework. In a novel contribution, we formulate the MCL localization approach using the Normal Distributions Transform (NDT) as an underlying representation for both map and sensor data. By relaxing the hard discretization assumption imposed by grid-map models and utilizing the piece-wise continuous NDT representation the proposed algorithm achieves substantially improved accuracy and repeatability. The proposed NDT-MCL algorithm is evaluated using offline data sets from both a laboratory and a real-world industrial environments. Additionally, we report a comparison of the proposed algorithm to grid-based MCL and to a commercial localization system when used in a closed-loop with the control system of an AGV platform. In all tests the proposed algorithm is demonstrated to provide performance superior to that of standard grid-based MCL and comparable to the performance of the commercial infrastructure based positioning system.

Autonomous vehicles operating in real-world industrial environments have to overcome numerous challenges, chief among which is the creation and maintenance of consistent 3D world models. This paper focuses on a particularly important challenge: mapping in dynamic environments. We introduce several improvements to the recently proposed Normal Distributions Transform Occupancy Map (NDT-OM) aimed for efficient mapping in dynamic environments. A careful consistency analysis is given based on convergence and similarity metrics specifically designed for evaluation of NDT maps in dynamic environments. We show that in the context of mapping with known poses the proposed method results in improved consistency and in superior runtime performance, when compared against 3D occupancy grids at the same size and resolution. Additionally, we demonstrate that NDT-OM features real-time performance in a highly dynamic 3D mapping and tracking scenario with centimeter accuracy over a 1.5km trajectory.

Autonomous vehicles operating in real-world industrial environments have to overcome numerous challenges, chief among which are the creation of consistent 3D world models and the simultaneous tracking of the vehicle pose with respect to the created maps. In this paper we integrate two recently proposed algorithms in an online, near-realtime mapping and tracking system. Using the Normal Distributions Transform (NDT), a sparse Gaussian Mixture Model, for representation of 3D range scan data, we propose a frame-to-model registration and data fusion algorithm - NDT Fusion. The proposed approach uses a submap indexing system to achieve operation in arbitrarily-sized environments. The approach is evaluated on a publicly available city-block sized data set, achieving accuracy and runtime performance significantly better than current state of the art. In addition, the system is evaluated on a data set covering ten hours of operation and a trajectory of 7.2km in a real-world industrial environment, achieving centimeter accuracy at update rates of 5-10 Hz.

Point set registration-the task of finding the best fitting alignment between two sets of point samples, is an important problem in mobile robotics. This article proposes a novel registration algorithm, based on the distance between Three-Dimensional Normal Distributions Transforms. 3D-NDT models - a sub-class of Gaussian Mixture Models with uniformly weighted, largely disjoint components, can be quickly computed from range point data. The proposed algorithm constructs 3D-NDT representations of the input point sets and then formulates an objective function based on the L2distance between the considered models. Analytic first and second order derivatives of the objective function are computed and used in a standard Newton method optimization scheme, to obtain the best-fitting transformation. The proposed algorithm is evaluated and shown to be more accurate and faster, compared to a state of the art implementation of the Iterative Closest Point and 3D-NDT Point-to-Distribution algorithms.

In this paper we propose a new grid based approach to model a dynamic environment. Each grid cell is assumed to be an independent Markov chain (iMac) with two states. The state transition parameters are learned online and modeled as two Poisson processes. As a result, our representation not only encodes the expected occupancy of the cell, but also models the expected dynamics within the cell. The paper also presents a strategy based on recency weighting to learn the model parameters from observations that is able to deal with non-stationary cell dynamics. Moreover, an interpretation of the model parameters with discussion about the convergence rates of the cells is presented. The proposed model is experimentally validated using offline data recorded with a Laser Guided Vehicle (LGV) system running in production use.

The Three-Dimensional Normal Distributions Transform (3D-NDT) is a spatial modeling technique with applications in point set registration, scan similarity comparison, change detection and path planning. This work concentrates on evaluating three common variations of the 3D-NDT in terms of accuracy of representing sampled semi-structured environments. In a novel approach to spatial representation quality measurement, the 3D geometrical modeling task is formulated as a classification problem and its accuracy is evaluated with standard machine learning performance metrics. In this manner the accuracy of the 3D-NDT variations is shown to be comparable to, and in some cases to outperform that of the standard occupancy grid mapping model.

Planning feasible paths in fully three-dimensional environments is a challenging problem. Application of existing algorithms typically requires the use of limited 3D representations that discard potentially useful information. This article proposes a novel approach to path planning that utilizes a full 3D representation directly: the Three-Dimensional Normal Distributions Transform (3D-NDT). The well known wavefront planner is modified to use 3D-NDT as a basis for map representation and evaluated using both indoor and outdoor data sets. The use of 3D-NDT for path planning is thus demonstrated to be a viable choice with good expressive capabilities.

This paper presents an overview of an autonomous robotic system for material handling. The system is being developed by extending the functionalities of traditional AGVs to be able to operate reliably and safely in highly dynamic environments. Traditionally, the reliable functioning of AGVs relies on the availability of adequate infrastructure to support navigation. In the target environments of our system, such infrastructure is difficult to setup in an efficient way. Additionally, the location of objects to handle are unknown, which requires runtime object detection and tracking. Another requirement to be fulfilled by the system is the ability to generate trajectories dynamically, which is uncommon in industrial AGV systems.

We propose a new approach to appearance based loop detection from metric 3D maps, exploiting the NDT surface representation. Locations are described with feature histograms based on surface orientation and smoothness, and loop closure can be detected by matching feature histograms. We also present a quantitative performance evaluation using two real-world data sets, showing that the proposed method works well in different environments.

To advance robotic science it is important to perform experiments that can be replicated by other researchers to compare different methods. However, these comparisons tend to be biased, since re-implementations of reference methods often lack thoroughness and do not include the hands-on experience obtained during the original development process. This paper presents a thorough comparison of 3D scan registration algorithms based on a 3D mapping field experiment, carried out by two research groups that are leading in the field of 3D robotic mapping. The iterative closest points algorithm (ICP) is compared to the normal distributions transform (NDT). We also present an improved version of NDT with a substantially larger valley of convergence than previously published versions.

Gas distribution modelling constitutes an ideal application area for mobile robots, which - as intelligent mobile gas sensors - offer several advantages compared to stationary sensor networks. In this paper we propose the Kernel DM+V algorithm to learn a statistical 2-d gas distribution model from a sequence of localized gas sensor measurements. The algorithm does not make strong assumptions about the sensing locations and can thus be applied on a mobile robot that is not primarily used for gas distribution monitoring, and also in the case of stationary measurements. Kernel DM+V treats distribution modelling as a density estimation problem. In contrast to most previous approaches, it models the variance in addition to the distribution mean. Estimating the predictive variance entails a significant improvement for gas distribution modelling since it allows to evaluate the model quality in terms of the data likelihood. This offers a solution to the problem of ground truth evaluation, which has always been a critical issue for gas distribution modelling. Estimating the predictive variance also provides the means to learn meta parameters and to suggest new measurement locations based on the current model. We derive the Kernel DM+V algorithm and present a method for learning the hyper-parameters. Based on real world data collected with a mobile robot we demonstrate the consistency of the obtained maps and present a quantitative comparison, in terms of the data likelihood of unseen samples, with an alternative approach that estimates the predictive variance.

We present a new algorithm for scan registration of colored 3D point data which is an extension to the Normal Distributions Transform (NDT). The probabilistic approach of NDT is extended to a color-aware registration algorithm by modeling the point distributions as Gaussian mixture-models in color space. We discuss different point cloud registration techniques, as well as alternative variants of the proposed algorithm. Results showing improved robustness of the proposed method using real-world data acquired with a mobile robot and a time-of-flight camera are presented.

The problem of appearance-based mapping and navigation in outdoor environments is far from trivial. In this paper, an appearance-based topological map, covering a large, mixed indoor and outdoor environment, is built incrementally by using panoramic images. The map is based on image similarity, so that the resulting segmentation of the world corresponds closely to the human concept of a place. Using high-resolution images and the epipolar constraint, the resulting map is shown to be very suitable for localization, even when the environment has undergone seasonal changes.

In this paper we present initial experiments towards environmental monitoring with a mobile platform. A prototype of a pollution monitoring robot was set up which measures the gas distribution using an ldquoelectronic noserdquo and provides three dimensional wind measurements using an ultrasonic anemometer. We describe the design of the robot and the experimental setup used to run trials under varying environmental conditions. We then present the results of the gas distribution mapping. The trials which were carried out in three uncontrolled environments with very different properties: an enclosed indoor area, a part of a long corridor with open ends and a high ceiling, and an outdoor scenario are presented and discussed.

This paper presents a vision-based approach to simultaneous localization and mapping (SLAM) in indoor/outdoor environments with minimalistic sensing and computational requirements. The approach is based on a graph representation of robot poses, using a relaxation algorithm to obtain a globally consistent map. Each link corresponds to a relative measurement of the spatial relation between the two nodes it connects. The links describe the likelihood distribution of the relative pose as a Gaussian distribution. To estimate the covariance matrix for links obtained from an omnidirectional vision sensor, a novel method is introduced based on the relative similarity of neighboring images. This new method does not require the determination of distances to image features using multiple-view geometry, for example. Combined indoor and outdoor experiments demonstrate that the approach can handle different environments (without modification of the parameters), and it can cope with violations of the ldquoflat floor assumptionrdquo to some degree and scales well with increasing size of the environment, producing topologically correct and geometrically accurate maps at low computational cost. Further experiments demonstrate that the approach is also suitable for combining multiple overlapping maps, e.g., for solving the multirobot SLAM problem with unknown initial poses.

This paper presents a vision-based approach to SLAM in large-scale environments with minimal sensing and computational requirements. The approach is based on a graphical representation of robot poses and links between the poses. Links between the robot poses are established based on odomety and image similarity, then a relaxation algorithm is used to generate a globally consistent map. To estimate the covariance matrix for links obtained from the vision sensor, a novel method is introduced based on the relative similarity of neighbouring images, without requiring distances to image features or multiple view geometry. Indoor and outdoor experiments demonstrate that the approach scales well to large-scale environments, producing topologically correct and geometrically accurate maps at minimal computational cost. Mini-SLAM was found to produce consistent maps in an unstructured, large-scale environment (the total path length was 1.4 km) containing indoor and outdoor passages.

This paper presents a novel use of spectral clustering algorithms to support cases where the entries in the affinity matrix are costly to compute. The method is incremental - the spectral clustering algorithm is applied to the affinity matrix after each row/column is added - which makes it possible to inspect the clusters as new data points are added. The method is well suited to the problem of appearance-based, on-line topological mapping for mobile robots. In this problem domain, we show that we can reduce environment-dependent parameters of the clustering algorithm to just a single, intuitive parameter. Experimental results in large outdoor and indoor environments show that we can close loops correctly by computing only a fraction of the entries in the affinity matrix. The accompanying video clip shows how an example map is produced by the algorithm.

This paper presents a system for autonomous change detection with a security patrol robot. In an initial step a reference model of the environment is created and changes are then detected with respect to the reference model as differences in coloured 3D point clouds, which are obtained from a 3D laser range scanner and a CCD camera. The suggested approach introduces several novel aspects, including a registration method that utilizes local visual features to determine point correspondences (thus essentially working without an initial pose estimate) and the 3D-NDT representation with adaptive cell size to efficiently represent both the spatial and colour aspects of the reference model. Apart from a detailed description of the individual parts of the difference detection system, a qualitative experimental evaluation in an indoor lab environment is presented, which demonstrates that the suggested system is able register and detect changes in spatial 3D data and also to detect changes that occur in colour space and are not observable using range values only.

This paper presents an approach for tracking multiple persons using a combination of colour and thermal vision sensors on a mobile robot. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is then incorporated into the tracker.

This paper presents a learning system that uses Q-learning with a resource allocating network (RAN) for behavior learning in mobile robotics. The RAN is used as a function approximator, and Q-learning is used to learn the control policy in 'off-policy' fashion that enables learning to be bootstrapped by a prior knowledge controller, thus speeding up the reinforcement learning. Our approach is verified on a PeopleBot robot executing a visual servoing based docking behavior in which the robot is required to reach a goal pose. Further experiments show that the RAN network can also be used for supervised learning prior to reinforcement learning in a layered architecture, thus further improving the performance of the docking behavior

This paper presents an algorithm that builds topological maps, using omnidirectional vision as the only sensor modality. Local features are extracted from images obtained in sequence, and are used both to cluster the images into nodes and to detect links between the nodes. The algorithm is incremental, reducing the computational requirements of the corresponding batch algorithm. Experimental results in a complex, indoor environment show that the algorithm produces topologically correct maps, closing loops without suffering from perceptual aliasing or false links. Robustness to lighting variations was further demonstrated by building correct maps from combined multiple datasets collected over a period of 2 months

This work addresses the capability to use concentration gridmaps to locate a static gas source. In previous works it was found that depending on the shape of the mapped gas distribution (corresponding to different airflow conditions) the gas source location can be sometimes approximated with high accuracy by the maximum in the concentration map while this is not possible in other cases. This paper introduces a method to distinguish both cases by analysing the shape of the obtained concentration map in terms of a model of the time-averaged gas distribution known from physics. The parameters of the model that approximates the concentration map most closely are determined by nonlinear least squares fitting using evolution strategies (ES). The best fit also provides a better estimate of the gas source position in situations where the concentration maximum estimate fails. Different methods to select the most truthful estimate are introduced in this work and a comparison regarding their accuracy is presented, based on a total of 34h of concentration mapping experiments.

As a sub-task of the general gas source localisation problem, gas source declaration is the process of determining the certainty that a source is in the immediate vicinity. Due to the turbulent character of gas transport in a natural indoor environment, it is not sufficient to search for instantaneous concentration maxima, in order to solve this task. Therefore, this paper introduces a method to classify whether an object is a gas source or not from a series of concentration measurements, recorded while the robot performs a rotation manoeuvre in front of a possible source. For three different gas source positions, a total of 288 declaration experiments were carried out at different robot-to-source distances. Based on these readings, two machine learning techniques (ANN, SVM) were evaluated in terms of their classification performance. With learning parameters that were optimised by grid search, a maximal hit rate of approximately 87.5% could be obtained using a support vector machine.

As a sub-task of the general gas source localisation problem, gas source declaration is the process of determining the certainty that a source is in the immediate vicinity. Due to the turbulent character of gas transport in a natural indoor environment, it is not sufficient to search for instantaneous concentration maxima, in order to solve this task. Therefore, this paper introduces a method to classify whether an object is a gas source from a series of concentration measurements, recorded while the robot performs a rotation manoeuvre in front of a possible source. For three different gas source positions, a total of 1056 declaration experiments were carried out at different robot-to-source distances. Based on these readings, support vector machines (SVM) with optimised learning parameters were trained and the cross-validation classification performance was evaluated. The results demonstrate the feasibility of the approach to detect proximity to a gas source using only gas sensors. The paper also presents an analysis of the classification rate depending on the desired declaration accuracy, and a comparison with the classification rate that can be achieved by selecting an optimal threshold value regarding the mean sensor signal.

This paper addresses the problem of mapping the features of a gas distribution by creating concentration gridmaps from the data collected by a mobile robot equipped with an electronic nose. By contrast to metric gridmaps extracted from sonar or laser range scans, a single measurement of the electronic nose provides information about a comparatively small area. To overcome this problem, a mapping technique is introduced that uses a Gaussian density function to model the decreasing likelihood that a particular reading represents the true concentration with respect to the distance from the point of measurement. This method is evaluated in terms of its suitability regarding the slow response and recovery of the gas sensors. The stability of the mapped features and the capability to use concentration gridmaps to locate a gas source are also discussed.

This paper describes the assembly of a mobile odour sensing system and investigates its practical operation in an indoor environment without a constant airflow. Lacking a constant airflow leads to a problem which cannot be neglected in real world applications. The response of the metal oxide gas sensors used is dominated by air turbulence rather than concentration differences. We show that this problem can be overcome by driving the robot with a constant speed, thus adding an extra constant airflow relative to the gas sensors location. If the robot's speed is not too low the system described proved to be well suited to detect even weak odour sources. Since driving with constant speed is an indispensable condition to perform the basic tasks of a mobile odour sensing system, a new localization strategy is proposed, which takes this into account.