An open-source software framework is presented that allows real-time underwater mapping with popular marine robotics components, namely a BlueRobotics BlueROV2 with its standard Ping360 Mechanical Scanning Sonar (MSS) and a A50 Doppler Velocity Log (DVL), which are low-cost devices for their respective types - if not even the most affordable ones on the market. The software runs with low computational power on a Raspberry Pi4. The framework builds upon Synthetic Scan Formation (SSF) where single MSS beams or scan-lines are embedded into a pose-graph. The rendering of scans is not only based on navigation, but based on the graph itself. Scans formed from scan-lines can be optimized by online Simultaneous Localization and Mapping (SLAM) and result in improved scans, based on the current state of the graph. In subsequent steps this leads to improved registration results. To this end, a combination of two different types of loop-closures is presented. Namely a consecutive loop closure, and a proximity based loop closure, which reduces the overall drift. The framework is validated in three different test-environments, namely a pool, a test-tank with a gantry for ground truth motion, and the flooded basement of a WW-II submarine bunker. Among others, it is shown that there is an increased accuracy compared to conventional SLAM and that the software is usable in real-time during a mission with the low-cost hardware.

In this paper, we introduce Fourier-SOFT 2D (FS2D) as a new robust registration method. FS2D operates in the frequency domain where it exploits the well-known decoupling of rotation and translation. The challenging part of determining the rotation parameter is solved here based on a projection of the Fourier magnitude on a sphere and the SO(3) Fourier Transform (SOFT). The underlying use case is underwater mapping with sonar, i.e., with very noisy and partially overlapping environment data under non-trivial localization and navigation challenges. Fourier-SOFT 2D is compared with openly available registration methods on two real-world datasets and a simulated dataset. Results show the robustness of FS2D, i.e., its capabilities to handle large amounts of noise and occlusions of consecutive scans. The implementation in C++ is openly available.

High-end laser range-finders provide accurate 3D data over long ranges. But their scans are inhomogeneous, i.e., the environment is non-uniformly sampled, as there is denser data in the near range than in the far range. Furthermore, the generation of a scan is time-consuming. Thus, it is desirable to cover an area by as few scans as possible, i.e., scanning is more time-efficient if the overlap between scans is as small as possible. However, these factors pose significant challenges for state-of-the-art registration algorithms. In this work, we present a divide-and-conquer method that uses an efficient strategy to check for possible registrations between partitions of two scans. As underlying registration method, Fourier-Mellin-SOFT (FMS) is used. FMS is quite robust against partial overlaps but its performance is significantly boosted by the presented partitioning method. As concrete use case, results from the digitization of a WWII submarine bunker as a large-scale cultural heritage site are presented.

Spectral methods such as the improved Fourier Mellin Invariant (iFMI) transform have proved to be faster, more robust and accurate than feature based methods on image registration. However, iFMI is restricted to work only when the camera moves in 2D space and has not been applied on omni-cameras images so far. In this work, we extend the iFMI method and apply a motion model to estimate an omni-camera's pose when it moves in 3D space. In the experiment section, we compare the extended iFMI method against ORB and AKAZE feature based approaches on three datasets, showing different types of environments: office, lawn and urban scenery (MPI-omni dataset). The results show that our method reduces the error of the camera pose estimation two to three times with respect to the feature registration techniques, while offering lower processing times.

Underwater robot interventions require a high level of safety and reliability. A major challenge to address is a robust and accurate acquisition of localization estimates, as it is a prerequisite to enable more complex tasks, e.g. floating manipulation and mapping. State-of-the-art navigation in commercial operations, such as oil& gas production (OGP), rely on costly instrumentation. These can be partially replaced or assisted by visual navigation methods, especially in deep-sea scenarios where equipment deployment has high costs and risks. Our work presents a multimodal approach that adapts state-of-the-art methods from on-land robotics, i.e., dense point cloud generation in combination with plane representation and registration, to boost underwater localization performance. A (a) two-stage navigation scheme is proposed that initially generates a coarse probabilistic map of the workspace, which is used to filter noise from computed point clouds and planes in the second stage. Furthermore, an adaptive decision-making approach is introduced that determines which perception cues to incorporate into the localization filter to optimize accuracy and computation performance. Our approach is investigated first in simulation and then validated with data from field trials in OGP monitoring and maintenance scenarios.

Superquadric surfaces play an important role in many different research fields due to their ability to model a variety of shapes with a small number of parameters. One of their core applications is the estimation of object shape characteristics from a set of discrete samples from the surface of an object. However, the corresponding optimization problem is prone to numerical instabilities in some regions of the parameter space. To mitigate this problem, lower bound constraints to the shape parameters are applied during the optimization thus limiting the range of shapes which can be accurately represented by superquadrics. Therefore, the exact modeling of very common shapes like cuboids and cylinders is error-prone in practice. In this article we investigate this problem and provide a numerically stable formulation for the evaluation of the superquadric surface function and for its gradient. This new formulation enables numerically stable fitting of superquadrics in the previously constrained region, i.e., in its full range including cuboids and cylinders. In addition, the new formulation also leads to faster convergence speed. The theoretical contributions are substantiated by experiments on synthetic as well as real data.

A topological shape analysis is proposed and utilized to learn concepts that reflect shape commonalities. Our approach is two-fold: i) a spatial topology analysis of point cloud segment constellations within objects. Therein constellations are decomposed and described in an hierarchical manner - from single segments to segment groups until a single group reflects an entire object. ii) a topology analysis of the description space in which segment decompositions are exposed in. Inspired by Persistent Homology, hidden groups of shape commonalities are revealed from object segment decompositions. Experiments show that extracted persistent groups of commonalities can represent semantically meaningful shape concepts. We also show the generalization capability of the proposed approach considering samples of external datasets.

Deep-sea robot operations demand a high level of safety, efficiency and reliability. As a consequence, measures within the development stage have to be implemented to extensively evaluate and benchmark system components ranging from data acquisition, perception and localization to control. We present an approach based on high-fidelity simulation that embeds spatial and environmental conditions from recorded real-world data. This simulation in the loop (SIL) methodology allows for mitigating the discrepancy between simulation and real-world conditions, e.g. regarding sensor noise. As a result, this work provides a platform to thoroughly investigate and benchmark behaviors of system components concurrently under real and simulated conditions. The conducted evaluation shows the benefit of the proposed work in tasks related to perception and self-localization under changing spatial and environmental conditions.

We present a hierarchical graph-based approach for unknown object discovery in RGB-D point clouds captured with a Kinect-like sensor from unstructured scenes. A two-step approach is proposed which first extracts meaningful regions from an input scene through over-segmentation. Secondly, a procedure is introduced to detect compositions of such regions that can represent primitive-shaped object candidates like boxes or cylinders. Complex-shaped objects are interpreted as a composition of primitive-shaped objects, for instance, a teddy bear can consist of two convex-shaped arms, legs, a convex-shaped head and torso. An ensemble of classifiers is trained to learn patterns from the appearances of such neighboring primitive shapes that constitute complex-shaped objects. Therein the appearance is described by a set of features from the texture and geometry domain. For the experiments, a dataset was prepared which is publicly available, containing a set of scenes which consists of 296 human-annotated object instances in total. Experiments show that the proposed hierarchical approach is capable to extract meaningful regions: an under-segmentation rate of 2.6% has been achieved. Furthermore, objects are segmented with a segmentation rate of 92.9% which reflects the capability of our approach to detect potential object candidates within unstructured scenes.

We present a damage-aware planning approach which determines the best sequence to manipulate a number of objects in a scene. This works on task-planning level, abstracts from motion planning and anticipates the dynamics of the scene using a physics simulation. Instead of avoiding interaction with the environment, we take unintended motion of other objects into account and plan manipulation sequences which minimize the potential damage. Our method can also be used as a validation measure to judge planned motions for their feasibility in terms of damage avoidance. We evaluate our approach on one industrial scenario (autonomous container unloading) and one retail scenario (shelf replenishment).

An extension based on attention and anticipation of a robot vision pipeline for object recognition in RGBD images from low-cost sensors like MS Kinect or ASUS Xtion is presented. This work originated in research on an industrial application scenario, namely shipping-container unloading, but it is applicable to advanced manipulation tasks in unstructured environments in general where the perception must be very robust while being as fast as possible. For these scenarios, we build on our previous work that proved to be competitive in cluttered scenes in table-top scenarios and which forms the backbone of our RGBD object recognition. It is further enhanced by two main contributions. First, a simple but very effective form of anticipation as top-down expectations of the evolution of the scene due to the actions of the robot is used to speed up the processing. Second, attention is used as a mechanism for further speed-up by focusing processing only on certain regions of interest of the scene based also on an anticipation mechanism. The method is analyzed in experiments using real-world data from an industrial demonstration set-up.

Given that 3D scan matching is such a central part of the perception pipeline for robots, thorough and large-scale investigations of scan matching performance are still surprisingly few. A crucial part of the scientific method is to perform experiments that can be replicated by other researchers in order to compare different results. In light of this fact, this paper presents a thorough comparison of 3D scan registration algorithms using a recently published benchmark protocol which makes use of a publicly available challenging data set that covers a wide range of environments. In particular, we evaluate two types of recent 3D registration algorithms - one local and one global. Both approaches take local surface structure into account, rather than matching individual points. After well over 100 000 individual tests, we conclude that algorithms using the normal distributions transform (NDT) provides accurate results compared to a modern implementation of the iterative closest point (ICP) method, when faced with scan data that has little overlap and weak geometric structure. We also demonstrate that the minimally uncertain maximum consensus (MUMC) algorithm provides accurate results in structured environments without needing an initial guess, and that it provides useful measures to detect whether it has succeeded or not. We also propose two amendments to the experimental protocol, in order to provide more valuable results in future implementations.

This paper proposes a new method for segmenting and classifying seamines on Synthetic Aperture Sonar (SAS) side scan images. The method uses an active contours approach and superellipse a-priori knowledge to segment the image in object, object-shadow and background areas. In contrast to other methods using superellipse constraints, the shape prior is incorporated directly into the segmentation process. This kind of segmentation has the advantage that afterwards the extracted superellipse parameters that describe the object and the object-shadow can directly be used as feature for a classification - this work is hence also of potential interest for general object recognition tasks in other application domains. Several different perspectives of implementing this idea into a suitable algorithm are introduced and compared with each other. Thus, for the evaluation of each method the extracted superellipse features are used for a support vector machine classification. An one against all confusion matrix is generated on a test data set. This result is compared to a related state of the art algorithm. It is shown that our new method is able to correctly classify 170 of 210 objects in a very challenging real world data set and that it yields significant better results than the state of the art comparison.

In this work we tackle the problem of planning grasps for an underactuated gripper which enable it to retrieve target objects from a cluttered environment. Furthermore, we investigate how additional manipulation capabilities of the gripping device, provided by active surfaces on the inside of the fingers, can lead to performance improvement in the grasp execution process. To this end, we employ a simple strategy, in which the target object is `pulled-in' towards the palm during grasping which results in firm enveloping grasps. We show the effectiveness of the suggested methods by means of experiments conducted in a real-world scenario.

Graph-based Simultaneous Localization and Mapping (SLAM) has experienced a recent surge towards robust methods. These methods take the combinatorial aspect of data association into account by allowing decisions of the graph topology to be made during optimization. In this paper, the Generalized Graph SLAM framework for SLAM under ambiguous data association is presented, and a formal description of using hyperedges to encode uncertain loop closures is given for the first time. The framework combines both hyperedges and multimodal Mixture of Gaussian constraints to cover all sources of ambiguity in SLAM. An extension of the authors' multimodal Prefilter method is developed to find good initial conditions in such a generalized multimodal hypergraph. Experiments on a real world 3D dataset show that the novel multimodal hypergraph Prefilter method is both significantly more robust and faster than other robust state-of-the-art methods.

We present an approach for object class learning using a part-based shape categorization in RGB-augmented 3D point clouds captured from cluttered indoor scenes with a Kinect-like sensor. We propose an unsupervised hierarchical learning procedure which allows to symbolically classify shape parts by different specificity levels of detailedness of their surface-structural appearance. Further, a hierarchical graphical model is learned that reflects the constellation of classified parts from the set of specificity levels learned in the previous step. Finally an energy minimization inference procedure is applied on the hierarchical graphical model to obtain the corresponding shape category of an object instance consisting of a set of shape parts. Experiments show, due to the proposed classification by learning the hierarchy of shape parts combined with generating the hierarchical graphical constellation model of shape parts, the additional evidence on different levels of shape detailedness is a major factor that leads to a more robust and accurate categorization compared to a flat approach. The experiments are conducted for shape categories (sack, barrel and parcel) which typically appear in visuoperceptual-challenging scenarios of logistic processes.

Underwater image mosaicking is an important tool for visual surveys, object detection, and as a means to control the underwater robot if done online. Such application areas can benefit significantly from a recent focus on robust methods for graph-based Simultaneous Localization and Mapping (SLAM). This paper focuses on two contributions: An approach to combine registration results from multiple methods in multimodal constraints and, up to the authors' knowledge, the first method to generate hyperedge constraints from state-of-the-art place recognition techniques. Both contributions are implemented within the Generalized Graph SLAM framework. Experimental results show that the methods generate informative constraints and that the authors' Prefilter method outperforms related methods on a large underwater image dataset processed with these methods.

An autonomous aerial or underwater robot can create photo-maps using a downward looking camera to not only compute its odometry visually but also to provide a useful and intuitively understandable representation of the environment explored by it. The improved Fourier Mellin Invariant (iFMI) registration is a spectral registration method, which has specific benefits, especially high robustness in featureless scenarios, but it only allows registrations of 2D translations, rotation, and scale. The method is extended here to incorporate tilt using parallax information. To this end, we extend the well-known four-point algorithm for planar homography. We show that using the decomposition of the planar homography to compute the tilt is very noise-prone, and propose a way of increasing this accuracy based on a parallax to noise metric. Although our general approach can be used with local scale invariant image features, we implement the tilt-correction based on an extension of our frequency-based approach to determine the image motion-field. Two experiments are presented to show the efficacy and applicability of our approach: An analysis of a simulated data set with ground truth is used to quantify the robustness of our novel method relative to the same four-point method using only SIFT (Scale Invariant Feature Transform) features. A second data set is used to present similar results with real-world data.

Mapping is an important task for mobile robots. But assessing the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. A new approach on map evaluation is presented here. It is based on Topology Graphs as a topological, abstracted representation of 2D grid maps. The Topology Graphs are derived from Voronoi Diagrams that get post-processed to capture the high-level spatial structures. Based on a similarity metric on vertices in Topology Graphs, the vertices can be matched across maps and spatial (dis)similarities and hence errors in the mapping can be identified and measured. More precisely, the vertex-similarity is the basis to match the structures of Topology Graphs up to the identification of subgraph isomorphisms through wave-front propagation. This allows to determine important map quality attributes up to very challenging structural elements like brokenness, i.e., the number of locally correct partitions in the candidate map and their relative placement towards each other. Experiments with real robot generated maps including examples from various teams in the RoboCup Rescue competition are used to validate the usefulness of this method for map quality assessment.

This paper presents an approach to acquire textured 3D models of objects without the need for sophisticated hardware infrastructures. The approach is inexpensive, using a low-cost Microsoft Kinect RGB-D sensor and Augmented Reality (AR) markers printed on paper sheets. The AR-markers can be freely placed in the scene, allowing the modeling of objects of various sizes, and the sensor can be moved by the hand of an untrained person. To generate usable models with this very inexpensive and simple setup, the sequence of RGB-D scans is embedded in a graph-based optimizer for automatic post-refinement. The main novelty of this contribution is the development of an uncertainty model for an AR-marker. The AR-marker uncertainty models are used as constraints in an optimization problem to better estimate the object pose. The models are in the end further fine-tuned by a standard point-based registration algorithm. The results section presents realistic models of various objects generated using this system, e.g., parcels, sport balls, human dolls etc. Additionally, a quantitative analysis is presented using objects of known dimensions.

We present an approach for object class learning using a part-based shape categorization in RGB-augmented 3D point clouds captured from cluttered indoor scenes with a Kinect-like sensor. A graph representation is used to detect and categorize object instances based on part-constellations found in scenes. No assumptions like objects being placed on planar surfaces or constraints on their poses are required. Our approach consists of the following steps: 1) a Mean-Shift-based over-segmentation of a point cloud into atomic patches; 2) use of topological and geometric features to merge surface-homogeneous atomic patches into super patches; 3) an unsupervised classification of these parts that allows to symbolically label distinctively unknown object parts by their surface-structural appearance; and finally, 4) a graph generation procedure that reflects the constellation of the detected parts from object instances of certain shape categories. Furthermore, an inference procedure is presented that processes extracted part constellations of a scene to detect and categorize object instances. Experiments with challenging, cluttered scenes show that the segmentation procedure provides salient parts of objects which lead to a good categorization performance using the graph-based constellation model concept.

We present Spectral Registration with Multilayer Resampling (SRMR) as a 6 Degrees Of Freedom (DOF) registration method for noisy 3D data with partial overlap. The algorithm is based on decoupling 3D rotation from 3D translation by a corresponding resampling process of the spectral magnitude of a 3D Fast Fourier Transform (FFT) calculation on discretized 3D range data. The registration of all 6DOF is then subsequently carried out with spectral registrations using Phase Only Matched Filtering (POMF). There are two main aspects for the fast and robust registration of Euler angles from spherical information in SRMR. First of all, there is the permanent use of phase matching. Second, based on the FFT on a discrete Cartesian grid, not only one spherical layer but also a complete stack of layers are processed in one step. Experiments are presented with challenging datasets with respect to interference and overlap. The results include the fast and robust registration of artificially transformed data for ground-truth comparison, scans from the Stanford Bunny dataset, high end 3D laser range finder (LRF) scans of a city center, and range data from a low-cost actuated LRF in a disaster response scenario.

An uncertainty estimation method for 6 degree of freedom (6-DoF) spectral registration is introduced here. The underlying 6-DoF registration method based on Phase Only Matched Filtering (POMF) is capable of dealing with very noisy sensor data. It is hence well suited for 3D underwater mapping, where relatively inaccurate sonar imaging devices have to be employed. An uncertainty estimation method is required to use this registration method in a Simultaneous Localization and Mapping (SLAM) framework. To our knowledge, the first such method for 6-DoF spectral registration is presented here. This new uncertainty estimation method treats the POMF results as probability mass functions (PMF). Due to the decoupling in the underlying method, yaw is computed by a one-dimensional POMF leading hence to a 1D PMF; roll and pitch are simultaneously computed and hence encoded in a 2D PMF. Furthermore, a 3D PMF is generated for the translation as it is determined by a 3D POMF. A normal distribution is fitted on each of the PMF to get the uncertainty estimate. The method is experimentally evaluated with simulated as well as real world sonar data. It is shown that it indeed can be used for SLAM, which significantly improves the map quality.

In this paper, we give an overview of the Jacobs Robotics entry to the ICRA'11 Solutions in Perception Challenge. We present our multi-pronged strategy for object recognition and localization based on the integrated geometric and visual information available from the Kinect sensor. Firstly, the range image is over-segmented using an edge-detection algorithm and regions of interest are extracted based on a simple shape-analysis per segment. Then, these selected regions of the scene are matched with known objects using visual features and their distribution in 3D space. Finally, generated hypotheses about the positions of the objects are tested by back-projecting learned 3D models to the scene using estimated transformations and sensor model. Our method won the second place among eight competing algorithms, only marginally losing to the winner.

This paper presents a robust spectral based 6 Degrees Of Freedoms (DOF) registration method for 3D data which is in addition capable to determine the scaling of the registered object. The principle is based on decoupling 3D rotation from 3D translation by a corresponding resampling process of the spectral magnitude using a 3D Fast Fourier Transform (FFT), a SO(3) Fourier transform and an integrated Mellin transform for a scale determination. The registration is then subsequently carried out, first the registration of rotation is done, in a second step the scale is determined and finally the translation is calculated. Results are presented with generated test series on public available data and data recorded with Laser Range Finders (LRF). Furthermore one example of a scan pair recorded under realistic conditions showing the robustness of this method is given.

There are many common error sources that influence mapping, e.g., salt and pepper noise as well as other effects occurring quite uniformly distributed over the map. On the other hand, there are also errors, which occur very rarely but with severe effects. These errors influence not only the local accuracy but the overall spatial layout of the map. Concrete examples include bump noise in the robot's pose or residual errors in Simultaneous Localization and Mapping (SLAM). Brokenness is presented here to capture one form of structural errors in grid maps. Concretely, brokenness measures the degree with which a map can be partitioned into regions that are locally consistent with ground truth but “off” relative to each other. The concept of brokenness is presented in a formal way and it is shown how it can be computed in an efficient way using recursive spectral registrations. Experimental results show that the metric can indeed be used to automatically determine one structural quality aspect of a map in a quantitative way.

A core challenge in probabilistic mapping is to extract meaningful uncertainty information from data registration methods. While this has been investigated in ICP-based scan matching methods, other registration methods have not been analyzed. In this paper, an uncertainty analysis of a Fourier Mellin based image registration algorithm is introduced, which to our knowledge is the first of its kind involving spectral registration. A covariance matrix is extracted from the result of a Phase-Only Matched Filter, which is interpreted as a probability mass function. The method is embedded in a pose graph implementation for Simultaneous Localization and Mapping (SLAM) and validated with experiments in the underwater domain.

Fitting optimum quadrics on segmented noisy range-image is a challenging task. The algebraic least-squares fit usually considered in the literature is biased, although it is fast to compute. We extend our previous work in planar patch extraction using a detailed sensor noise model to the extraction of quadrics. First, the fast least-squares method is modified to detect degeneracy automatically and it is used to aid segmentation. The final segmented quadric patch is then refined using a numerical maximum likelihood approach which employs the sensor range error model. Experimental results for artificial data and two different 3D sensors are provided to show the feasibility of our approach.

The recently introduced Minimum Uncertainty Maximum Consensus (MUMC) algorithm for 3D scene registration using planar-patches is tested in a large outdoor urban setting without any prior motion estimate whatsoever. With the aid of a new overlap metric based on unmatched patches, the algorithm is shown to work successfully in most cases. The absolute accuracy of its computed result is corroborated for the first time by ground-truth obtained using reflective markers. There were a couple of unsuccessful scan-pairs. These are analyzed for the reason of failure by formulating two kinds of overlap metrics: one based on the actual overlapping surface-area and another based on the extent of agreement of range-image pixels. We conclude that neither metric in isolation is able to predict all failures, but that both taken together are able to predict the difficulty level of a scan-pair vis-à-vis registration by MUMC.

Surface-patches based 3D mapping in a real world underwater scenario is presented. It is based on a 6 degrees of freedom registration of sonar data. Planar surfaces are fitted into the sonar data and the subsequent registration method maximizes the overall geometric consistency within a search-space to determine correspondences between the planes. This approach has previously only been used on high quality range data from sensors on land robots like laser range finders. It is shown here that the algorithm is also applicable to very noisy, coarse sonar data. The 3D map presented is of a large underwater structure, namely the Lesumer Sperrwerk, a flood gate north of the city of Bremen, Germany. It is generated from 18 scans collected using a Tritech Eclipse sonar.

The online generation of underwater image maps or mosaicking is of high interest for underwater robots, e.g., for autonomous navigation, exploration, or object detection. Here, a cooperative approach is presented that addresses the particular challenges of the severe constraints on communication bandwidth in the underwater domain. Concretely, a special update strategy for a cooperatively maintained pose graph as basis for Simultaneous Localization and Mapping (SLAM) is introduced. The strategy tries to transmit the most relevant information within the limits of the communication bandwidth to maximize the quality of the cooperative map. It is shown in experiments with simulations based on real world data that the strategy leads to near optimal results while obeying the severe bandwidth constraints of realistic underwater communication.

We present a robot-pose-registration algorithm, which is entirely based on large planar-surface patches extracted from point clouds sampled from a three-dimensional (3-D) sensor. This approach offers an alternative to the traditional point-to-point iterative-closest-point (ICP) algorithm, its point-to-plane variant, as well as newer grid-based algorithms, such as the 3-D normal distribution transform (NDT). The simpler case of known plane correspondences is tackled first by deriving expressions for least-squares pose estimation considering plane-parameter uncertainty computed during plane extraction. Closed-form expressions for covariances are also derived. To round-off the solution, we present a new algorithm, which is called minimally uncertain maximal consensus (MUMC), to determine the unknown plane correspondences by maximizing geometric consistency by minimizing the uncertainty volume in configuration space. Experimental results from three 3-D sensors, viz., Swiss-Ranger, University of South Florida Odetics Laser Detection and Ranging, and an actuated SICK S300, are given. The first two have low fields of view (FOV) and moderate ranges, while the third has a much bigger FOV and range. Experimental results show that this approach is not only more robust than point- or grid-based approaches in plane-rich environments, but it is also faster, requires significantly less memory, and offers a less-cluttered planar-patches-based visualization.

In this work, we utilize a recently studied more accurate range noise model for 3D sensors to derive from scratch the expressions for the optimum plane which best fits a point-cloud and for the combined covariance matrix of the plane's parameters. The parameters in question are the plane's normal and its distance to the origin. The range standard-deviation model used by us is a quadratic function of the true range and is a function of the incidence angle as well. We show that for this model, the maximum-likelihood plane is biased, whereas the least-squares plane is not. The plane-parameters' covariance matrix for the least-squares plane is shown to possess a number of desirable properties, e.g., the optimal solution forms its null-space and its components are functions of easily understood terms like the planar-patch's center and scatter. We verify our covariance expression with that obtained by the eigenvector perturbation method. We further compare our method to that of renormalization with respect to the theoretically best covariance matrix in simulation. The application of our approach to real-time range-image registration and plane fusion is shown by an example using a commercially available 3D range sensor. Results show that our method has good accuracy, is fast to compute, and is easy to interpret intuitively.

A fast and robust method for visual odometry based on the Fourier-Mellin Invariant (FMI) descriptor is presented. It extends previous FMI based approaches in two ways. First, a logarithmic representation of the spectral magnitude of the FMI descriptor is used. Second, a filter on the frequency where the shift is supposed to appear is applied. It is shown with experiments with an Unmanned Aerial Vehicle that this improved Fourier-Mellin Invariant (iFMI) method is is indeed an advancement and well suited for online visual odometry to generate large photo maps.

This article addresses fast 3D mapping by a mobile robot in a predominantly planar environment. It is based on a novel pose registration algorithm based entirely on matching features composed of plane-segments extracted from point-clouds sampled from a 3D sensor. The approach has advantages in terms of robustness, speed and storage as compared to the voxel based approaches. Unlike previous approaches, the uncertainty in plane parameters is utilized to compute the uncertainty in the pose computed by scan-registration. The algorithm is illustrated by creating a full 3D model of a multi-level robot testing arena.

Exploration of unknown environments remains one of the fundamental problems of mobile robotics. It is also a prime example for a task that can benefit significantly from multi-robot teams. We present an integrated system for semi-autonomous cooperative exploration, augmented by an intuitive user interface for efficient human supervision and control. In this preliminary study we demonstrate the effectiveness of the system as a whole and the intuitive interface in particular. Congruent with previous findings, results confirm that having a human in the loop improves task performance, especially with larger numbers of robots. Specific to our interface, we find that even untrained operators can efficiently manage a decently sized team of robots.

Pixel array based 3D range sensors like time of flight sensors (e.g. Swiss-ranger) are commonly used for spatial mapping. Analogous to a laser range finder beam, each pixel measures distances within its conical field of view. However, unlike laser range finders, these sensors use incoherent visible or near visible light for range measurement. This, combined with their relatively long maximum ranges, means that the convenient thin beam assumption can no longer be used to update an occupancy grid map using a probabilistic sensor model. We present an analysis of a scenario where a pixel beam is intersected by more than one object and where a unimodal probabilistic distribution assumption causes spurious objects to appear in the detected scene. This adversely affects path-planning based on such maps because perfectly good escape routes are blocked off. A two step multimodal Gaussian mixture model based procedure is presented which is able to detect multiple obstacles per pixel and hence ameliorate the problem. The results of experiments done using a time of flight based sensor are presented.

Robot mapping is a task that can benefit a lot from cooperative multi-robot systems. In multi-robot simultaneous localization and mapping (SLAM), it becomes very important how efficiently a given map can be shared among the robot team. To this end, the recently proposed pose graph map representation is used, adapted for use in a particle filter based mapping algorithm, and compared to the standard occupancy grid representation. Through analysis of corner cases as well as experiments with real robot data, the two map representations are thoroughly compared. It is shown that the pose graph representation allows for much more efficient communication of map updates than the standard occupancy grid.

A fast but nevertheless accurate approach for surface extraction from noisy 3D point clouds is presented. It consists of two parts, namely a plane fitting and a polygonalization step. Both exploit the sequential nature of 3D data acquisition on mobile robots in form of range images. For the plane fitting, this is used to revise the standard mathematical formulation to an incremental version, which allows a linear computation. For the polygonalization, the neighborhood relation in range images is exploited. Experiments are presented using a time-of-flight range camera in form of a Swissranger SR-3000. Results include lab scenes as well as data from two runs of the rescue robot league at the RoboCup German Open 2007 with 1,414, respectively 2,343 sensor snapshots. The 36ldr106, respectively 59ldr106points from the two point clouds are reduced to about 14ldr103, respectively 23ldr103planes with only about 0.2 sec of total computation time per snapshot while the robot moves along. Uncertainty analysis of the computed plane parameters is presented as well.

This paper presents a new technique for the update of a probabilistic spatial occupancy grid map using a forward sensor model. Unlike currently popular inverse sensor models, forward sensor models can be found experimentally and can represent sensor characteristics better. The formulation is applicable to both 2D and 3D range sensors and does not have some of the theoretical and practical problems associated with the current approaches which use forward models. As an illustration of this procedure, a new prototype 3D forward sensor model is derived using a beam represented as a spherical sector. Furthermore, this model is used for fusion of point-clouds obtained from different 3D sensors, in particular, time-of-flight sensors (Swiss-ranger, laser range finders), and stereo vision cameras. Several techniques are described for an efficient data-structure representation and implementation. The range beams from different sensors are fused in a common local Cartesian occupancy map. Experimental results of this fusion are presented and evaluated using Hough-transform performed on the grid.

Mapping can potentially be speeded up in a significant way by using multiple robots exploring different parts of the environment. But the core question of multirobot mapping is how to integrate the data of the different robots into a single global map. A significant amount of research exists in the area of multirobot mapping that deals with techniques to estimate the relative robots poses at the start or during the mapping process. With map merging, the robots in contrast individually build local maps without any knowledge about their relative positions. The goal is then to identify regions of overlap at which the local maps can be joined together. A concrete approach to this idea is presented in form of a special similarity metric and a stochastic search algorithm. Given two maps m and m', the search algorithm transforms m' by rotations and translations to find a maximum overlap between m and m'. In doing so, the heuristic similarity metric guides the search algorithm toward optimal solutions. Results from experiments with up to six robots are presented based on simulated as well as real-world map data

The CubeSystem is a collection of hardware- and software-components for fast robot prototyping. The main goal of the Cubesystem project is to provide an open source collection of generic building blocks that can be freely combined into an application. This paper describes the first release of the CubeSystem, that evolved in more than five years of research and development. The benefits of the CubeSystem are illustrated by several applications, ranging from educational activities to industrial projects.

In behavior-oriented robotics, the control of a system is distributed over various processes or behaviors running in virtual parallel. However, the assignment of processing power to different processes is a non-trivial task. Existing approaches to this problem, like rate-monotonic scheduling, focus on the fulfilment of deadlines, i.e., upper time-bounds. For behavioral control, the periodicity of processes is also of interest. Here, a novel scheduling algorithm for behavioral processes at different time-scales is introduced, which ensures time-optimal and periodically balanced execution.

The so-called RoboGuard is a mobile security device which is tightly integrated into the existing surveillance framework developed and marketed by Quadrox, a Belgian SME. RoboGuards are semi-autonomous mobile robots providing video streams via wireless Intranets to existing watchguard systems, supplemented by various basic and optional behaviors. RoboGuards fill several market-niches. Especially, they are a serious alternative to the standard approach of using closed circuit television for surveillance. Low production cost, user friendliness, and support for easy add-on functionalities were important design targets, which were achieved by following the AI paradigm of behavior-oriented robotics.

Describes the issues in robotics from a viewpoint of edutainment through a series of activities in the Robot World Cup Initiative and related events, such as the International Robot Games Festival (Robofesta) supported by the Japanese government to promote creative and imaginative education programs, RoboCup Jr. which is designed for kids and the younger generation to play RoboCup games with easily constructible platforms, development of small legged robots for pets in the house or games, and education projects in system engineering. Finally, concluding remarks for future activities are given.