For a variety of use cases, the utility of autonomous mobile robots is strengthened by a capacity to explore spatial environments with particular focus on objects or phenomena of interest. To generally support such a capacity, an agile, application-agnostic interest operator is constructed to visually guide focused autonomous exploration. An interest operator is proposed that serves to identify unique features in a scene in an application-agnostic manner by maintaining a sort of “short-term memory” and identifying regions that are unique within the context of what has already been ascertained about the scene, not necessarily what is unique or of interest for a given application. The proposed interest operator is particularly suited to render-based mapping and perception, taking advantage of the natural characterization of spaces comprising an environment as free, occupied, and unknown, and the rich, highly-detailed texture information supported in graphical renderings of visual scenes. With focus on two scales of image features, one being driven by discontinuities between surfaces and the other based on variations in texture of surfaces in a region, the proposed interest operator can inform robot sensing and navigation decisions and resource allocation during autonomous exploration. Results from numerical experiments demonstrate the efficacy of the approach.

We consider the task of actively exploring unstructured environments to facilitate robotic grasping of occluded objects. Typically, the geometry and locations of these objects are not known a priori. We mount an RGB-D sensor on the robot gripper to maintain a 3D voxel map of the environment during exploration. The objective is to plan the motion of the sensor in order to search for feasible grasp handles that lie within occluded regions of the map. In contrast to prior work that generates exploration trajectories by sampling, we directly optimize the exploration trajectory to find grasp handles. Since it is challenging to optimize over the discrete voxel map, we encode the uncertainty of the positions of the occluded grasp handles as a mixture of Gaussians, one per occluded region. Our trajectory optimization approach encourages exploration by penalizing a measure of the uncertainty. We then plan a collision-free trajectory for the robot arm to the detected grasp handle. We evaluated our approach by actively exploring and attempting 300 grasps. Our experiments suggest that compared to the baseline method of sampling 10 trajectories, which successfully grasped 58% of the objects, our active exploration formulation with trajectory optimization successfully grasped 93% of the objects, was 1.3× faster, and had 3.2× fewer failed grasp attempts.

Localization and modeling of stairways by mobile robots can enable multi-floor exploration for those platforms capable of stair traversal. Existing approaches focus on either stairway detection or traversal, but do not address these problems in the context of path planning for the autonomous exploration of multi-floor buildings. We propose a system for detecting and modeling ascending stairways while performing simultaneous localization and mapping, such that the traversability of each stairway can be assessed by estimating its physical properties. The long-term objective of our approach is to enable exploration of multiple floors of a building by allowing stairways to be considered during path planning as traversable portals to new frontiers. We design a generative model of a stairway as a single object. We localize these models with respect to the map, and estimate the dimensions of the stairway as a whole, as well as its steps. With these estimates, a robot can determine if the stairway is traversable based on its climbing capabilities. Our system consists of two parts: a computationally efficient detector that leverages geometric cues from dense depth imagery to detect sets of ascending stairs, and a stairway modeler that uses multiple detections to infer the location and parameters of a stairway that is discovered during exploration. We demonstrate the performance of this system when deployed on several mobile platforms using a Microsoft Kinect sensor.

Concurrently estimating the 6-DOF pose of multiple cameras or robots - cooperative localization - is a core problem in contemporary robotics. Current works focus on a set of mutually observable world landmarks and often require inbuilt egomotion estimates; situations in which both assumptions are violated often arise, for example, robots with erroneous low quality odometry and IMU exploring an unknown environment. In contrast to these existing works in cooperative localization, we propose a cooperative localization method, which we call mutual localization, that uses reciprocal observations of camera-fiducials to obviate the need for egomotion estimates and mutually observable world landmarks. We formulate and solve an algebraic formulation for the pose of the two camera mutual localization setup under these assumptions. Our experiments demonstrate the capabilities of our proposal egomotion-free cooperative localization method: for example, the method achieves 2cm range and 0.7 degree accuracy at 2m sensing for 6-DOF pose. To demonstrate the applicability of the proposed work, we deploy our method on Turtlebots and we compare our results with ARToolKit [1] and Bundler [2], over which our method achieves a tenfold improvement in translation estimation accuracy.

Conversion of unorganized point clouds to surface reconstructions is increasingly required in the mobile robotics perception processing pipeline, particularly with the rapid adoption of RGB-D (color and depth) image sensors. Many contemporary methods stem from the work in the computer graphics community in order to handle the point clouds generated by tabletop scanners in a batch-like manner. The requirements for mobile robotics are different and include support for real-time processing, incremental update, localization, mapping, path planning, obstacle avoidance, ray-tracing, terrain traversability assessment, grasping/manipulation and visualization for effective human-robot interaction. We carry out a quantitative comparison of Greedy Projection and Marching cubes along with our voxel planes method. The execution speed, error, compression and visualization appearance of these are assessed. Our voxel planes approach first computes the PCA over the points inside a voxel, combining these PCA results across 2×2×2 voxel neighborhoods in a sliding window. Second, the smallest eigenvector and voxel centroid define a plane which is intersected with the voxel to reconstruct the surface patch (3-6 sided convex polygon) within that voxel. By nature of their construction these surface patches tessellate to produce a surface representation of the underlying points. In experiments on public datasets the voxel planes method is 3 times faster than marching cubes, offers 300 times better compression than Greedy Projection, 10 fold lower error than marching cubes whilst allowing incremental map updates.

Many robotics platforms are capable of ascending stairways, but all existing approaches for autonomous stair climbing use stairway detection as a trigger for immediate traversal. In the broader context of autonomous exploration, the ability to travel between floors of a building should be compatible with path planning, such that the robot can traverse a stairway at a time that is appropriate to its navigation goals. No system yet presented is capable of both localizing stairways on a map and estimating their properties, functions that in combination would enable stairways to be considered as traversable terrain in a path planning algorithm. We propose a method for modeling stairways as objects and localizing them on a map, such that they can be subsequently traversed if they are of dimensions that the robotic platform is capable of climbing. Our system consists of two parts: a computationally efficient detector that leverages geometric cues from depth imagery to detect sets of ascending stairs, and a stairway modeler that uses multiple detections to infer the location and parameters of a stairway that is discovered during exploration. This video demonstrates the performance of the system in a number of real-world situations, modeling and localizing a variety of stairway types in both indoor and outdoor environments.

In order to achieve good and timely volumetric mapping for mobile robots, we improve the speed and accuracy of multi-resolution voxel map building from 3D data. Mobile robot capabilities, such as SLAM and path planning, often involve algorithms that query a map many times and this lookup is often the bottleneck limiting the execution speed. As such, fast spatial proximity queries has been the topic of much active research. Various data structures have been researched including octrees, k-d trees, approximate nearest neighbours and even dense 3D arrays. We tackle this problem by extending previous work that stores the map as a hash table containing occupied voxels at multiple resolutions. We apply Bloom filters to the problem of spatial querying and voxel maps for the example application of SLAM. Their efficacy is demonstrated building 3D maps with both simulated and real 3D point cloud data. Looking up whether a voxel is occupied is three times faster than the hash table and within 10% of the speed of querying a dense 3D array, potentially the upper limit to query speed. Map generation was done with scan to map alignment on simulated depth images, for which the true pose is available. The calculated poses exhibited sub-voxel error of 0.02m and 0.3 degrees for a typical indoor scene with a map resolution of 0.04m.

The tasks of region or object segmentation and environment change detection in a 3D context are investigated and tested on an autonomous skid steer loader. This is achieved through a technique analogous to background subtraction utilising 3D scan data which is first aligned before a voxel sub traction operation against a prior map. We highlight the close relationships between the scan-to-map alignment, background subtraction and 3D scan-to-map matching problems. The presented approaches take advantage of previous work on the multi-resolution occupied voxels list (MROL) representations for 3D spatial maps. This prior work is augmented to provide a mechanism for fast local 6DOF alignment with the same data structures (MROL) that have previously been shown to allow efficient global localisation. The new approach is then compared to an iterative closest point (ICP) implementation and was found to execute in a similar amount of time, but was more robust and accurate. The local alignment algorithm is inherently more amenable to the MROL representation with an associated reduction in implementation complexity and negligible parameter tuning. The hash value basis of MROL results in a map representation that can be both updated and queried in constant time regardless of mapped volume. The approach described was tested on an autonomous skid steer loader as part of the dig-planning process by segmenting piles of material and detecting humans in the scene.

The characteristics of a variety of 3D lattices are assessed for their performance when applied to typical robotics problems. The lattices studied are the Cubic, Body Centred Cubic (BCC), Face Centred Cubic (FCC), hexagonal prismatic and finally the Mean Centred Cuboidal (MCC). An algorithm for generic quantization to any low dimensional lattice is presented allowing this analysis to be easily extended to other 3D lattices of interest. Tests are undertaken on uniform sampled random data and laser range data from mobile robot platforms including an autonomous skid steer loader. The improvements in accuracy, memory requirements and consistency under rotation for the alternative lattices over the cubic lattice are typically 5-10%. The radial distribution of lattice points is studied through the distribution of points assigned to the same lattice cell and that in neighbouring cells. For instance only 12 neighbouring cells need checking for the FCC lattice as opposed to the cubic lattice which requires 26. Not only are fewer checks required but the distance variation associated with points in adjacent voxels of the FCC lattice is substantially lower than that of a cubic lattice. For the FCC lattice these point distances have a 30% smaller standard deviation and a 40% smaller range. These results make algorithms, such as collision checking and scan/map matching, which often involve many proximity checks, significantly faster and more accurate.

A novel and inexpensive mechanism for acquiring 3D data suitable for indoor mobile robots is introduced. The scanner is currently capable of ranging up to 4 m with an accuracy of 0.05 m and costs less than 500 USD. The scans cover the whole 360 degrees around the robot at heights from floor level to 1 m. The data so obtained is particularly applicable to navigation, mapping and obstacle avoidance. The scanner consists of a webcam, visual encoder disk, angled laser line projector and determines range based on triangulation. The scan rate depends upon the frame rate of the camera and its rotation speed.

An alternative to the conventional quantization for occupied voxel lists in both 2D and 3D is presented. The performance metrics of the hexagonal lattice in 2D and the face centred and body centred cubic lattices in 3D are investigated and compared to their square and cubic counterparts. It is found that quantization to alternative lattices yields some improvements. Ultimately, the D3 or face centred cubic lattice is highlighted for its lower quantization error, lower rotation variability and higher order rotational symmetry. It has three times less occupied voxel count pose variability than a standard cubic occupied voxel list. These improvements have implications for SLAM and path planning.

Map building through cooperative localisation (co-location) using circular geometric targets and a SICK laser range scanner is investigated. The tenet of co-location is circle detection in laser range data. Two methods for circle detection, a Range Weighted Circular Hough Transform (RWCHT) and a novel squared-residual voting strategy are compared and their performance assessed. The custom squared-residual voting strategy outperforms the RWCHT in all respects and is subsequently used for localisation and map building. The results include robust continuous localisation at speeds of 0.2m/s with 98% of scan frames used and an error of less than 0.03m. This localisation accuracy helps build maps of 96% quality and occupancy grids of cluttered environments despite the presence of distractors.