We propose a novel four-stage pipeline densely reconstructing depth from video sequences with small baselines with the goal of being fast. In our pipeline we make use of the sub-pixel precision of direct photometric bundle adjustment to reduce the number of tracked points required to estimate an accurate pose. Instead of using the exhaustive plane sweeping approach of existing small baseline methods, dense depth maps are calculated efficiently using an algorithm inspired by PatchMatch. Instead of a stereo matching error, our algorithm minimizes the variance over multiple frames with a robust mean estimation. The experiment results suggest that our method has the ability to cope with a wider range of baselines and sequence sizes. We also compare qualitative results on real small motion clips from Ha et al. in addition to our own, and show that our method outputs dense depth maps of similar or better quality and at least 10x faster.

Recent work in dense monocular 3D reconstruction relies on dense pixel correspondences and assumes brightness constancy and saliency, and thus are fundamentally unable to reconstruct low-textured or non-lambertian objects such as glass or metal. Occlusion boundaries differ from texture in that each unique view generates a unique set of occlusions. By detecting and solving for the depths of occlusion boundaries, we show how dense reconstructions of challenging objects can be integrated with existing monocular reconstruction algorithms by compensating with an increasing number of unique views. In a sequence containing the Stanford Bunny (8.5 cm) the points of our reconstructed edge cloud have an RMS error of 2.3 mm.

We describe an interactive approach for visual object analysis which exploits the ability of a robot to manipulate its environment. Knowledge of objects' mechanical properties is important in a host of robotics tasks, but their measurement can be impractical due to perceptual or mechanical limitations. By applying a periodic stimulus and matched video filtering and analysis pipeline, we show that even stiff, fragile, or low-texture objects can be distinguished based on their mechanical behaviours. We construct a novel, linear filter exploiting periodicity of the stimulus to reduce noise, enhance contrast, and amplify motion by a selectable gain - the proposed filter is significantly simpler than previous approaches to motion amplification. We further propose a set of statistics based on dense optical flow derived from the filtered video, and demonstrate visual object analysis based on these statistics for objects offering low contrast and limited deflection. Finally, we analyze 7 object types over 59 trials under varying illumination and pose, demonstrating that objects are linearly distinguishable under this approach, and establish the viability of estimating fluid level in a cup from the same statistics.

Legged robots offer the potential of locomotion across various types of terrains. Different terrains require different gait patterns to enable greater traversal efficiency. Consequently, as a legged robot transitions from one type of terrain to another, the gait pattern should be adapted so as to maximise traction and energy efficiency. This paper explores the use of power consumption as estimated by the robot in real-time for guiding this gait transition in the case of statically-stable locomotion. While moving, the robot autonomously assesses its power consumption, relates it to the traction, and switches between gaits so as to maximise efficiency. In this way, the robot only needs proprioceptive sensors and consequently does not require velocity estimation, ground imaging or profiling to maintain efficient locomotion across different terrains. The approach has been tested on a hexapod robot traversing a variety of terrain types and stiffness, including concrete, grass, mulch and leaf litter. The experimental results show that gait switching on energetics alone enables traction maintenance and efficient locomotion across different terrains. We also present comparisons between the power consumption metric used in this work and cost of transport which is used in the literature for characterising energetics for legged locomotion.

For agile, accurate autonomous robotics, it is desirable to plan motion in the presence of uncertainty. The Partially Observable Markov Decision Process (POMDP) provides a principled framework for this. Despite the tremendous advances of POMDP-based planning, most can only solve problems with a small and discrete set of actions. This paper presents General Pattern Search in Adaptive Belief Tree (GPS-ABT), an approximate and online POMDP solver for problems with continuous action spaces. Generalized Pattern Search (GPS) is used as a search strategy for action selection. Under certain conditions, GPS-ABT converges to the optimal solution in probability. Results on a box pushing and an extended Tag benchmark problem are promising.

This paper introduces autonomous sorting of moving coins via a robot as a multi-level task that supports the principled study of robotics fundamentals including kinematics, dynamics, perception, motion planning, controls, and optimization based around a widely obtainable, standardized, low-cost object (a coin). The paper also presents a demonstrated solution to this in the form of the CHARM (Coin Handling Arm for Robotics Mastery) robot, which addresses the autonomous coin sorting problem using an economical kit made from commodity computing hardware and three Dynamixel servomotors. From a learning perspective, this problem facilitates interdisciplinary practice across subject and grade levels with an algorithmic foundation that is central to modern robotics. Evidence supporting this approach is illustrated from case studies of student projects and, in particular, the CHARM robot. Beyond practice alone, by presenting a challenging (but manageable) research problem, we found that the coin sorting task teaches robotics in a principled way. Further, algorithmic complexity tiers the problem to academic levels. While motivated by robotics education, the (optimal) coin sorting problem may also be seen as an archetype problem for manipulation/motion-planning research. Thus, this also promotes a research foundation supporting later research opportunities.

This paper presents a novel approach for constructing light-weight, low-cost air-deployable sensor modules and miniature unmanned aerial vehicles which consist of printed circuit boards as integrated electronic, structural and aerodynamic lift-producing devices. In this way, each aircraft can be a single-piece, single-manufacturing step assembly, reducing materials, processing and labour costs. This approach allows the fabrication of devices that are sufficiently inexpensive so as to be economic for single-use or disposable applications, such as long-range sensor deployment for environmental monitoring. We present two example low-cost, proof-of-concept aircraft employing electro-aeromechanical structures - rotary-wing and fixed-wing - and demonstrate their flight performance.

Air conditioning systems are generally the largest systems in a home, both physically and energetically, and thus central to home automation efforts. Finding a balance between user comfort and efficiency is a complex problem given the considerable variation present. This paper focuses on comfort, as opposed to absolute temperature, as gauged using simple, sparse user inputs. A thermal model of the house is learned which accounts for weather data and exogenous factors such as occupancy. By incorporating user feedback, a Learning-Based Model Predictive Controller (LBMPC) is able adapt to home conditions and more efficiently operate the system. In contrast to previous efforts which operate in office spaces and to a set point, this work is adapted and tested in a typical home environment and closes a control loop on user comfort. The controller considers that the user's comfort levels may change during the day, for example when the user is in bed, or not at home. It shows that complex systems may be automated without extensive tweaking by the user and in a manner that considers user comfort, time of day, and related factors to reduce energy consumption.

From exploring planets to cleaning homes, the reach and versatility of robotics is vast. The integration of actuation, sensing and control makes robotics systems powerful, but complicates their simulation. This paper introduces a versatile, scalable, yet powerful general-purpose robot simulation framework called V-REP. The paper discusses the utility of a portable and flexible simulation framework that allows for direct incorporation of various control techniques. This renders simulations and simulation models more accessible to a general-public, by reducing the simulation model deployment complexity. It also increases productivity by offering built-in and ready-to-use functionalities, as well as a multitude of programming approaches. This allows for a multitude of applications including rapid algorithm development, system verification, rapid prototyping, and deployment for cases such as safety/remote monitoring, training and education, hardware control, and factory automation simulation.

Model uncertainty complicates most kinodynamic motion planning and control approaches due to their reliance on accurate forward prediction. If the model uncertainty is significant, a generated path or control strategy based on forward simulation of this model is potentially invalid and expensive to track (if possible). This paper explores the use of system identification/estimation to tune model parameters. Framed as an extension to rapidly exploring random tree (RRT) methods, it updates the model so that reachable actions added to the tree have more fidelity. This can be viewed as a mixture of a model predictive control (MPC) for local planning with an approximate-model global planner providing sub-goals and thus overcoming the limited lookahead caused by model uncertainty. The benefits of this approach are illustrated for a 3 DOF serial manipulator controlled by computed torque control operating under large external disturbances. In this case, the approach provides operation under intermittent feedback and disturbance observation. Tracking and actuator utilization are also improved over solutions found via conventional methods.

Observing human motion patterns is informative for social robots that share the environment with people. This paper presents a methodology to allow a robot to navigate in a complex environment by observing pedestrian positional traces. A continuous probabilistic function is determined using Gaussian process learning and used to infer the direction a robot should take in different parts of the environment. The approach learns and filters noise in the data producing a smooth underlying function that yields more natural movements. Our method combines prior conventional planning strategies with most probable trajectories followed by people in a principled statistical manner, and adapts itself online as more observations become available. The use of learning methods are automatic and require minimal tuning as compared to potential fields or spline function regression. This approach is demonstrated testing in cluttered office and open forum environments using laser and vision sensing modalities. It yields paths that are similar to the expected human behaviour without any a priori knowledge of the environment or explicit programming.

Operational space control has a number of desirable characteristics but is sensitive to model accuracy. For heavy machines the dynamics are difficult to model due to their friction and dynamic coupling, thus making full compensation imprecise. This work presents an approach in which a simplified model gives partial compensation via an open-loop feedforward input, pre-calculated in forward simulation. In this way, effects that are difficult to compensate for can be partially corrected without causing instability. Since the reference trajectory is known a priori, dynamic model parameters are tuned in its neighbourhood, reducing the burden of global modelling. The feasibility and performance of this approach is shown experimentally via improved free motion tracking of an excavator arm. This framework further supports efforts for direct impedance control between bucket tip and soil.

Trajectory generation and control of large equipment in open field environments involves systematically and robustly operating in uncertain and dynamic terrain. This paper presents an integrated motion planning and control system for tracked vehicles. Flexible path-end adjustments and adaptive look-ahead are introduced to a state lattice planning approach with waypoint control. For a given processing horizon, this increases search coverage and reduces planning error. This tramming approach has been successfully fielded on a 98-ton autonomous blast hole drill rig used in iron ore mining in Western Australia. The system has undergone extensive testing and is now integrated into a production environment. This work is a key element in a larger program aimed at developing a fully autonomous, remotely operated mine.

This paper presents an algorithm for segmenting 3D point clouds. It extends terrain elevation models by incorporating two types of representations: (1) ground representations based on averaging the height in the point cloud, (2) object models based on a voxelisation of the point cloud. The approach is deployed on Riegl data (dense 3D laser data) acquired in a campus type of environment and compared against six other terrain models. Amongst elevation models, it is shown to provide the best fit to the data as well as being unique in the sense that it jointly performs ground extraction, overhang representation and 3D segmentation. We experimentally demonstrate that the resulting model is also applicable to path planning.

A new control strategy based on feedback motion planning is presented for solving nonlinear control problems in constrained environments. The algorithm explores the state-space using a bi-directional rapidly exploring random tree (biRRT) in order to find a feasible trajectory between an initial and goal state. By incrementally scheduling LQR controllers, it attempts to connect states so as to link the two trees. These attempts are evaluated by verifying that the connected state is inside the controllable area of an infinite time horizon controller at the goal. This allows for a rapid delineation of equivalent neighborhoods in the state-space. As a result, random exploration is terminated as soon as a feasible solution is made possible by feedback means, avoiding oversampling and partially introducing optimal actions at the neighborhood of the connection. The algorithm is demonstrated and compared against a biRRT using single-link pendulum and cart-pole swing-up tasks amongst obstacles, the latter showing a nearly order of magnitude more efficient search.

Trotting and galloping allow a quadruped to rapidly traverse rough terrain. Modeling this motion, which is only dynamically stable, is of importance for legged robot operation and for quadrupedal animal motion estimation. Derived from an eight-step galloping cycle, this study presents a kinetic hybrid model in which the states vary based on the principal forces present. As compared to foot contact, or kinematic, hybrid quadruped models this reduces the maximum number of possible states from 120 to 6 and provides an alternative to foot contact monitoring. This approach was tested on a trotting quadruped robot equipped with an inertial sensor aided by video. This was processed using an EKF estimator framework to give attitude estimates at rates of up to 250 Hz with 5deg error.

An important tool in clarifying various theories governing the dynamics of rapid quadrupedal gaits, such as the trot and gallop, is the measurement of body attitude. Such measurements are complicated in open field environments because of the large ranges and high data rates needed due to the speeds (7 m/s) and rapid shifts in dynamics present. To address this a lightweight inertial sensing harness is introduced with sensing design based on the KOLT robot. Its mass center is collocated with the subject so as to reduce dynamic bias. This work combines dynamic gait system identification and motion estimation and is demonstrated on a Labrador retriever (Canis lupus familiaris) through measurements of the gallop over long spans (20 m) and at data rates comparable with gait laboratories (400 Hz). The results are consistent with laboratory measurements, but seem to suggest a roll and yaw cross-coupling during gallop.

Manipulator operations are complicated by sensor noise, mechanical compliance, and system bias. These uncertainties are compounded in field environments, such as those encountered in humanitarian demining. By taking advantage of the generally flat and static structure of the terrain, a series of adaptive corrections and filters refine a sensed topographical model and generate a trajectory that is robust to inherent inaccuracies and modeling errors. Experimental testing on a mobile robot using stereo-vision as modeling sensor indicates that this method provides a tracking precision of plusmn5 mm on relatively flat ground. As such, it will keep the attached mine sensors close to the ground, improving effectiveness

Dynamic legged locomotion entails navigating terrain at high speed. The impact shocks from rapid footfalls, pivotal for such mobility, introduce large impulses that saturate motion measurement. A biomimetic approach is presented in which visual information, in the form of optical flow, complements information from inertial sensors. The motion is then determined using a two-phase Hybrid Extended Kalman Filter. Experimentation in determining attitudes on a robotic leg platform shows a reduction in drift over inertial approaches and in delay over visual approaches. In tests with 6g impulses, pose was recovered within 5 deg rms with angular rate errors limited to 10 deg/sec at frequencies up to 250 Hz.

Legged robots offer exceptional mobility in uncharted terrains. Their dynamic nature yields unrivaled mobility, but serves to destabilize the motion estimation process that underlies legged operations. In particular, the discontinuous foot fall patterns and flight phases result in severe impulses, which, in turn, result in excessive accumulation of drift by inertial sensors. Ground range measurements, amongst several others, are robust to this drift yet are limited in application due to their low-bandwidth and sensitivity to ground conditions. In considering the attitude estimation problem for this dynamic legged locomotion, this paper develops a pose calculation method based on ground range measurements. This is used in conjunction with a hybrid Extended Kalman Filter that takes advantage of the ballistic nature of the flight phases. Results indicate that this combination provides rapid, robust estimates of attitude necessary for extended dynamic legged operations. In single leg experiments, which were conducted using low-cost sensing hardware, this method had an RMS error of < 1 °, half that of a non-hybrid EKF approach.

Localization and measurement of displacement are critical aspects to the operation and control of mobile robots. The motion of desktop robots, a class of mobile robots designed for use on a table, can be considered a special case of the general localization problem since the vehicle is primarily operated over a flat surface. Using the assumption that the motion of a desktop robot is essentially planar, this paper presents a novel method that senses the motion of two points to obtain both the position and orientation of the robot in two-dimensional space. The paper details how this method can be implemented using low-cost, off-the-shelf sensor hardware components and demonstrates its application in the Desktop-Bot, a compact desktop robot. Experimental testing validated features of this planar localization such as: estimation with minimal mean error (or drift), no external sensing hardware apparatus (i.e., on-board sensing), fast-update rates (/spl sim/50 Hz) and robustness to external occlusion.

A new concept for coordinating large-scale autonomous robot exploration teams having populations in the thousands is introduced along with several illustrative scenarios. Inspired by constructs found within the human immune system, the Immunology-derived Distributed Autonomous Robotics Architecture (IDARA) was developed so that exploratory actions are refined and followed by specific and mediated responses. The foundations of the IDARA model are derived from the immune network model; in particular, interactions derived from the innate immune system are integrated to yield a stronger first-order, all-purpose search strategy. Using this architecture as a foundation, the work develops methods for kilorobotic exploration in dynamic environments. As characterized via computer simulations with robot populations of up to 1,500 robots, IDARA-based exploration proved to be a robust and compact method for largescale multirobot coordination that combines reflexive and deliberative control methods in an opportunistic fashion, mimicking the human immune system.

This paper presents a new concept called the Immunology-derived Distributed Autonomous Robotics Architecture (IDARA) for the manipulation of "kilorobots" (large multi-robot colonies) modeled on the actions of the human immune system. The paper presents the development of the IDARA algorithms for the control and coordination of kilorobots for robot exploration tasks in four mapping scenarios. As characterized via computer simulations with robot populations of up to 1,500, IDARA-based exploration proved to be an efficient, robust, and compact method for large-scale multirobot control that combines the speed of reflexive methods with the precision of deliberative control.