In this paper, we introduce a method for visual relocalization using the geometric information from a 3D surfel map. A visual database is first built by global indices from the 3D surfel map rendering, which provides associations between image points and 3D surfels. Surfel reprojection constraints are utilized to optimize the keyframe poses and map points in the visual database. A hierarchical camera relocalization algorithm then utilizes the visual database to estimate 6-DoF camera poses. Learned descriptors are further used to improve the performance in challenging cases. We present evaluation under real-world conditions and simulation to show the effectiveness and efficiency of our method, and make the final camera poses consistently well aligned with the 3D environment.

This paper presents a novel nonlinear disturbance rejection approach for high precision model-based control of hydraulic robots. While most disturbance rejection approaches make use of observers, we propose a novel adaptive Unscented Kalman Filter to estimate the disturbances in an unbiased minimum-variance sense. The filter is made adaptive such that there is no need to tune the covariance matrix for the disturbance estimation. Furthermore, whereas most model-based control approaches require the linearization of the system dynamics, our method is nonlinear which means that no linearization is required. Through extensive simulations as well as real hardware experiments, we demonstrate that our proposed approach can achieve high precision tracking and can be readily applied to most robotic systems even in the presence of uncertainties and external disturbances. The proposed approach is also compared to existing approaches which demonstrates its superior tracking performance.

We present a fully-integrated sensing and control system which enables mobile manipulator robots to execute building tasks with millimeter-scale accuracy on building construction sites. The approach leverages multi-modal sensing capabilities for state estimation, tight integration with digital building models, and integrated trajectory planning and whole-body motion control. A novel method for high-accuracy localization updates relative to the known building structure is proposed. The approach is implemented on a real platform and tested under realistic construction conditions. We show that the system can achieve sub-cm end-effector positioning accuracy during fully autonomous operation using solely onboard sensing.

In this paper we present a system for the in situ33In the context of building construction, “in situ” means that fabrication takes place at the structure's final location directly on the building site. fabrication of a full-scale, load-bearing, and doubly-curved steel reinforced concrete wall. Two complementary vision-based sensing systems provide the feedback necessary to build a 12 meter long steel wire mesh as part of a novel digital building process. The sensing systems provide estimates of the robot pose, referenced to the CAD model of the building site, as well as feedback on the accuracy of the built structure over the course of construction. This second piece of information is used to adapt the building plan to compensate for system inaccuracies and material deformations which occur during buildup. In this way, the structure was successfully built with 98% of the total geometry within 2 centimeters of the designed position. To the best of our knowledge, this is the largest structure which has been built by a mobile robot using solely vision-based sensing.

This work addresses the problem of kinematic trajectory planning for mobile manipulators with non-holonomic constraints, and holonomic operational-space tracking constraints. We obtain whole-body trajectories and time-varying kinematic feedback controllers by solving a Constrained Sequential Linear Quadratic Optimal Control problem. The employed algorithm features high efficiency through a continuous-time formulation that benefits from adaptive step-size integrators and through linear complexity in the number of integration steps. In a first application example, we solve kinematic trajectory planning problems for a 26 DoF wheeled robot. In a second example, we apply Constrained SLQ to a real-world mobile manipulator in a receding-horizon optimal control fashion, where we obtain optimal controllers and plans at rates up to 100 Hz.

In this work we present a co-located task space sensing and control system designed to control the end-effector motion of a mobile manipulator in the presence of dynamic and unknown base motion. We present a method for generating end-effector motion estimates at 1 kilohertz for use in real time control through visual-inertial sensor fusion. We show that a Moving Horizon Estimator outperforms Kalman filter-based methods in generating accurate predictive estimates for use in real time. We use this estimator to close a task space control loop directly at the end-effector, assuming no prior knowledge of the base pose and motion. We demonstrate the performance of this system on a hydraulically actuated arm which performs task-space tracking tasks in the presence of significant unknown base motion.

Despite the prevalent use of robotic technologies in industrial manufacturing, their use on building construction sites is still very limited. This is mainly due to the unstructured nature of construction sites and the fact that the structures built must be larger than the machines which build them. This paper addresses these difficulties by presenting a repositioning and localization system which, using purely on-board sensing, allows a mobile robot to maintain a high degree of end-effector positioning accuracy while moving among numerous building positions during a building task. We introduce a newly developed machine, called the In situ Fabricator (IF), whose goal is to bring digital fabrication to the construction site. The capabilities of the repositioning and localization system is demonstrated through the construction of a vertical stack of bricks, in which the IF repositions itself after placing each brick. With this experiment, we demonstrate that the IF can build with sub-centimeter accuracy over long building sequences.