Important challenges in retinal microsurgery in-clude prolonged operating time, inadequate force feedback, and poor depth perception due to a constrained top-down view of the surgery. The introduction of robot-assisted technology could potentially deal with such challenges and improve the surgeon's performance. Motivated by such challenges, this work develops a strategy for autonomous needle navigation in retinal microsurgery aiming to achieve precise manipulation, reduced end-to-end surgery time, and enhanced safety. This is accomplished through real-time geometry estimation and chance-constrained Model Predictive Control (MPC) resulting in high positional accuracy while keeping scleral forces within a safe level. The robotic system is validated using both open-sky and intact (with lens and partial vitreous removal) ex vivo porcine eyes. The experimental results demonstrate that the generation of safe control trajectories is robust to small motions associated with head drift. The mean navigation time and scleral force for MPC navigation experiments are 7.208 s and 11.97 mN, which can be considered efficient and well within acceptable safe limits. The resulting mean errors along lateral directions of the retina are below 0.06 mm, which is below the typical hand tremor amplitude in retinal microsurgery.

This paper introduces a novel, small form-factor, aerial vehicle research platform for agile object detection, classification, tracking, and interaction tasks. General-purpose hardware components were designed to augment a given aerial vehicle and enable it to perform safe and reliable grasping. These components include a custom collision tolerant cage and low-cost Gripper Extension Package, which we call GREP, for object grasping. Small vehicles enable applications in highly constrained environments, but are often limited by computational resources. This work evaluates the challenges of pick-and-place tasks, with entirely onboard computation of object pose and visual odometry based state estimation on a small platform, and demonstrates experiments with enough accuracy to reliably grasp objects. In a total of 70 trials across challenging cases such as cluttered environments, obstructed targets, and multiple instances of the same target, we demonstrated successfully grasping the target in 93 % of trials. Both the hardware component designs and software framework are released as open-source, since our intention is to enable easy reproduction and application on a wide range of small vehicles.

This paper offers a multi-layer planning approach for autonomous surface vessels (ASVs) that must adhere to good seamanship practices and the International Regulations for Prevention of Collisions at Sea (COLREGS) [1]. The approach combines novel situational awareness logic with motion primitive-based planners in a receding horizon framework. Further, ship domain and ship arena concepts are used to develop risk metrics that capture COLREGS compliance and the notion of good seamanship. By relying on metrics-driven motion planning as opposed to rule-based conditions, the proposed framework scales naturally to non-trivial single-vessel and multi-vessel situations. The planner is evaluated using adaptive, simulation-based testing to statistically compare the performance to other standard methods. Finally, proof-of-concept field experiments are presented on a subscale platform.

Learning robust policies for robotic systems operating in presence of uncertainty is a challenging task. For safe navigation, in addition to the natural stochasticity of the environment and vehicle dynamics, the perception uncertainty associated with dynamic entities, e.g. pedestrians, must be accounted for during motion planning. To this end, we construct an algorithm with built-in robustness to uncertainty by directly minimizing an upper confidence bound on the expected cost of trajectories instead of employing a standard approach based on minimizing the expected cost itself. Perception uncertainty is incorporated into the policy search framework by predicting each pedestrian’s intent belief and propagating their state distribution in time using closed-loop goal-directed dynamics. We train the policy in simulation and show that it could be transferred to an agile ground vehicle for successful autonomous robot navigation in presence of pedestrians with perception uncertainty. We further show the superior performance of this policy over a policy that does not consider pedestrian intent and perception uncertainty.

This paper addresses the problem of identifying whether/how a black-box autonomous system has regressed in performance when compared to previous versions. The approach analyzes performance datasets (typically gathered through simulation-based testing) and automatically extracts test parameter clusters of predicted performance regression. First, surrogate modeling with quantile random forests is used to predict regions of performance regression with high confidence. The predicted regression landscape is then clustered in both the output space and input space to produce groupings of test conditions ranked by performance regression severity. This approach is analyzed using randomized test functions as well as through a case study to detect performance regression in autonomous surface vessel software.

A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tooltip distance to the retina and providing haptic or auditory feedback. However, automating the tool-navigation task itself remains unsolved and largely unexplored. Such a capability, if reliably automated, could serve as a building block to streamline complex procedures and reduce the chance for tissue damage. Towards this end, we propose to automate the tool-navigation task by learning to mimic expert demonstrations of the task. Specifically, a deep network is trained to imitate expert trajectories toward various locations on the retina based on recorded visual servoing to a given goal specified by the user. The proposed autonomous navigation system is evaluated in simulation and in physical experiments using a silicone eye phantom. We show that the network can reliably navigate a needle surgical tool to various desired locations within 137 μm accuracy in physical experiments and 94 μm in simulation on average, and generalizes well to unseen situations such as in the presence of auxiliary surgical tools, variable eye backgrounds, and brightness conditions.

The current state-of-the-art for testing and evaluation of autonomous surface vehicle (ASV) decision-making is currently limited to one-versus-one vessel interactions by determining compliance with the International Regulations for Prevention of Collisions at Sea, referred to as COLREGS. Strict measurement of COLREGS compliance, however, loses value in multi-vessel encounters, as there can be conflicting rules which make determining compliance extremely subjective. This work proposes several performance metrics to evaluate ASV decision-making based on the concept of "good seamanship," a practice which generalizes to multi-vessel encounters. Methodology for quantifying good seamanship is presented based on the criteria of reducing the overall collision risk of the situation and taking early, appropriate actions. Case study simulation results are presented to showcase the seamanship performance criteria against different ASV planning strategies.

One of the significant challenges of moving from manual to robot-assisted retinal surgery is the loss of perception of forces applied to the sclera (sclera forces) by the surgical tools. This damping of force feedback is primarily due to the stiffness and inertia of the robot. The diminished perception of tool-to-eye interactions might put the eye tissue at high risk of injury due to excessive sclera forces or extreme insertion of the tool into the eye. In the present study therefore a 1-dimensional adaptive control method is customized for 3-dimensional control of sclera force components and tool insertion depth and then implemented on the velocity-controlled Johns Hopkins Steady-Hand Eye Robot. The control method enables the robot to perform autonomous motions to make the sclera force and/or insertion depth of the tool tip to follow pre-defined desired and safe trajectories when they exceed safe bounds. A robotic light pipe holding application in retinal surgery is also investigated using the adaptive control method. The implementation results indicate that the adaptive control is able to achieve the imposed safety margins and prevent sclera forces and insertion depth from exceeding safe boundaries.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. During surgery the tool needs to be inserted into the eyeball while maintaining constant contact with the sclera. Any unexpected manipulation could cause extreme tool-sclera contact force (scleral force) thus damage the sclera. The introduction of robotic assistance could enhance and expand the surgeon's manipulation capabilities during surgery. However, the potential intra-operative danger from surgeon's misoperations remains difficult to detect and prevent by existing robotic systems. Therefore, we propose a method to predict imminent unsafe manipulation in robot-assisted retinal surgery and generate feedback to the surgeon via auditory substitution. The surgeon could then react to the possible unsafe events in advance. This work specifically focuses on minimizing sclera damage using a force-sensing tool calibrated to measure small scleral forces. A recurrent neural network is designed and trained to predict the force safety status up to 500 milliseconds in the future. The system is implemented using an existing "steady hand" eye robot. A vessel following manipulation task is designed and performed on a dry eye phantom to emulate the retinal surgery and to analyze the proposed method. Finally, preliminary validation experiments are performed by five users, the results of which indicate that the proposed early warning system could help to reduce the number of unsafe manipulation events.

This work develops a technique for using robot motion trajectories to create a high quality stochastic dynamics model that is then leveraged in simulation to train control policies with associated performance guarantees. We demonstrate the idea by collecting dynamics data from a 1/5 scale agile ground vehicle, fitting a stochastic dynamics model, and training a policy in simulation to drive around an oval track at up to 6.5 m/s while avoiding obstacles. We show that the control policy can be transferred back to the real vehicle with little loss in predicted performance. We compare this to an approach that uses a simple analytic car model to train a policy in simulation and show that using a model with stochasticity learned from data leads to higher performance in terms of trajectory tracking accuracy and collision probability. Furthermore, we show empirically that simulation-derived performance guarantees transfer to the actual vehicle when executing a policy optimized using a deep stochastic dynamics model fit to vehicle data.

In this paper, we focus on adaptive sampling on a Gaussian Processes (GP) using the receding-horizon Cross-Entropy (CE) trajectory optimization. Specifically, we employ the GP upper confidence bound (GP-UCB) as the optimization criteria to adaptively plan sampling paths that balance the exploitation-exploration trade-off. Path planning at the initial stage focuses on exploring and learning a model of the environment, and later, on exploiting the learned model to focus sampling around regions that exhibit extreme sensory measurements and much higher spatial variability, denoted as the Region of Interest (ROI). The integration of the CE trajectory optimization allows the sampling density to be dynamically adjusted based on the latest sensory measurements, thus providing an efficient sampling strategy for sensing and localizing the ROI. We demonstrate the effectiveness of the proposed method in exploring simulated scalar fields with single or multiple ROIs. Field experiments with an Unmanned Surface Vehicle (USV) in a coastal bathymetry mapping mission validate the approach's capability in quickly exploring and mapping the given area, and then focusing and increasing the sampling density around the deepest region, as a surrogate for e.g. the extremal concentration of a pollutant in the environment.

This work addresses the problem of motion planning among obstacles for quadrotor platforms under external disturbances and with model uncertainty. A novel Nonlinear Model Predictive Control (NMPC) optimization technique is proposed which incorporates specified uncertainties into the planned trajectories. At the core of the procedure lies the propagation of model parameter uncertainty and initial state uncertainty as high-confidence ellipsoids in pose space. The quadrotor trajectories are then computed to avoid obstacles by a required safety margin, expressed as ellipsoid penetration while minimizing control effort and achieving a user-specified goal location. Combining this technique with online model identification results in robust obstacle avoidance behavior. Experiments in outdoor scenarios with virtual obstacles show that the quadrotor can avoid obstacles robustly, even under the influence of external disturbances.

This work studies the design of reliable control laws of robotic systems operating in uncertain environments. We introduce a new approach to stochastic policy optimization based on probably approximately correct (PAC) bounds on the expected performance of control policies. An algorithm is constructed which directly minimizes an upper confidence bound on the expected cost of trajectories instead of employing a standard approach based on the expected cost itself. This algorithm thus has built-in robustness to uncertainty, since the bound can be regarded as a certificate for guaranteed future performance. The approach is evaluated on two challenging robot control scenarios in simulation: a car with side slip and a quadrotor navigating through obstacle-ridden environments. We show that the bound accurately predicts future performance and results in improved robustness measured by lower average cost and lower probability of collision. The performance of the technique is studied empirically and compared to several existing policy search algorithms.

Model-based control of dynamical systems typically requires accurate domain-specific knowledge and specifications of possibly proprietary system components. In the context of autonomous driving, steering actuator dynamics can be difficult to model due to an integrated proprietary power steering control module. While first-principles models derived from physics laws can often approximate the system behavior, it remains generally difficult to capture non-physically derived behavior based on proprietary software algorithms in the power steering system. To overcome this limitation, this work instead employs a recurring neural network to model the steering dynamics of an autonomous vehicle. The resulting model is then integrated into a Nonlinear Model Predictive Control scheme to generate feedforward steering commands for embedded control. The proposed approach is compared to traditional first-principles steering modeling through on-vehicle experiments and statistical data validation. As a result, it is shown that the neural network model can be automatically generated with less domain-specific knowledge, can predict steering dynamics more accurately, and perform comparably to a high-fidelity first principles model when used for controlling the steering system of a self-driving vehicle.

Task and motion planning subject to Linear Temporal Logic (LTL) specifications in complex, dynamic environments requires efficient exploration of many possible future worlds. Model-free reinforcement learning has proven successful in a number of challenging tasks, but shows poor performance on tasks that require long-term planning. In this work, we integrate Monte Carlo Tree Search with hierarchical neural net policies trained on expressive LTL specifications. We use reinforcement learning to find deep neural networks representing both low-level control policies and task-level “option policies” that achieve high-level goals. Our combined architecture generates safe and responsive motion plans that respect the LTL constraints. We demonstrate our approach in a simulated autonomous driving setting, where a vehicle must drive down a road in traffic, avoid collisions, and navigate an intersection, all while obeying rules of the road.

Surgeons often manually palpate tissue or organs in order to find tumors or other anatomical structures. Information about organ geometry and tissue stiffness gained from palpation can also be extremely useful in robotic surgery for diagnosis, surgical guidance, and registration to other preoperative information. However, it is not always easy to obtain, even if the robot is equipped with force sensors. This paper reports our approach for concurrent estimation of stiffness and surface geometry, using a continuous motion similar to a sweeping palpation motion used by surgeons. Our method relies on force data captured by a tactile sensor rigidly attached to an end-effector probe. We use Gaussian processes to simultaneously estimate geometry and stiffness. The method is not tied to any specific robotic platform and is consistent with a variety of palpation strategies. For simplicity, we discuss the results based on two different palpation primitives. This is our first step towards developing an adaptive high fidelity model reconstruction and path optimization technique.

We propose a learning-from-demonstration approach for grounding actions from expert data and an algorithm for using these actions to perform a task in new environments. Our approach is based on an application of sampling-based motion planning to search through the tree of discrete, high-level actions constructed from a symbolic representation of a task. Recursive sampling-based planning is used to explore the space of possible continuous-space instantiations of these actions. We demonstrate the utility of our approach with a magnetic structure assembly task, showing that the robot can intelligently select a sequence of actions in different parts of the workspace and in the presence of obstacles. This approach can better adapt to new environments by selecting the correct high-level actions for the particular environment while taking human preferences into account.

This work introduces a hybrid visual servoing technique for differentially flat, underactuated systems that is well suited for aggressive dynamics. Standard Position-Based Visual Servoing (PBVS) and Image-Based Visual Servoing (IBVS) approaches for underactuated systems, such as quadrotors, oftentimes do not explicitly ensure that the relevant image features stay in the camera's field of view, especially while the system is performing agile maneuvers. We present a control technique that is designed to mitigate this issue and that results in increased robustness. Given a goal image, we first solve a constrained Perspective-n-Point (PnP) problem to find an equilibrium pose which aligns the camera with the goal. We then formulate the task of navigating to the goal pose as an optimal control problem, where a cost over the resulting image feature tracks along the trajectory is minimized which implicitly keeps features in the field of view over the course of the trajectory. The optimization is performed over a polynomial parametrization of the flat outputs of the system to decrease the dimensionality of the optimization. Simulations and physical experiments are performed with a quadrotor system to benchmark the algorithm's performance against a typical PBVS approach.

This paper considers pick-and-place tasks using aerial vehicles equipped with manipulators. The main focus is on the development and experimental validation of a nonlinear model-predictive control methodology to exploit the multi-body system dynamics and achieve optimized performance. At the core of the approach lies a sequential Newton method for unconstrained optimal control and a high-frequency low-level controller tracking the generated optimal reference trajectories. A low cost quadrotor prototype with a simple manipulator extending more than twice the radius of the vehicle is designed and integrated with an on-board vision system for object tracking. Experimental results show the effectiveness of model-predictive control to motivate the future use of real-time optimal control in place of standard ad-hoc gain scheduling techniques.

This paper studies an optimization-based approach for solving optimal estimation and optimal control problems through a unified computational formulation. The goal is to perform trajectory estimation over extended past horizons and model-predictive control over future horizons by enforcing the same dynamics, control, and sensing constraints in both problems, and thus solving both problems with identical computational tools. Through such systematic estimation-control formulation we aim to improve the performance of autonomous systems such as agile robotic vehicles. This work focuses on sequential sweep trajectory optimization methods, and more specifically extends the method known as differential dynamic programming to the parameter-dependent setting in order to enable the solutions to general estimation and control problems.

This paper is concerned with robustness analysis of decision making under uncertainty. We consider a class of iterative stochastic policy optimization problems and analyze the resulting expected performance for each newly updated policy at each iteration. In particular, we employ concentration-of-measure inequalities to compute future expected cost and probability of constraint violation using empirical runs. A novel inequality bound is derived that accounts for the possibly unbounded change-of-measure likelihood ratio resulting from iterative policy adaptation. The bound serves as a high-confidence certificate for providing future performance or safety guarantees. The approach is illustrated with a simple robot control scenario and initial steps towards applications to challenging aerial vehicle navigation problems are presented.

This paper is concerned with optimal trajectory generation for robotic multi-body systems. The focus is on discrete optimal control methods which operate intrinsically in the state space system manifold and do not require coordinate charts or projections. This is accomplished by defining both the dynamics and the optimal control solution as sequences of vector fields mapping to curves on the Lie group through retraction maps, and defining variations and differentiation with respect to such vector fields. As a result, standard trajectory optimization methods can be easily extended to the Lie group setting without loss of efficiency. The methods are illustrated with three numerical examples: a quadrotor, an aerial vehicle with manipulators, and a simple nonholonomic system.

We consider the optimal control of mechanical systems on Lie groups and develop numerical methods that exploit the structure of the state space and preserve the system motion invariants. Our approach is based on a coordinate-free variational discretization of the dynamics that leads to structure-preserving discrete equations of motion. We construct necessary conditions for optimal trajectories that correspond to discrete geodesics of a higher order system and develop numerical methods for their computation. The resulting algorithms are simple to implement and converge to a solution in very few iterations. A general software implementation is provided and applied to two example systems: an underactuated boat and a satellite with thrusters.

This article is concerned with the animation and control of vehicles with complex dynamics such as helicopters, boats, and cars. Motivated by recent developments in discrete geometric mechanics, we develop a general framework for integrating the dynamics of holonomic and nonholonomic vehicles by preserving their state-space geometry and motion invariants. We demonstrate that the resulting integration schemes are superior to standard methods in numerical robustness and efficiency, and can be applied to many types of vehicles. In addition, we show how to use this framework in an optimal control setting to automatically compute accurate and realistic motions for arbitrary user-specified constraints.

This paper addresses the optimal control of nonholonomic systems through provably correct discretization of the system dynamics. The essence of the approach lies in the discretization of the Lagrange-d'Alembert principle which results in a set of forced discrete Euler-Lagrange equations and discrete nonholonomic constraints that serve as equality constraints for the optimization of a given cost functional. The method is used to investigate optimal trajectories of wheeled robots.

The paper presents two different methods for mobile robot tracking and following of a fast-moving person in outdoor unstructured and possibly dynamic environment. The robot is equipped with laser range-finder and omnidirectional camera. The first method is based on visual tracking only and while it works well at slow speeds and controlled conditions, its performance quickly degrades as conditions become more difficult. The second method which uses the laser and the camera in conjunction for tracking performs well in dynamic and cluttered outdoor environments as long as the target occlusions and losses are temporary. Experimental results and analysis are presented for the second approach

We present an outdoor terrain planner that finds near optimal trajectories under dynamic and kinematic constraints. The planner can find solutions in close to real time by relaxing some of the assumptions associated with costly rigid body simulation and complex terrain surface interactions. Our system is based on control-driven Proba bilistic Roadmaps and can efficiently find and optimize a near time-minimum trajectory. We present simulated results with artificial environments, as well as a real robot experiment using Segway Robotic Mobile Platform.