Robotic adaptation to unanticipated operating conditions is crucial to achieving persistence and robustness in complex real world settings. For a wide range of cutting-edge robotic systems, such as micro- and nano-scale robots, soft robots, medical robots, and bio-hybrid robots, it is infeasible to anticipate the operating environment a priori due to complexities that arise from numerous factors including imprecision in manufacturing, chemo-mechanical forces, and poorly understood contact mechanics. Drawing inspiration from data-driven modeling, geometric mechanics (or gauge theory), and adaptive control, we employ an adaptive system identification framework and demonstrate its efficacy in enhancing the performance of principally kinematic locomotors (those governed by Rayleigh dissipation or zero momentum conservation). We showcase the capability of the adaptive model to efficiently accommodate varying terrains and iteratively modified behaviors within a behavior optimization framework. This provides both the ability to improve fundamental behaviors and perform motion tracking to precision. Notably, we are capable of optimizing the gaits of the Purcell swimmer using approximately 10 cycles per link, which for the nine-link Purcell swimmer provides a factor of ten improvement in optimization speed over the state of the art. Beyond simply a computational speed up, this tenfold improvement may enable this method to be successfully deployed for in-situ behavior refinement, injury recovery, and terrain adaptation, particularly in domains where simulations provide poor guides for the real world.

It is challenging to perform system identification on soft robots due to their underactuated, high-dimensional dynamics. In this work, we present a data-driven modeling framework, based on geometric mechanics (also known as gauge theory) that can be applied to systems with low-bandwidth control of the system’s internal configuration. This method constructs a series of connected models comprising actuator and locomotor dynamics based on data points from stochastically perturbed, repeated behaviors. By deriving these connected models from general formulations of dissipative Lagrangian systems with symmetry, we offer a method that can be applied broadly to robots with first-order, low-pass actuator dynamics, including swelling-driven actuators used in hydrogel crawlers. These models accurately capture the dynamics of the system shape and body movements of a simplified swimming robot model. We further apply our approach to a stimulus-responsive hydrogel simulator that captures the complexity of chemomechanical interactions that drive shape changes in biomedically relevant micromachines. Finally, we propose an approach of numerically optimizing control signals by iteratively refining models, which is applied to optimize the input waveform for the hydrogel crawler. This transfer to realistic environments provides promise for applications in locomotor design and biomedical engineering.

The gaits of locomoting systems are typically designed to maximize some sort of efficiency, such as cost of transport or speed. Equally important is the ability to modulate such a gait to effect turning maneuvers. For drag-dominated systems, geometric mechanics provides an elegant and practical framework for both ends—gait design and gait modulation. Within this framework, “constraint curvature” maps can be used to approximate the net displacement of robotic systems over cyclic gaits. Gait optimization is made possible under a previously reported “soap-bubble” algorithm. In this work, we propose both local and global gait morphing algorithms to modify a nominal gait to provide single-parameter steering control. Using a simplified swimmer, we numerically compare the two approaches and show that for modest turns, the local approach, while suboptimal, nevertheless proves effective for steering control. A potential advantage of the local approach is that it can be readily applied to soft robots or other systems where local approximations to the constraint curvature can be garnered from data, but for which obtaining an exact global model is infeasible.

Needle insertions serve a critical role in a wide variety of medical interventions. Steerable needles provide a means by which to enhance existing percutaneous procedures and afford the development of entirely new ones. Here, we present a new time-varying model for the torsional dynamics of a steerable needle, along with a new controller that takes advantage of the model. The torsional model incorporates time-varying mode shapes to capture the changing boundary conditions caused during insertion of the needle into the tissue. Extensive simulations demonstrate the improvement over a model that neglects torsional dynamics, and illustrates the possible effect of torsional model order on efficacy. Pilot feedback control experiments, conducted in artificial tissue (plastisol) under stereo image guidance, validate the overall approach: our results substantially out-perform previously reported experimental results on controlling tip-steerable needles.

Insects rely on sensory cues-tactile, hygrometric, thermal, olfactory-gathered with a pair of head-mounted antennae to perform a wide variety of sensory guided tasks. Many questions regarding the potential impact of specific mechanical design features on antennal performance can be directly and thoroughly assessed using an artificial robotic model of an antenna. Here we describe a highly tunable, modular tactile robotic model antenna and experimentally test its tactile sensing performance using a custom testbed. Exploratory experiments demonstrate the importance mechanical “tuning” on tactile navigation performance. With this model robotic antenna, numerous mechanosensory manipulations are possible, providing a new experimental platform for future testing of specific biological hypotheses.

A flexible needle can be accurately steered by robotically controlling the orientation of the bevel tip as the needle is inserted into tissue. Here, we demonstrate the significant effect of friction between the long, flexible needle shaft and the tissue, which can cause a significant discrepancy between the orientation of the needle tip and the orientation of the base where the needle is controlled. Our experiments show that several common phantom tissues used in needle steering experiments impart substantial frictional forces to the needle shaft, resulting in a lag of over 45° for a 10 cm insertion depth in some phantoms; clinical studies have reported torques large enough to could cause similar errors during needle insertions. Such angle discrepancies will result in poor performance or failure of path planners and image-guided controllers, since the needles used in percutaneous procedures are too small for state-of-the-art imaging to accurately measure the tip angle. To compensate for the angle discrepancy, we develop a model for the rotational dynamics of a needle being continuously inserted into tissue and show how a PD controller is sufficient to compensate for the rotational dynamics.

Image guidance promises to improve targeting accuracy and broaden the scope of medical procedures performed with needles. This paper takes a step toward automating the guidance of a flexible tip-steerable needle as it is inserted into human tissue. We build upon a previously proposed nonholonomic model of needles that derive steering from asymmetric bevel forces at the tip. The bevel-tip needle is inserted and rotated at its base in order to steer it in six degrees of freedom. As a first step for control, we show that the needle tip can be automatically guided to a planar slice of tissue as it is inserted. Our approach keeps the physician in the loop to control insertion speed. The distance of the needle tip position from the plane of interest is used to drive an observer-based feedback controller which we prove is locally asymptotically stable. Numerical simulations demonstrate a large domain of attraction and robustness of the controller in the face of parametric uncertainty and measurement noise. Physical experiments with tip-steerable Nitinol needles inserted into a transparent plastisol tissue phantom under stereo image guidance validate the effectiveness of our approach.

This paper presents a new class of thin, dexterous continuum robots, which we call active cannulas due to their potential medical applications. An active cannula is composed of telescoping, concentric, precurved superelastic tubes that can be axially translated and rotated at the base relative to one another. Active cannulas derive bending not from tendon wires or other external mechanisms but from elastic tube interaction in the backbone itself, permitting high dexterity and small size, and dexterity improves with miniaturization. They are designed to traverse narrow and winding environments without relying on ldquoguidingrdquo environmental reaction forces. These features seem ideal for a variety of applications where a very thin robot with tentacle-like dexterity is needed. In this paper, we apply beam mechanics to obtain a kinematic model of active cannula shape and describe design tools that result from the modeling process. After deriving general equations, we apply them to a simple three-link active cannula. Experimental results illustrate the importance of including torsional effects and the ability of our model to predict energy bifurcation and active cannula shape.

Active cannulas are remotely actuated thin continuum robots with the potential to traverse narrow and winding environments without relying on "guiding" environmental reaction forces. These features seem ideal for procedures requiring passage through narrow openings to access air-filled cavities (e.g. surgery in the throat and lung). Composed of telescoping concentric pre-curved elastic tubes, an active cannula is actuated at its base by translation and axial rotation of component tubes. Using minimum energy principles and Lie Group theory, we present a framework for the kinematics of multi-link active cannulas. This framework permits testing of the hypothesis that overall cannula shape locally minimizes stored elastic energy. We evaluate in particular whether the torsional energy in the long, straight transmission between actuators and the curved sections is important. Including torsion in the kinematic model enables us to analytically predict experimentally observed bifurcation in the energy landscape. Independent calibration procedures based on bifurcation and tip and feature positions enable model parameter identification, producing results near ranges expected from tube material properties and geometry. Experimental results validate the kinematic framework and demonstrate the importance of modeling torsional effects in order to describe bifurcation and accurately predict active cannula shape.

The discrimination of the direction of movement of sensory images is critical to the control of many animal behaviors. We propose a parsimonious model of motion processing that generates direction selective responses using short-term synaptic depression and can reproduce salient features of direction selectivity found in a population of neurons in the midbrain of the weakly electric fish Eigenmannia virescens. The model achieves direction selectivity with an elementary Reichardt motion detector: information from spatially separated receptive fields converges onto a neuron via dynamically different pathways. In the model, these differences arise from convergence of information through distinct synapses that either exhibit or do not exhibit short-term synaptic depression--short-term depression produces phase-advances relative to nondepressing synapses. Short-term depression is modeled using two state-variables, a fast process with a time constant on the order of tens to hundreds of milliseconds, and a slow process with a time constant on the order of seconds to tens of seconds. These processes correspond to naturally occurring time constants observed at synapses that exhibit short-term depression. Inclusion of the fast process is sufficient for the generation of temporal disparities that are necessary for direction selectivity in the elementary Reichardt circuit. The addition of the slow process can enhance direction selectivity over time for stimuli that are sustained for periods of seconds or more. Transient (i.e., short-duration) stimuli do not evoke the slow process and therefore do not elicit enhanced direction selectivity. The addition of a sustained global, synchronous oscillation in the gamma frequency range can, however, drive the slow process and enhance direction selectivity to transient stimuli. This enhancement effect does not, however, occur for all combinations of model parameters. The ratio of depressing and nondepressing synapses determines the effects of the addition of the global synchronous oscillation on direction selectivity. These ingredients, short-term depression, spatial convergence, and gamma-band oscillations, are ubiquitous in sensory systems and may be used in Reichardt-style circuits for the generation and enhancement of a variety of biologically relevant spatiotemporal computations.

The interplay between robotics and neuromechanics facilitates discoveries in both fields: nature provides roboticists with design ideas, while robotics research elucidates critical features that confer performance advantages to biological systems. Here, we explore a system particularly well suited to exploit the synergies between biology and robotics: high-speed antenna-based wall following of the American cockroach (Periplaneta americana). Our approach integrates mathematical and hardware modeling with behavioral and neurophysiological experiments. Specifically, we corroborate a prediction from a previously reported wall-following template - the simplest model that captures a behavior - that a cockroach antenna-based controller requires the rate of approach to a wall in addition to distance, e.g., in the form of a proportional-derivative (PD) controller. Neurophysiological experiments reveal that important features of the wall-following controller emerge at the earliest stages of sensory processing, namely in the antennal nerve. Furthermore, we embed the template in a robotic platform outfitted with a bio-inspired antenna. Using this system, we successfully test specific PD gains (up to a scale) fitted to the cockroach behavioral data in a "real-world" setting, lending further credence to the surprisingly simple notion that a cockroach might implement a PD controller for wall following. Finally, we embed the template in a simulated lateral-leg-spring (LLS) model using the center of pressure as the control input. Importantly, the same PD gains fitted to cockroach behavior also stabilize wall following for the LLS model.

Physicians perform percutaneous therapies in many diagnostic and therapeutic procedures. Image guidance promises to improve targeting accuracy and broaden the scope of needle interventions. In this paper, we consider the possibility of automating the guidance of a flexible bevel-tip needle as it is inserted into human tissue. We build upon a previously proposed nonholonomic kinematic model to develop a nonlinear observer-based controller. As a first step for control, we show that flexible needles can be automatically controlled to remain within a planar slice of tissue as they are inserted by a physician; our approach keeps the physician in the loop to control insertion speed. In the proposed controller, the distance of the needle tip position from the plane of interest is used as a feedback signal. Numerical simulations demonstrate the stability and robustness of the controller in the face of parametric uncertainty. We also present results from pilot physical experiments with phantom tissue under stereo image guidance.

Lie group symmetry in a mechanical system can lead to a dimensional reduction in its dynamical equations. Typically, the symmetries that one exploits are intrinsic to the mechanical system at hand, e.g. invariance of the system's Lagrangian to some group of motions. In the present work we consider symmetries that arise from an extrinsic control task, rather than the intrinsic structure of configuration space, constraints, or system dynamics. We illustrate this technique with several examples. In the examples, the reduction enables us to design essentially global feedback controllers on the reduced systems.We apply task-induced symmetry and reduction to a recently developed 6 DOF kinematic model of steerable bevel-tip needles. The resulting controllers cause the needle tip to track a subspace of its configuration space. We envision that the methodology presented in this paper will form the basis for a new planning and control framework for needle steering.

Traditionally, visual servoing is separated into tracking and control subsystems. This separation, though convenient, is not necessarily well justified. When tracking and control strategies are designed independently, it is not clear how to optimize them to achieve a certain task. In this work, we propose a framework in which spatial sampling kernels - borrowed from the tracking and registration literature - are used to design feedback controllers for visual servoing. The use of spatial sampling kernels provides natural hooks for Lyapunov theory, thus unifying tracking and control and providing a framework for optimizing a particular servoing task. As a first step, we develop kernel-based visual servos for a subset of relative motions between camera and target scene. The subset of motions we consider are 2D translation, scale, and roll of the target relative to the camera. Our approach provides formal guarantees on the convergence/stability of visual servoing algorithms under putatively generic conditions.

It may come as no surprise that a simple "planar unicycle" (or "skate") model can successfully follow a wall at high speed under PD-control. What would be surprising is that such a simple control mechanism may underly the control of one of nature's fastest terrestrial insects, the American cockroach. For this paper, we implemented the same controller (up to scale) believed to govern cockroach wall following to successfully control two models of the cockroach: a differential-drive mobile robot with a flexible artificial antenna and a lateral leg spring model with moving center of pressure as the control input. These physical and numerical experiments demonstrate the sufficiency of the cockroach's putative controller in a real-world setting with unmodeled effects, and suggest how the nervous system might guide leg placement in response to sensory stimuli. This hierarchy of models may prove useful in generating prescriptive hypotheses for biological testing and hence elucidating the general principles that underly sensor-guided animal locomotion.

We have developed a new class of continuously flexible snake-like robots, called active cannulas, that consist of several telescoping pre-curved superelastic tubes. The devices derive bending actuation not from tendon wires or other external mechanisms, but from elastic energy stored in the backbone itself. This allows active cannulas to have a small diameter and a high degree of dexterity, which should enable them to navigate through complex anatomy to sites inaccessible by current surgical robotic devices. Active cannulas may also enhance patient safety because their inherent compliance mitigates potential trauma from inadvertent tool-tissue collision. A consequence of our design is that dexterity improves with miniaturization. A kinematic description of active cannula shape requires a model of the elastic interaction of telescoping pre-curved flexible tubes, and we derive a two-"link" beam mechanics-based model. Experiments using curved nitinol tubes and wires validate the model.

Feedback from antennae - long, flexible tactile sensors - enables cockroaches and other arthropods to rapidly maneuver through poorly lit and cluttered environments. Inspired by their performance, we created a wall-following controller for a dynamic wheeled robot using tactile antenna feedback. We show this controller is stable for a wide range of control gains and robot system parameters. To test the controller, we constructed a two-link antenna that uses potentiometers and capacitive contact sensors. Experiments based on the prototype demonstrate that our controller robustly tracks unexpected corners ranging from -60° to +900.

Fine needles facilitate diagnosis and therapy because they enable minimally invasive surgical interventions. This paper formulates the problem of steering a very flexible needle through firm tissue as a nonholonomic kinematics problem, and demonstrates how planning can be accomplished using diffusion-based motion planning on the Euclidean group, SE(3). In the present formulation, the tissue is treated as isotropic and no obstacles are present. The bevel tip of the needle is treated as a nonholonomic constraint that can be viewed as a 3D extension of the standard kinematic cart or unicycle. A deterministic model is used as the starting point, and reachability criteria are established. A stochastic differential equation and its corresponding Fokker-Planck equation are derived. The Euler-Maruyama method is used to generate the ensemble of reachable states of the needle tip. Inverse kinematics methods developed previously for hyper-redundant and binary manipulators that use this probability density information are applied to generate needle tip paths that reach the desired targets.

This paper presents a global diffeomorphism from a visible set of rigid-body configurations, a subset of SE(3), to an image space. Using the diffeomorphism, we develop an image-based, essentially global, dynamic visual servoing algorithm that keeps features in the field of view and avoids self-occlusions. The approach is geometric in the sense that the visible set and its corresponding image are differentiable manifolds, and the diffeomorphism is global. The mapping to image space and the resulting Jacobian rely on a specific target geometry, a sphere with a known radius marked with an "arrow" feature point. The paper presents simulation experiments for a more typical visual target comprised of a collection of isolated feature points. In this setting, the diffeomorphism to image space is approximate, nevertheless, the simulations converge for a wide variety of target geometries and initial conditions.

We explore the benefits of multiple views for visual servoing (VS) by commanding a single camera to first "peer" at a scene from two vantage points, thus acquiring a set "reference" images, prior to executing a visual position task. Our approach completely decouples the translational and rotational components of our controller: epipoles from the reference views drive the translational error to zero, while the rotational degrees of freedom maintain all of the features in the field of view (FOV). We furnish a simple Lyapunov stability proof that demonstrates a large domain of attraction while maintaining all features in the FOV. Finally, we present simulated experiments that suggest robustness to measurement noise and large variations in the baseline between the reference views.

We present a purely rotational visual servoing algorithm, which aligns the orientation between two cameras at different locations in space. Specifically, our kinematic controller steers a set of so-called bi-tangent lines to intersect at the epipole using a purely image-based bi-tangent line Jacobian. Bi-tangent lines, i.e. lines joining corresponding features on the superposition of two views of a scene, can be defined for both points and contours, so we apply our controller to both feature types. Simulated experiments demonstrate the parametric robustness of the proposed method.

We consider the problem of having a group of nonholonomic mobile robots equipped with omnidirectional cameras maintain a desired leader-follower formation. Our approach is to translate the formation control problem from the configuration space into a separate visual servoing task for each follower. We derive the questions of motion of the leader in the image plane of the follower and propose two control schemes for the follower. The first one is based on feedback linearization and is either string stable or leader-to-formation stable, depending on the sensing capabilities of the followers. The second one assumes a kinematic model for the evolution of the leader velocities and combines a Luenberger observer with a linear control law that is locally stable. We present simulation results evaluating our vision-based follow-the-leader control strategies.

This paper presents a framework for visual servoing that guarantees convergence to a visible goal from almost every initially visible configurations while maintaining full view of all the feature points along the way. The method applies to first- and second-order fully actuated plant models. The solution entails three components: a model for the "occlusion-free" configurations; a change of coordinates from image to model coordinates; and a navigation function for the model space. We present three example applications of the framework, along with experimental validation of its practical efficacy.

We describe a hybrid planar image-based servo algorithm which, for a simplified planar convex rigid body, converges to a static goal for all initial conditions within the workspace of the camera. This is achieved by using the sequential composition of a palette of continuous image based controllers. Each sub-controller, based on a specified set of collinear feature points, is shown to converge for all initial configurations in which the feature points are visible. Furthermore, the controller guarantees that the body will maintain a "visible" orientation, i.e. the feature points will always be in view of the camera. This is achieved by introducing a change of coordinates from SE(2) to an image plane measurement of three points, and imposing a navigation function in that coordinate system. Our intuition suggests that appropriately generalized versions of these ideas may be extended to SE(3).

We describe work-in-progress toward a nonlinear image-based rigid body dynamic triangulator which we believe tracks a moving target from "essentially all" initial conditions (all initial conditions except a set of measure zero). The dynamic triangulator depends on the goal state only through its image plane position and velocity and requires a navigation function, imposed directly upon image features, to serve as a regressor for a gradient-like state update law.