Aerial manipulation describes a process that includes physical interaction between an unmanned aircraft system (UAS) and its environment. We aim to apply aerial manipulation to sample leaves and small branches from rain forest trees. Current approaches to aerial manipulation involve extended periods of UAS-environment interaction, during which forces and moments can lead to a loss in attitude or position control in underactuated multicopters. By adapting intelligent foot placement strategies found in dynamically stable hopping robots, this work proposes a strategy involving carefully managed intentional collisions between the UAS and its environment. We designed an attitude controller denoted a Velocity Matching controller that aligns a UAS-mounted pogo-stick foot with the center of mass velocity vector during collision approach to maximize UAS ability to recover a hover state after collision. We propose the use of a flight envelope involving altitude and horizontal speed states to assess recoverability prior to initiating each approach to collision. We identify this flight envelope from a simulation study built on a model of flight in Conventional Waypoint Following and Velocity Matching control modes as well as a model of collision response. Experimental flight testing evaluates the simulation-based envelope resulting in an actual envelope that is somewhat smaller but similarly shaped to the envelope identified in simulation.

Soft fluidic actuators are becoming popular for their backdrivability, potential for high power density, and their support for power supply through flexible tubes. Control and design of such actuators requires serviceable models that describe how they relate fluid pressure and flow to mechanical force and motion. We present a simple 2-port model of a bellows actuator that accounts for the relationships among fluid and mechanical variables imposed by the kinematics of the deforming bellows structure and accounts for elastic energy stored in the actuator's thermoplastic material structure. Elastic energy storage due to axial deformation is captured by revolving a differential strip whose linear elastic behavior is a nonlinear function of the actuator length. The model is evaluated through experiments in which either actuator length and pressure or force and pressure are imposed. The model has an error of 9.8% of the force range explored and yields insight into the effects of geometry changes. The resulting model can be used for model-based control or actuator design across the full operating range and can be exercised under either imposed force or imposed actuator length.

When driving conditions exceed the design limits of an automation system, transfers of control back to a human driver become necessary. Such transfers may compromise safety, since a driver typically requires time to get back into the loop. Various schemes for combining human and automatic control over steering have been proposed to address the challenges of transferring control smoothly. This article presents the findings from a simulator study that compared four such schemes. The driver initiated a transfer of control either by pressing a button and assuming full control of the vehicle (switching transition) or by overpowering an automation that continued to act on the vehicle. Two additional factors in the$\text{2} \times \text{2} \times \text{2}$factorial design are the availability of haptic feedback and the inclusion of a secondary task. To examine the robustness of each scheme, all driving scenarios involved nominal and off-nominal conditions. The findings from this study demonstrate the benefits of haptic shared control (i.e., overpowering transitions with haptic feedback) for facilitating transfers of control between a human driver and vehicle automation. Haptic-feedback-enhanced drivers’ awareness of automation input and the status of the vehicle while the overpowering transitions improved the smoothness of interventions, compared to switching transitions.

Controlling soft robots with precision is a challenge due to the difficulty of constructing models that are amenable to model-based control design techniques. Koopman operator theory offers a way to construct explicit dynamical models of soft robots and to control them using established model-based control methods. This approach is data driven, yet yields an explicit control-oriented model rather than just a “black-box” input-output mapping. This work describes a Koopman-based system identification method and its application to model predictive control (MPC) design for soft robots. Three MPC controllers are developed for a pneumatic soft robot arm via the Koopman-based approach, and their performances are evaluated with respect to several real-world trajectory following tasks. In terms of average tracking error, these Koopman-based controllers are more than three times more accurate than a benchmark MPC controller based on a linear state-space model of the same system, demonstrating the utility of the Koopman approach in controlling real soft robots.

Passive transmission-based robotic devices are capable of providing motion guidance while ensuring user safety and engagement. To circumvent some of the drawbacks associated with steering continuously variable transmissions based on rolling contacts, we are exploring a class of discretely variable devices, based on brakes and hydrostatic transmissions. Previously available control methods for discretely variable devices were built on velocity fields and only developed to stabilize a one-dimensional (1-D) target manifold. For n-DOF devices, methods to stabilize target manifolds of dimensions 1 to n-1 are of interest. In this article, we contribute constraint field methods that stabilize n$-$1 dimensional target manifolds while leaving the orthogonal subspace free to the control of the operator. We also contribute force-modulated single degree of freedom (DOF) velocity fields, which add between 1 and n-2 virtual DOFs to the motion of devices whose physical constraints leave one DOF. Control performance is demonstrated in simulation for 3-DOF devices capable of imposing 1-D or 2-D constraints and in experiment for 2-DOF devices imposing 1-D constraints. Our experimental apparatus features digital hydraulic transmissions that are easily configured for an n-dimensional space and capable of imposing constraints of any dimension, thus motivating the contributed methods.

Automatic emergency steering maneuvers can be used to avoid more obstacles than emergency braking alone. While a steer-by-wire system can decouple the driver who might act as a disturbance during the emergency steering maneuver, the alternative in which the steering wheel remains coupled can enable the driver to cover for automation faults and conform to regulations that require the driver to retain control authority. In this paper we present results from a driving simulator study with 48 participants in which we tested the performance of three emergency steering intervention schemes. In the first scheme, the driver was decoupled and the automation system had full control over the vehicle. In the second and third schemes, the driver was coupled and the automation system was either given a high impedance or a low impedance. Two types of unexpected automation faults were also simulated. Results showed that a high impedance automation system results in significantly fewer collisions during intended steering interventions but significantly higher collisions during automation faults when compared to a low impedance automation system. Moreover, decoupling the driver did not seem to significantly influence the time required to hand back control to the driver. When coupled, drivers were able to cover for a faulty automation system and avoid obstacles to a certain degree, though differences by condition were significant for only one type of automation fault.

In this paper, we develop theoretical foundations for a new class of rehabilitation robot: body powered devices that route power between a user's joints. By harvesting power from a healthy joint to assist an impaired joint, novel bimanual and self-assist therapies are enabled. This approach complements existing robotic therapies aimed at promoting recovery of motor function after neurological injury. We employ hydraulic transmissions for routing power, or equivalently for coupling the motions of a user's joints. Fluid power routed through flexible tubing imposes constraints within a limb or between homologous joints across the body. Variable transmissions allow constraints to be steered on the fly, and simple valve switching realizes free space and locked motion. We examine two methods for realizing variable hydraulic transmissions: using valves to switch among redundant cylinders (digital hydraulics) or using an intervening electromechanical link. For both methods, we present a rigorous mathematical framework for describing and controlling the resulting constraints. Theoretical developments are supported by experiments using a prototype fluid-power exoskeleton.

In this paper, we explore key design parameters for integrating fluidic logic and actuators for a very-large shape display for application in braille and tactile graphics. We present a simple model of pressure-controlled flow valves, which are analogous to electric transistors. The model is used to highlight the design of a valve that achieves hysteresis (noise immunity) and pressure-gain (signal propagation), both critical goals for creating fluidic logic circuits. Empirical results from a family of valves fabricated with graded design parameters and assembled in a logical NOT gate configuration back up the model. A pressure-gain of up to 26 kPa is achieved for a 80 kPa pressure supply, enabling the output of any logic gate to serve as the input of another in a sequential logic operation. Using the pressure-gain valves, we successfully built a latching memory unit that can be integrated with a pneumatic actuator capable of driving a braille pin. The design is sufficiently scalable and manufacturable to support the realization of a large dense array of pins at braille spacing.

The use of haptic display to refer cues sensed electronically from a prosthetic terminal device promises to improve the function of myoelectrically controlled upper limb prostheses. This promise is often evaluated in experiments involving non-amputees, though the availability of additional haptic feedback from an intact hand (auxiliary feedback) may confound attempts to use non-amputees as stand-ins for amputees. In this paper we test the influence of auxiliary haptic feedback on myoelectric control performance by introducing various grasp conditions in a compensatory tracking task. We ask non-amputees to compensate for the motion of a random signal by producing myoelectric control signals with a hard object, soft object, or no object (requiring co-contraction) in their grasp. The error signal is displayed through a squeeze band worn about the upper arm or a visual display. Our results suggest that the main difference between tracking with haptic and visual feedback is low-frequency drift, and that auxiliary feedback does not substantially influence task performance. Despite the drift, our results show that participants are able to respond to cues presented through the squeeze band in the compensatory tracking task.

This paper presents a novel variable transmission system that is based on the concept of digital hydraulics. In the proposed system, sets of rolling-diaphragm cylinders are mounted via different effective lever arms to an input and output joint. A variable subset of these cylinders is connected via three-way two-position on/off valves to a common hydraulic manifold. This introduces a controllable constraint on the hydraulic flow and creates a programmable hydraulic transmission. With three single-acting cylinders, we could realize 37 different transmission ratios. We investigated the nonholonomic flow constraint analytically, in simulation, and with an experimental prototype. Using water as fluid, we show that a very stiff transmission (124.2 Nm/rad) can be achieved within the range of ±6°. Theoretical transmission ratios are tracked with R-squared values of more than 0.996 and backlash is smaller than 1.4%. Furthermore, we show the applicability of the proposed transmission in the simulation of a body-powered knee-ankle exoskeleton.

Achieving dexterous volitional control of an upper-limb prosthetic device will require multimodal sensory feedback that goes beyond vision. Haptic display is well-positioned to provide this additional sensory information. Haptic display, however, includes a diverse set of modalities that encode information differently. We have begun to make a comparison between two of these modalities, force feedback spanning the elbow, and amplitude-modulated vibrotactile feedback, based on performance in a functional grasp and lift task. In randomly ordered trials, we assessed the performance of N=11 participants (8 able-bodied, 3 amputee) attempting to grasp and lift an object using an EMG controlled gripper under three feedback conditions (no feedback, vibrotactile feedback, and force feed-back), and two object weights that were undetectable by vision. Preliminary results indicate differences between able-bodied and amputee participants in coordination of grasp and lift forces. In addition, both force feedback and vibrotactile feedback contribute to significantly better task performance (fewer slips) and better adaptation following an unpredicted weight change. This suggests that the development and utilization of internal models for predictive control is more intuitive in the presence of haptic feedback.

Advanced prosthetic hands offer the promise of great dexterity; however, myoelectric control techniques, successful with low degree-of-freedom prosthetics, are often set aside by amputees due to the lack of important sensations of touch and effort experienced in the interaction between prosthetic hand and task. In this paper, we explore the efficacy of various modalities of feedback (visual, tactile, visual and tactile, and none) conveying proprioceptive information, specifically the error in joint angles between a desired and actual pose of a virtual prosthetic hand. Our analysis of performance in achieving and maintaining a desired prosthetic hand pose indicates a significant effect of feedback condition, with visual and visual+tactile outperforming tactile alone and a no-feedback condition. Further, the combination of tactile and visual feedback does not seem to have significant drawbacks over visual feedback alone. For tasks that rely on proprioception in the absence of visual feedback, or when attention must be focused elsewhere, we see a performance benefit to the inclusion of tactile cueing, with no lags in reaction times or requirements for increased effort measured by muscle activation.

We introduced haptic cues to the serial reaction time (SRT) task alongside the standard visual cues to explore the relative contributions of haptic and visual cues to motor memory and perceptual memory. Motor learning is a complex process that likely depends on the availability of haptic sensory feedback. Motor learning is also often an implicit process, wherein the learner is not even aware that learning is taking place. The SRT task was devised to study implicit learning. Reaction times (RTs) are reduced with practice even in the absence of explicit learning about a sequence embedded in a train of cues. We adopted an SRT protocol developed by Willingham in 1999 to determine whether haptic cues contribute differently than visual cues to the balance of motor and perceptual learning. Experimental results involving 32 participants showed that sequence learning occurs implicitly with haptic stimuli to much the same extent as visual stimuli. Also, it was revealed that the dependence on motor memory (as opposed to perceptual memory) was greater in the group responding to haptic cues than the group responding to visual cues. Comparing the dependence on motor versus perceptual memory, the haptic group reached marginal significance (p=.09), whereas the visual group difference was not significant (p=.23).

Haptic cues take on meaning as a function of the context in which they are experienced. In interaction with objects in the physical environment, the context always includes a mechanical contact, at which point force and motion variables can be identified, and across which power may flow. In interaction with objects in a virtual or remote environment, it is not necessary for the contact across which haptic responses are rendered to be the same as the contact at which exploratory actions are applied. In this paper, we ask whether force/motion coupling has a significant impact on manual performance or cognitive load. We conducted an experiment in which n=7 participants attempted, while acting through a teleoperator, to discriminate three objects by their stiffness under two conditions. In one condition physical force/motion coupling was present, in the other it was not. To assess cognitive load, we engaged participants in a simultaneous cognitive task that included a response time measure. Results indicated no difference in manual discrimination performance. After rejecting the datasets of three of our participants based on inconsistent strategy by condition, we observed a small, non-significant trend toward lower cognitive load in the condition with physical coupling. Establishing a robust trend will require additional participants. While results are preliminary, we offer our paradigm as an important direction for new inquiry into the distinctions and interrelationships between information and its presentation in various haptic interface applications. Our work is aimed in particular at developing haptic feedback for use in prosthetic applications.

In cognitive science and philosophy, the conception of cognition has evolved rapidly in recent years. It is no longer strictly a process that happens inside the head, intervening between perception and action. Instead, cognition is considered something that subsumes both perception and action, engages the body, and even incorporates the response of objects in the environment to actions taken upon them. One speaks of cognition being embodied and extended. Cognitive processes such as problem solving might actually involve the coupled dynamics of the body and the environment, where the finding of problem solutions involves interactively driving the coupled dynamics toward behaviors that satisfy the task goals of an individual. Researchers in haptics have long been working with the notions of dynamical coupling between the body and environment, with interwoven definitions of perception and action, and with flexible conceptions of the proximal and distal and the physical and virtual. It is perhaps striking to suppose that the activity during which our users wield our interface devices might be considered cognition itself. We suggest in this paper that the field of haptics is uniquely positioned to draw upon and to contribute to active debates in cognitive science, and that a new interpretation of “interface” can be useful in launching such an endeavor.

This paper analyzes inherent conflicts between model-matching goals of haptic rendering and passivity requirements for coupled stability. We apply results from complex analysis to prove that certain linear passive virtual environments cannot be rendered passively with a desired level of accuracy and over a given finite bandwidth. One practical consequence is that, under appropriate hypotheses, passivity will be violated when accurately rendering inertia in a virtual environment that is less than the inertia of the uncompensated hardware dynamics. In a related result, we show that there exists a waterbed-type tradeoff between performance and phase lag in the rendered dynamics. Both design constraints arise from feedback-bandwidth limitations and not sampled-data effects, quantization, or nonlinearities. The key to our analysis is an interpretation of a Bode gain-phase integral relationship that relates magnitude at low frequencies to phase at high frequencies. The performance limitation and the waterbed tradeoff are illustrated through an experimental study.

In this paper, we show that for haptic rendering using position feedback, the structure of the feedback loop imposes a fundamental tradeoff between accurate rendering of virtual environments and sensitivity of closed-loop responses to hardware variations and uncertainty. Due to this tradeoff, any feedback design that achieves high-fidelity rendering incurs a quantifiable cost in terms of sensitivity. Analysis of the tradeoff reveals certain combinations of virtual environment and haptic device dynamics for which performance is achieved only by accepting very poor sensitivity. This analysis may be used to show that certain design specifications are infeasible and may guide the choice of hardware to mitigate the tradeoff severity. We illustrate the predicted consequences of the tradeoff with an experimental study.

A new minimum-distance tracking algorithm is presented for moving convex bodies represented using tiled-together parametric surface patches. The algorithm is formulated by differentiating the geometric minimization problem with respect to time. This produces a hybrid dynamical system that incorporates dependence on rigid body motion, surface shape, and surface boundary interconnectedness. The minimum distance between a pair of previously identified closest features is found by feedback stabilizing the dynamical equations and numerically solving the resulting closed-loop system equations. Maintenance of the minimum distance and the associated closest points during motion is achieved through the action of a feedforward controller and a switching algorithm. The feedforward controller simultaneously accounts for surface shape and motion while the switching controller triggers updates to the extremal feature pair when extremal points on one body cross between Voronoi regions of the other body. In contrast to previously available minimum distance determination algorithms, attractive properties of the new algorithm include a means of determining the highest gain K that maintains stability under a given discretization scheme and a large and easily characterized basin of attraction of the stabilized closest points. These properties may be used to achieve higher computational efficiency. Simulation results are presented for various planar and spatial systems composed of a body and point or composed of two bodies.

A new extremal distance tracking algorithm is presented for parametric curves and surfaces undergoing rigid body motion. The essentially geometric extremization problem is transformed into a dynamical control problem by differentiating with respect to time. Extremization is then solved with the design of a stabilizing controller. We use a feedback linearizing controller. The controller simultaneously accounts for the surface shape and motion while asymptotically achieving (and maintaining) the extremal pair. Thus collision detection takes place in a framework fully analogous to the framework used for the simulation of dynamical response.

A general framework is presented for the design and analysis of cobot controllers. Cobots are inherently passive robots intended for direct collaborative work with a human operator. While a human applies forces and moments, the controller guides motion by tuning the cobot's set of continuously variable transmissions. In this paper, a path-following controller is developed that steers the cobot so as to asymptotically approach and follow a preplanned path. The controller is based on feedback linearization. Generality across cobot architectures is assured by designing the controller in task space and developing transformations between each of four spaces: task space, joint space, a set of coupling spaces, and steering space.

We describe a new robot architecture: the collaborative robot, or cobot. Cobots are intended for direct physical interaction with a human operator. The cobot can create smooth, strong virtual surfaces and other haptic effects within a shared human/cobot workspace. The kinematic properties of cobots differ markedly from those of robots. Most significantly, cobots have only one mechanical degree of freedom, regardless of their taskspace dimensionality. The instantaneous direction of motion associated with this single degree of freedom is actively servo-controlled, or steered, within the higher dimensional taskspace. This paper explains the kinematics of cobots and the continuously variable transmissions (CVT) that are essential to them. Powered cobots are introduced, made possible by a parallel interconnection of the CVT. We discuss the relation of cobots to conventionally actuated robots and to nonholonomic robots. Several cobots in design, prototype, or industrial testbed settings illustrate the concepts discussed.

A general framework is presented for the design and analysis of cobot controllers. Cobots are inherently passive robots intended for direct collaborative work with a human operator. While a human applies forces and moments, the controller guides motion by tuning the cobot's set of continuously variable transmissions. In this paper, a path following controller is developed that steers the cobot so as to asymptotically approach and follow a pre-planned path. The controller is based on feedback linearization. Generality across cobot architectures is assured by designing the controller in task space and developing transformations between each of four spaces: task space, joint space, a set of coupling spaces, and steering space.

Cobots are a class of mechanically passive robotic devices, intended for direct physical collaboration with a human operator. The operator supplies all motive power while the cobot enforces software-defined guiding surfaces, or constraints. Cobots are intrinsically passive, safe devices. This is because, rather than employ powered actuators to produce constraint forces, cobots use "steerable" nonholonomic joints. Constraint forces are mechanical in origin, yet software defined. The simplest possible cobot is a unicycle which is steered by a servo system acting under computer control, but which is moved by a human operator. The unicycle cobot requires essentially no consideration of kinematics. Two fundamental control modes of the unicycle cobot, "virtual caster" and "constraint tracking", are reviewed. More complicated cobots, such as the three-wheeled "Scooter", require a set of kinematic tranformations relating configuration space to joint space. These transformations play a role in cobot control like that of the Jacobian in robot control.