Supernumerary Robotic Limbs (SRLs) have been successfully applied in bracing and as an assistive technology for people with disabilities. These tasks only require perception internal to the SRL-human system. However, SRLs show promise in applications requiring external perception such as opening a door when one’s hands are full. One path toward developing SRLs that accomplish these tasks is to use human-in-the-loop control, thus leveraging the human’s superior perception system to help the SRLs. However, the effects on the user of controlling additional limbs are unclear. This article presents an experimental study where humans, wearing two single degree of freedom SRLs, were instructed to minimize the position error between the subject’s natural and robotic limbs and the corresponding targets, one for each limb. First, subjects performed worse with their natural limbs when asked to perform the task with two natural and two robotic limbs as opposed to with just their natural limbs, suggesting that shared control could help. Second, subjects moved their natural limbs together followed by moving their SRLs together. This informs both the choice of control scheme for the SRLs and the division of labor within a task. Third, subjects showed significant concurrent use of the natural and robotic limbs.

In human motor control studies, end-effector (e.g., hand) trajectories have been successfully modeled using optimization principles. Yet, it remains unclear how such trajectories are updated when the end-effector or task goals are perturbed. Here, we present an approach to human and robotic task-level trajectory planning and modification using geometrical invariance and optimization, allowing to adapt learned movements to a priori unknown boundary conditions. The optimization criterion represents a tradeoff between smoothness (minimum jerk) and accuracy (jerk-accuracy model). We show that planning maximally smooth movements allows recovery from perturbations by superimposing specific affine orbits on maximally smooth preplanned trajectories. The generated trajectories are compared with those resulting from other recent approaches used in robotics. Finally, we discuss conditions for affine invariance of maximally smooth task-space trajectories. Possible applications of this study to both human motor control and robotics research studies are discussed.

A number of robotic studies have recently turned to biological inspiration in designing control schemes for flexible robots. Examples of such robots include continuous manipulators inspired by the octopus arm. However, the control strategies used by an octopus in moving its arms are still not fully understood. Starting from a dynamic model of an octopus arm and a given set of muscle activations, we develop a simulation technique to characterize the stiffness throughout a motion and at multiple points along the arm. By applying this technique to reaching and bending motions, we gain a number of insights that can help a control engineer design a biologically inspired impedance control scheme for a flexible robot arm. The framework developed is a general one that can be applied to any motion for any dynamic model. We also propose a theoretical analysis to efficiently estimate the stiffness analytically given a set of muscle activations. This analysis can be used to quickly evaluate the stiffness for new static configurations and dynamic movements.

Human movements show several prominent features; movement duration is nearly independent of movement size (the isochrony principle), instantaneous speed depends on movement curvature (captured by the 2/3 power law), and complex movements are composed of simpler elements (movement compositionality). No existing theory can successfully account for all of these features, and the nature of the underlying motion primitives is still unknown. Also unknown is how the brain selects movement duration. Here we present a new theory of movement timing based on geometrical invariance. We propose that movement duration and compositionality arise from cooperation among Euclidian, equi-affine and full affine geometries. Each geometry posses a canonical measure of distance along curves, an invariant arc-length parameter. We suggest that for continuous movements, the actual movement duration reflects a particular tensorial mixture of these canonical parameters. Near geometrical singularities, specific combinations are selected to compensate for time expansion or compression in individual parameters. The theory was mathematically formulated using Cartan's moving frame method. Its predictions were tested on three data sets: drawings of elliptical curves, locomotion and drawing trajectories of complex figural forms (cloverleaves, lemniscates and limaçons, with varying ratios between the sizes of the large versus the small loops). Our theory accounted well for the kinematic and temporal features of these movements, in most cases better than the constrained Minimum Jerk model, even when taking into account the number of estimated free parameters. During both drawing and locomotion equi-affine geometry was the most dominant geometry, with affine geometry second most important during drawing; Euclidian geometry was second most important during locomotion. We further discuss the implications of this theory: the origin of the dominance of equi-affine geometry, the possibility that the brain uses different mixtures of these geometries to encode movement duration and speed, and the ontogeny of such representations.

Some studies suggest that complex arm movements in humans and monkeys may optimize several objective functions, while others claim that arm movements satisfy geometric constraints and are composed of elementary components. However, the ability to unify different constraints has remained an open question. The criterion for a maximally smooth (minimizing jerk) motion is satisfied for parabolic trajectories having constant equi-affine speed, which thus comply with the geometric constraint known as the two-thirds power law. Here we empirically test the hypothesis that parabolic segments provide a compact representation of spontaneous drawing movements. Monkey scribblings performed during a period of practice were recorded. Practiced hand paths could be approximated well by relatively long parabolic segments. Following practice, the orientations and spatial locations of the fitted parabolic segments could be drawn from only 2-4 clusters, and there was less discrepancy between the fitted parabolic segments and the executed paths. This enabled us to show that well-practiced spontaneous scribbling movements can be represented as sequences ("words") of a small number of elementary parabolic primitives ("letters"). A movement primitive can be defined as a movement entity that cannot be intentionally stopped before its completion. We found that in a well-trained monkey a movement was usually decelerated after receiving a reward, but it stopped only after the completion of a sequence composed of several parabolic segments. Piece-wise parabolic segments can be generated by applying affine geometric transformations to a single parabolic template. Thus, complex movements might be constructed by applying sequences of suitable geometric transformations to a few templates. Our findings therefore suggest that the motor system aims at achieving more parsimonious internal representations through practice, that parabolas serve as geometric primitives and that non-Euclidean variables are employed in internal movement representations (due to the special role of parabolas in equi-affine geometry).

The two-thirds power law, an empirical law stating an inverse non-linear relationship between the tangential hand speed and the curvature of its trajectory during curved motion, is widely acknowledged to be an invariant of upper-limb movement. It has also been shown to exist in eyemotion, locomotion and was even demonstrated in motion perception and prediction. This ubiquity has fostered various attempts to uncover the origins of this empirical relationship. In these it was generally attributed either to smoothness in hand- or joint-space or to the result of mechanisms that damp noise inherent in the motor system to produce the smooth trajectories evident in healthy human motion. We show here that white Gaussian noise also obeys this power-law. Analysis of signal and noise combinations shows that trajectories that were synthetically created not to comply with the power-law are transformed to power-law compliant ones after combination with low levels of noise. Furthermore, there exist colored noise types that drive non-power-law trajectories to power-law compliance and are not affected by smoothing. These results suggest caution when running experiments aimed at verifying the power-law or assuming its underlying existence without proper analysis of the noise. Our results could also suggest that the power-law might be derived not from smoothness or smoothness-inducing mechanisms operating on the noise inherent in our motor system but rather from the correlated noise which is inherent in this motor system.

In telerobotic systems human actions are mapped to robot actions. In an illustrative object manipulation experiment various human grasps were translated to configurationally similar robotic grasps. The experiment's results highlight the problems and suboptimal performance incurred when such a resemblance is maintained. A new approach to telerobotics based on the construction of object-action pairs is presented. Actions are identified in the context of the object they are being performed on according to features extracted from the human grasp and transport motion. A priori knowledge is introduced to the robot controller using object centered programming and a relational database.

Intelligent sensing, planning, and control of a prototype robotic melon harvester is described. The robot consists of a Cartesian manipulator mounted on a mobile platform pulled by a tractor. Black and white image processing is used to detect and locate the melons. Incorporation of knowledge-based rules adapted to the specific melon variety reduces false detections. Task, motion and trajectory planning algorithms and their integration are described. The intelligent control system consists of a distributed blackboard system with autonomous modules for sensing, planning and control. Procedures for evaluating performance of the robot performing in an unstructured and changing environment are described. The robot was tested in the field on two different melon cultivars during two different seasons. Over 85% of the fruit were successfully detected and picked.

This work deals with the problem of end-effector trajectory modification for a robot manipulator when it must respond to unexpected changes in target location. Trajectory modification and corrections are particularly important in dealing with dynamic tasks. In this paper, we present and discuss the superposition strategy derived from the study of arm trajectory modification in human subjects. According to this strategy, the motion toward the initial target location continues unmodified as planned from its beginning to its end even after the target location has unexpectedly changed. However, a trajectory leading from the first target to the final one is added vectorially to the initial one to yield the combined modified motion. A method for choosing the temporal parameters of this trajectory modification scheme is suggested so as to minimize the total travelling time under existing kinematic constraints (including both joint and hand space constraints). Then, a variant of this strategy is presented, dealing with trajectory modification in the case that the targets (both the initial and final ones) specify the desired end-point orientation rather than position.

We present a general scheme for learning sensorimotor tasks, which allows rapid online learning and generalization of the learned knowledge to unfamiliar objects. The scheme consists of two modules, the first generating candidate actions and the second estimating their quality. Both modules work in an alternating fashion until an action which is expected to provide satisfactory performance is generated, at which point the system executes the action. We developed a method for off-line selection of heuristic strategies and quality predicting features, based on statistical analysis. The usefulness of the scheme was demonstrated in the context of learning visually guided grasping. We consider a system that coordinates a parallel-jaw gripper and a fixed camera. The system learns to estimate grasp quality by learning a function from relevant visual features to the quality. An experimental setup using an AdeptOne manipulator was developed to test the scheme.

A scheme for learning to grasp objects using visual information is presented. The learning problem is divided into two separate subproblems: choosing grasping points and predicting the quality of a given grasp. For each grasp we store location parameters that code the locations of the grasping points, quality parameters that are relevant features for the assessment of grasp quality, and the associated grade. The location parameters, using a special coding which is not object specific, are used to locate grasping points on new target objects. A function from the quality parameters to the grade is learned from examples. Grasp quality for novel situations can be generalized and estimated using the learned function. An experimental setup using an AdeptOne manipulator was developed to test this scheme. The system had demonstrated an ability to grasp a relatively wide variety of objects, and its performance had significantly improved with practice following a small number of trials. The knowledge learned for a set of objects was successfully generalized to new objects.

An evaluation of the associative search network (ASN) learning scheme when used for learning control parameters for robot motion is presented. The control method used is impedance control in which the controlled variables are the dynamic relations between the motion variables of the robot manipulator's tip and the forces exerted by the tip. The main task used is that of wiping a surface whose geometry is not precisely known. The learning scheme does not use a model of the robot and its environment. It is a stochastic scheme that uses a single scalar value as a measure of the system performance. The scheme is found to perform quite well. A few variants of the main scheme are discussed. Modifying the virtual trajectory, externally to the ASN scheme, shows an improved performance.>

A near-minimum-time task-planning algorithm for fruit-harvesting robots having to pick fruits at N given locations is presented. For the given kinematic and inertial parameters of the manipulator, the algorithm determines the near-optimal sequence of fruit locations through which the arm should pass and finds the near-minimum-time path between these points. The sequence of motions was obtained by solving the traveling salesman problem (TSP) using the distance along the geodesics in the manipulator's inertia space, between every two fruit locations, as the cost to be minimized. The proposed algorithm was applied to define the motions of a citrus-picking robot and was tested for a cylindrical robot on fruit position data collected from 20 trees. Significant reduction in the required computing time was achieved by dividing the volume containing the fruits into subvolumes and estimating the geodesic distance rather than calculating it.>