Nearly all state-of-the-art SLAM algorithms are designed to exploit patterns in data from specific sensing modalities, such as time-of-flight and structured light depth sensors, or RGB cameras. This specialization increases localization accuracy in domains where the given modality detects many high-quality features, but comes at the cost of decreasing performance in other, less favorable environments. For robotic systems that may experience a wide variety of sensing conditions, this difficulty in generalization presents a significant challenge. In this paper, we propose running several computationally cheap SLAM front ends in parallel and choosing the most promising feature set online. This problem is similar to the Algorithm Selection Problem (ASP), but has several complicating factors that preclude application of existing methods. We first provide an extension of the ASP formalism that captures the unique challenges in the SLAM setting, and then, based on this formalism, we propose modeling the SLAM ASP as a partially observable Markov decision process (POMDP). Our experiments show that dynamically selecting SLAM front ends, even myopically, improves localization robustness compared to selecting a static front end, and that using a POMDP policy provides even greater improvement.

Autonomous systems often use approximate planners that exploit state abstractions to solve large MDPs in real-time decision-making problems. However, these planners can eliminate details needed to produce effective behavior in autonomous systems. We therefore propose a novel model, a partially abstract MDP, with a set of abstract states that each compress a set of ground states to condense irrelevant details and a set of ground states that expand from a set of expanded abstract states to retain relevant details. This papers offers (1) a definition of a partially abstract MDP that (2) generalizes its ground MDP and its abstract MDP and exhibits bounded optimality depending on its abstract MDP along with (3) a lazy algorithm for planning and execution in autonomous systems. The result is a scalable approach that computes near-optimal solutions to large problems in minutes rather than hours.

Deep learning has been successful in a variety of applications that have high-dimensional state spaces such as object recognition, video games, and machine translation. Deep neural networks can automatically learn important features from high-dimensional state given large training datasets. However, the success of deep learning in robot systems in the realworld is limited due to the cost of obtaining these large datasets. To overcome this problem, we propose an information-theoretic, intrinsically motivated, self-labeling mechanism using closed-loop control states. Taking this approach biases exploration to informative interactions-as such, a robot requires much less training to achieve reliable performance. In this paper, we explore the impact such an approach has on learning how to grasp objects. We evaluate different intrinsic motivators present in the literature applied appropriately in our framework and discuss the benefits and drawbacks of each.

In this work, we introduce a framework for performing dexterous manipulations on the humanoid robot Robonaut-2. This framework memorizes how actions change perceptions and can learn a sequence of actions based on demonstrations. With the anthropomorphic Robonaut-2 hand and arm, a variety of manipulation tasks such as grasping novel objects, rotating a drill for grasping, and tightening a bolt with a ratchet can be accomplished. This framework was also used to compete in the IROS2018 Fan Robotic Challenge that requires manipulating a hand fan and was a winner of the phase I modality A competition.

We propose an intelligent visuomotor system that interacts with the environment and memorizes the consequences of actions. As more memories are recorded and more interactions are observed, the agent becomes more capable of predicting the consequences of actions and is, thus, better at planning sequences of actions to solve tasks. In previous work, we introduced the aspect transition graph (ATG) which represents how actions lead from one observation to another using a directed multi-graph. In this work, we propose a novel aspect representation based on hierarchical CNN features, learned with convolutional neural networks, that supports manipulation and captures the essential affordances of an object based on RGB-D images. In a traditional planning system, robots are given a pre-defined set of actions that take the robot from one symbolic state to another. However symbolic states often lack the flexibility to generalize across similar situations. Our proposed representation is grounded in the robot's observations and lies in a continuous space that allows the robot to handle similar unseen situations. The hierarchical CNN features within a representation also allow the robot to act precisely with respect to the spatial location of individual features. We evaluate the robustness of this representation using the Washington RGB-D Objects Dataset and show that it achieves state of the art results for instance pose estimation. We then test this representation in conjunction with an ATG on a drill grasping task on Robonaut-2. We show that given grasp, drag, and turn demonstrations on the drill, the robot is capable of planning sequences of learned actions to compensate for reachability constraints.

In this work, we provide a solution for posturing the anthropomorphic Robonaut-2 hand and arm for grasping based on visual information. A mapping from visual features extracted from a convolutional neural network (CNN) to grasp points is learned. We demonstrate that a CNN pre-trained for image classification can be applied to a grasping task based on a small set of grasping examples. Our approach takes advantage of the hierarchical nature of the CNN by identifying features that capture the hierarchical support relations between filters in different CNN layers and locating their 3D positions by tracing activations backwards in the CNN. When this backward trace terminates in the RGB-D image, important manipulable structures are thereby localized. These features that reside in different layers of the CNN are then associated with controllers that engage different kinematic subchains in the hand/arm system for grasping. A grasping dataset is collected using demonstrated hand/object relationships for Robonaut-2 to evaluate the proposed approach in terms of the precision of the resulting preshape postures. We demonstrate that this approach outperforms baseline approaches in cluttered scenarios on the grasping dataset and a point cloud based approach on a grasping task using Robonaut-2.

This paper presents an active, model-based recognition system. It applies information theoretic measures in a belief-driven planning framework to recognize objects using the history of visual and manual interactions and to select the most informative actions. A generalization of the aspect graph is used to construct forward models of objects that account for visual transitions. We use populations of these models to define the belief state of the recognition problem. This paper focuses on the impact of the belief-space and object model representations on recognition efficiency and performance. A benchmarking system is introduced to execute controlled experiments in a challenging mobile manipulation domain. It offers a large population of objects that remain ambiguous from single sensor geometry or from visual or manual actions alone. Results are presented for recognition performance on this dataset using locomotive, pushing, and lifting controllers as the basis for active information gathering on single objects. An information theoretic approach that is greedy over the expected information gain is used to select informative actions, and its performance is compared to a sequence of random actions.

We propose a log-space solution for robotic path planning with harmonic functions that solves the long-standing numerical precision problem. We prove that this algorithm: (1) performs the correct computations in log-space, (2) returns the true equivalent path using the log-space mapping, and (3) has a strong error bound given its convergence criterion. We evaluate the algorithm on 7 problem domains. A Graphics Processing Unit (GPU) implementation is also shown to greatly improve performance. We also provide an open source library entitled epic with extensive ROS support and demonstrate this method on a real humanoid robot: the uBot-6. Experiments demonstrate that the log-space solution rapidly produces smooth obstacle-avoiding trajectories, and supports planning in exponentially larger real-world robotic applications.

The use of robots in stroke rehabilitation has become a popular trend in rehabilitation robotics. However, despite the acknowledged value of customized service for individual patients, research on programming adaptive therapy for individual patients has received little attention. The goal of the current study is to model teletherapy sessions in the form of a generative process for autonomous therapy that approximate the demonstrations of the therapist. The resulting autonomous programs for therapy may imitate the strategy that the therapist might have employed and reinforce therapeutic exercises between teletherapy sessions. We propose to encode the therapist's decision criteria in terms of the patient's motor performance features. Specifically, in this work, we apply Latent Dirichlet Allocation on the batch data collected during teletherapy sessions between a single stroke patient and a single therapist. Using the resulting models, the therapeutic exercise targets are generated and are verified with the same therapist who generated the data.

We report the results of standardized tests on a single subject with a stroke at 4, 20 and 28 weeks after completion of the study. These results follow from previous work[1]. The subject demonstrated sustained improvement in motor function 28 weeks after completing the study. In addition to quantitative results, the questionnaire results by the subject and the spouse testify that the subjective user experience was also positive. This further advocates the use of general purpose robots to complement human therapists.

We describe a robot system that autonomously acquires skills through interaction with its environment. The robot learns to sequence the execution of a set of innate controllers to solve a task, extracts and retains components of that solution as portable skills, and then transfers those skills to reduce the time required to learn to solve a second task.

In this case study, we examine the functional utility of an embodied agent as an interactive medium in stroke rehab. A set of physical rehab exercises is conducted through the direct engagement of an embodied agent, the uBot-5. Based on the preliminary data, we argue that a general-purpose embodied agent has a potential to functionally complement human therapists in providing rehab to stroke patients.

In this paper, a multi-agent search scheme is presented that supports the recognition of activities and, thus, learning methods for cooperative human-robot interaction. In our approach, stochastic models of human search activity are used to estimate state for HRI. The robot updates a Probabilistic Distribution Function of the target object using the observations and the estimated state of human peers. By this means the robot can choose places to search to compensate the behavior of human peers. This paper also presents an implicit interface design for robot assisted tasks, which allows the robot to infer the intention of the user and to provide assistance autonomously. It reduces the cognitive workload of the user and therefore is useful for elder care applications. The effectiveness and the efficiency of the proposed approaches are demonstrated in the experimental results.

In recent years, the potential of robot-mediated upper extremity therapy has been widely acknowledged in stroke rehabilitation. The primary focus of rehabilitation robotics has been on building exoskeleton robots as well as devising control strategies that leverage robotic forces to adaptively assist and resist the movements of patients' impaired arms during repetitive reach-touch exercises. In our single subject case study, a robot presents exercise target positions that are successively more difficult to the patient. This serves to adaptively challenge the patient, while inducing voluntary reach-touch movements. Through data collected during 12 weeks of therapy with a single subject, we demonstrate that a general purpose robot can induce desired therapeutic exercise movements from a patient. In addition, we have observed that the challenge level can be adapted as the patient improves his motor function, which leads to observable improvements in the motor function of the patient.

We introduce CST, an algorithm for constructing skill trees from demonstration trajectories in continuous reinforcement learning domains. CST uses a changepoint detection method to segment each trajectory into a skill chain by detecting a change of appropriate abstraction, or that a segment is too complex to model as a single skill. The skill chains from each trajectory are then merged to form a skill tree. We demonstrate that CST constructs an appropriate skill tree that can be further refined through learning in a challenging continuous domain, and that it can be used to segment demonstration trajectories on a mobile manipulator into chains of skills where each skill is assigned an appropriate abstraction.

A key problem in robot grasping is that of positioning the manipulator contacts so that an object can be grasped. In unstructured environments, contact positions are typically planned based on range or visual measurements that are used to reconstruct object geometry. However, because it is difficult to measure the complete object geometry precisely in common grasp scenarios, it is useful to employ additional techniques to adjust or refine the grasp using local information only. In particular, grasp control techniques can be used to improve a grasp by adjusting the contact configuration after making initial contact with an object by using measurements of local object geometry at the contacts. This paper proposes three variations on null-space grasp control: an approach that combines multiple grasp objectives to improve a grasp. Two of these variations are theoretically demonstrated to converge to force-closure configurations for arbitrary convex objects when grasping with two contacts. All variations are found to converge in simulation. Robot-grasping experiments are reported that show the approach to be useful in practice.

In this paper we present the design of a decentralized vision-based object search system that can be used for elder care in a smart environment. In our approach, each autonomous search agent maintains separate estimates of the probability density function (PDF) of the object location and makes independent decisions about its search process. Asynchronous cooperative search is achieved by transmitting perceptual information among the agents. Our work also investigates how context such as the detection history and density of activity by people influence the estimation of the prior PDF of the target and the use of this information to improve the search efficiency. Our experimental results demonstrate that the proposed cooperative search strategy is efficient and the methods we use to incorporate contextual information into the target's posterior PDF can improve the efficiency further.

We present a framework for the programming of manipulation behavior by means of an intrinsic reward function that encourages the building of deep control knowledge. We show how this framework can be used to teach new manipulation skills in a hierarchical and incremental fashion. We demonstrate the contributions of this paper on a humanoid robot through three incremental learning stages.

Any given task can be solved in a number of ways, whether through path-planning, modeling, or control techniques. In this paper, we present a methodology for natural task decomposition through the use of intrinsically meaningful potential fields. Specifically, we demonstrate that using classical conditioning measures in a concurrent control framework provides a domain-general means for solving tasks. Among the conditioning measures we use are manipulability [T. Yoshikawam, 1985], localizability [J. Uppala et al., 2002], and range of motion. To illustrate the value of our approach we demonstrate its applicability to an industrially relevant inspection task.

Most mobile robots in existence today have large, heavy, statically stable bases. To enable manipulation, actuated arms and effectors are attached to the base providing limited postural control. On the other hand, some of the latest developments in humanoid robotics have succeeded in reducing the footprint and raising the center of mass of a robot using whole body postural control and balancing. By comparing the magnitude of various forces that can be applied to the environment, we demonstrate the advantages of whole body postural control. The results presented suggest that for pushing, pulling, or carrying types of tasks, using whole body postural control can lead to higher performance by allowing a platform to apply more force to the environment

In robot and sensor networks, one of the key challenges is to decide when and where to deploy sensory resources to gather information of optimal value. The problem is essentially one of planning, scheduling and controlling the sensors in the network to acquire data from an environment that is constantly varying. The dynamic nature of the problem precludes the use of traditional rule-based strategies that can handle only quasi-static context changes. Automatic context derivation procedures are thus essential for providing fault recovery and fault pre-emption in such systems. We posit that the quality of a sensor network configuration depends on sensor coverage and geometry, sensor allocation policies, and the dynamic processes in the environment. In this paper, we show how these factors can be manipulated in an adaptive framework for robust run-time resource management. We demonstrate our ideas in a people tracking application using a network of multiple cameras. The task specification for our multi-camera network is one of allocating a camera pair that can best localize a human subject given the current context. The system automatically derives policies for switching between camera pairs that enable robust tracking while being attentive to performance measures. Our approach is unique in that we do not make any a priori assumptions about the scene or the activities that take place in the scene. Models of motion dynamics in the scene and the camera network configuration steer the policies to provide robust tracking.

We describe a method for predicting user intentions as part of a human-robot interface. In particular, we show that funnels, i.e., geometric objects that partition an input space, provide a convenient means for discriminating individual objects and for clustering sets of objects for hierarchical tasks. One advantage of the proposed implementation is that a simple parametric model can be used to specify the shape of a funnel, and a straightforward heuristic for setting initial parameter values appears promising. We discuss the possibility of adapting the user interface with machine learning techniques, and we illustrate the approach with a humanoid robot performing a variation of a standard peg-insertion task.

In dexterous manipulation, an object must be reconfigured while maintaining a stable grasp. This may require that the object be re-grasped in order to avoid finger workspace limits. We present a set of closed-loop controllers designed to achieve force-related objectives such as wrench closure, and show how they may be concurrently combined. Furthermore, we show that dexterous manipulation behavior may be generated by sequencing concurrent combinations of these controllers. We show that dexterous manipulation can be viewed as a task that is accomplished in the context of a wrench closure constraint. We hypothesize this approach can generalize to any task that must be accomplished while maintaining a set of constraints.

We investigate an approach for simultaneously committing to mul- tiple activities, each modeled as a temporally extended action in a semi-Markov decision process (SMDP). For each activity we de- fine a set of admissible solutions consisting of the redundant set of optimal policies, and those policies that ascend the optimal state- value function associated with them. A plan is then generated by merging them in such a way that the solutions to the subordinate activities are realized in the set of admissible solutions satisfying the superior activities. We present our theoretical results and em- pirically evaluate our approach in a simulated domain.

Although it is mechanically possible for a robot manipulator to grasp using non-fingertip contacts, there are few examples of this. We call non-fingertip grasping such as grasping with proximal finger phalanges or grasping with the sides of arms "whole body grasping". While robotic demonstrations are rare, humans commonly use whole body grasps to interact with the world. One of the distinctive features of whole body grasping is the kinematic coupling among potential contacts. This kinematic coupling introduces extra constraints into the grasp synthesis problem. In this paper, we extend recent grasp control techniques to whole body grasping. We show how the grasp control may be parameterized with a set of contact points on the surface of the robot manipulator that enables the grasp search to handle the extra kinematic coupling constraints and find whole body grasps.

Multiple, independent robot platforms promise significant advantage with respect to robustness and flexibility. However, coordination between otherwise independent robots requires the exchange of information; either implicitly (as in gestural communication), or explicitly (as in message passing in a communication network.) In either case, control processes resident on all coordinated peers must participate in the collective behavior. This paper evaluates the potential to scale such a coupled control framework to many participating individuals, where scalability is evaluated in terms of the schedulability of coupled, distributed control processes. We examine how schedulability affects the scalability of a robot system, and discuss an algorithm used for off-line schedulability analysis of a distributed task model. We present a distributed coordinated search task and analyze the schedulability of the designed task structure. We are able to analyze communication delays in the system that put upper bounds on the size of the robot teams. We show that hierarchical methods can be used to overcome the scalability problem. We propose that schedulability analysis should be an integrated part of a multi-robot team design process.

Teams of robots can be redundant with respect to a given task. This redundancy can be exploited to pursue additional objectives during the execution of the task. In this paper, we describe a control-based method to exploit such redundancy for the execution of additional behavior, leading to the improvement of overall performance. The control-based method provides a suitable mechanism for combining controllers with different objectives. The mechanism ensures that the subordinate controllers do not interfere with the superior controllers. Thus it allows to build controllers exhibiting complex behavior from simple primitives, while maintaining their provable performance characteristics. The effectiveness of the framework is demonstrated by experiments with a multi-robot exploration task.

Measures of dynamic manipulability summarize a manipulator's capacity to generate accelerations for arbitrary tasks, and such measures are useful tools for the design and control of general-purpose robots. Existing measures, however, downplay the effects of velocity or else ignore them altogether. In this paper we derive the relationship between joint velocity and end-effector acceleration, and through case studies we demonstrate that velocity has a complex, non-negligible effect on manipulability. We also provide evidence that movement near a singularity is beneficial for certain tasks.

Presents techniques for exploiting redundancy in teams of mobile robots. In particular, we address tasks involving the kinematic coordination of several communicating robots. Teams area modeled as highly redundant spatial mechanisms for which multi-objective, concurrent controllers are constructed using a generalization of null-space control. The goal is to develop a methodology in which the robustness and error suppression in a control theoretic substrate can be used to preserve critical properties in teams of reactive robots. The resulting "safe" control options can then be explored while guaranteeing global compliance with system specifications. The proposed architecture depends on a set of concurrent, low-dimensional control processes that interact in a well-defined manner. Cascaded null space projections and coordination templates are used to manage control interactions across platforms that actively maintain constraints for pairs of robots. Pairwise policies can then be combined to represent coordinated, multi-robot tasks. To illustrate the approach, we demonstrate a distributed control that maintains critical connectivity in line-of-sight communication networks.

A reactive and concurrent control framework for viewpoint control is developed. The viewpoint control task is decomposed into three control objectives namely; obstacle avoidance, visibility and precision. A moving object is tracked in two panoramic sensors using color, and a scalar uncertainty metric of the object position estimate is introduced. Individual control objectives are accomplished by planning paths using harmonic functions and the task is accomplished using a new subject-to composition operator. It is shown that the system is stable under this composition. The system is demonstrated for tracking a human subject using a fixed panoramic sensor and another panoramic sensor mounted on a mobile platform.

Object manipulation using multi-fingered dextrous robot hands poses formidable control challenges due to the complexity of the manipulator, its workspace limitations. To address large-scale manipulation tasks, finger gaits are often used to reposition fingers. In this paper, an approach to finger gaiting is presented which constructs behavior on-line by activating combinations of reusable feedback control laws with formal stability and convergence properties drawn from a control basis. Finger gaits are constructed as finite state control strategies in a DEDS framework, while actual contact locations and object motions are computed reactively based on local contact information. This leads to manipulation strategies that are robust with respect to a limited range of perturbations, object geometries, and manipulator kinematics. To demonstrate this, two finger gaits are constructed and applied to different object geometries and robot hands.

Much of the tradition in robot grasping is rooted in geometrical, planning-based approaches in which it is assumed that object geometries are well modeled a priori. Some recent approaches have chosen instead to deal with objects of unknown geometry. These techniques treat grasping as an active sensory-driven problem. At any given time, finger contacts are incrementally displaced along the object's local surface using a single control law. In this paper, we extend this approach by allowing multiple control laws to be active simultaneously. Three control laws are combined by projecting the actions of subordinate control laws into other control law nullspaces. The resulting composite controller finds grasps that are more robust than the component primitives in isolation. Finally, we show how this approach may be used on hand/arm manipulation systems with arbitrary kinematics.

A set of impedance controllers is used for both state estimation and tracking control on a mobile robot. State estimation is based on the states of a family of impedance controllers and tracking is implemented through a single controller from this set. Reinforcement learning techniques are used to create switching policies that optimize time or energy in a path tracking task.

We present a methodology used to implement artificial animated creatures. Our control architecture considers using local sensory information to provide the creatures with perception, deals with dynamics and kinematics issues and provides a well structured way to control the creature resources. Three artificial creatures with different sensors/actuators/dynamics characteristics show the versatility of the architecture. As a practical result, by using the same control structure, the artificial animated creatures are able to perform different tasks, like visual monitoring of their environments and progression on a rough terrain.

Many realistic visual recognition tasks are "open" in the sense that the number and nature of the categories to be learned are not initially known, and there is no closed set of training images available to the system. We argue that open recognition tasks require incremental learning methods, and feature sets that are capable of expressing distinctions at any level of specificity or generality. We describe progress toward such a system that is based on an infinite combinatorial feature space. Feature primitives can be composed into increasing complex and specific compound features. Distinctive features are learned incrementally, and are incorporated into dynamically updated Bayesian network classifiers. Experimental results illustrate the applicability and potential of our approach.

A critical problem in sensory interpretation is the derivation of the right mapping between sensor signals and properties of the world. In this regard, the method proposed uses active sensing to acquire environmental features for reliable navigation in indoor environments. The features utilized are lines, corners, and edges extracted from common indoor room features such as walls, room corners and room edges, respectively. These features are abundant in indoor environments and allow for closed-form location estimates from time-of-flight sonar information. Subsequently, a multiple hypothesis testing procedure is utilized to accurately localize and identify those features. Experiments using a new active sonar system, implemented using a 3 DOF binaural sonar head, shows that the system is able to correctly localize and identify indoor environmental features, and that active sensing strategies based on sensor reconfiguration makes the process of feature extraction more accurate.

We present mechanisms for attention control and pattern categorization as the basis for robot cognition. For attention, we gather information from attentional feature maps extracted from sensory data constructing salience maps to decide where to foveate. For identification, multi-feature maps are used as input to an associative memory, allowing the system to classify a pattern representing a region of interest. As a practical result, our robotic platforms are able to select regions of interest and perform shifts of attention focusing on the selected regions, and to construct and maintain attentional maps of the environment in an efficient manner.

According to recent theories of sensorimotor development in biological systems, the dynamics of physical interaction with the world encodes control knowledge. Control is derived by reinforcing and learning to predict constructive patterns of interaction, and behavior is an artifact of coupled dynamical systems with a number of controllable degrees of freedom. For grasping and manipulation, we propose a closed-loop control process that is parametric in the number and identity of contact resources. In this paper, we show how control decisions can be made by estimating patterns of membership in a family of prototypical dynamic models. A grasp controller can thus be tuned continuously online to optimize performance over a variety of object geometries. This same process can be used to estimate the haptic category in which the object resides. We illustrate how a grasping policy that is incrementally optimal for several objects can be acquired using our Salisbury hand with tactile sensor feedback.

Humans learn strategies for visual discrimination through interaction with their environment. Discrimination skills are refined as demanded by the task at hand, and are not a priori determined by any particular feature set. Tasks are typically incompletely specified and evolve continually. This work presents a general framework for learning visual discrimination that addresses some of these characteristics. It is based on an infinite combinatorial feature space consisting of primitive features such as oriented edgels and texture signatures, and compositions thereof. Features are progressively sampled from this space in a simple-to-complex manner. A simple recognition procedure queries learned features one by one and rules out candidate object classes that do not sufficiently exhibit the queried feature. Training images are presented sequentially to the learning system, which incrementally discovers features for recognition. Experimental results on two databases of geometric objects illustrate the applicability of the framework.

This work explores techniques for sonar sensor fusion in the context of environmental feature detection and identification for navigation tasks. By detecting common features in indoor environments and using them as landmarks, a robot can navigate reliably, recovering its pose when necessary. Preliminary results on a multiple hypothesis testing procedure for feature localization and identification show that accurate feature information can be acquired with adequate sonar models and configurations. In addition, a method that associates sonar configuration with the precision of feature extraction is discussed, as well as its utility for guiding an active sonar sensor.

This paper presents an online technique for constructing walking gaits for legged robots on irregular terrain. The approach involves designating sequences of closed-loop control laws derived from a control basis. A control basis is a set of artificial potential functions that are device and task independent. A variety of walking contexts can be achieved by varying the sequence of controllers drawn from the control basis, avoiding the need to re-plan leg placements or redesign task-specific behaviors. A control basis previously shown to span the task of walking over flat terrain is shown to also span the task of locomotion over unknown piece-wise planar terrain by incrementally modifying the control composition policy. The resulting gaits are aperiodic, discontinuous, non-sequential, and incorporate controlled slippage of the feet.

This paper addresses the online grasp synthesis problem, in which statically stable grasp configurations are to be generated given instantaneous contact information. We use the preimage of a given grasp controller to address object resolution, friction compensation, and resource allocation in a online fashion. The approach can be applied to piecewise smooth object geometries, and it can be extended to accommodate any number of contacts and 3D objects.

This paper presents a distributed control approach to legged locomotion that constructs behavior online by activating combinations of reusable feedback control laws drawn from a control basis. Sequences of such controller activation result in flexible aperiodic step sequences based on local sensory information. Different tasks are achieved by varying the composition functions over the same basis controllers, rather than by geometric planning of leg placements or the design of new task-specific behaviors. In addition, the device-independent nature of the control basis allows its generalization not only over task domains, but also over different hardware platforms. To show the applicability of this approach, a control basis and two generic control gaits for four-legged walking are introduced and tested on an even terrain walking task in an unknown environment.

This paper presents a method for coordinating the motion of multiple manipulators that share a common workspace. The approach employs independent motion controllers for each manipulator coupled by way of common configuration space constraints-each robot is represented as a constraint in the other configuration spaces. The harmonic function-based controller for each robot guarantees collision-free motion up to dynamic limits. Heuristic constraints are introduced to compensate for the lack of global information in the resulting distributed controller. The relationship between the distributed controller and the task planner is discussed and results of an experimental implementation are presented.

Control based on artificial potentials results in, bounded-torque controllers. The authors extend this notion by devising energy-reference controllers that use harmonic potentials to produce collision-free kinodynamic behavior. In addition, the phase space structure of a manipulator controlled by an artificial potential "copies" the critical point structure of that potential. This result is used in conjunction with harmonic potentials to generate and control convergent and repetitive motions for a manipulator.

This article discusses a set of three controllers that can be used as a basis for visual servoing tasks. These controllers map image plane feature errors to actuator commands through visuo-motor Jacobians. It is shown how unary and binary image plane constraints can be used to specify visual servoing tasks. A specific visual servoing task is described and the experimental results are presented.

This paper discusses how grasp synthesis can be cast as a control composition problem. Two elemental grasp controllers are presented and composed into a general purpose grasp controller through the use of standard optimal control techniques. The resulting grasp synthesis procedure is scalable on the number of contacts, is not tied to any specific object representation scheme, and can incorporate kinematic constraints into its resolution.>

This paper discusses the issues involved in developing a grasp controller, within the framework of control composition, and introduces control pre-imaging. Pre-imaging is a design technique for augmenting the performance of a baseline controller through the opportune activation of metalevel control actions. Such metalevel control actions are activated whenever the current system state is associated to past unsuccessful patterns of interaction between the existing predictable, stable controller and the environment. The technique is illustrated by the implementation of a grasp controller capable of generating grasp configurations through successive, local refinements, given the position and normal of each contact.>

We present a practical method for autonomous synthesis of appropriate admittance behavior for robust high-precision robotic assembly. Because our approach relies on online learning of the appropriate admittance through repeated attempts at the assembly operation, we are able to circumvent the problems alternative approaches have in trying to model the interactions between the robot and its environment. Test results on the peg-in-hole insertion task show that the performance of our approach compares favorably with that of other methods recently proposed for high-precision chamferless peg-in-hole insertion.>

This paper employs proprioceptive information (joint angles and torques) to estimate properties of the contact between a planar robot and an unknown object without specifically requiring strategic manipulator motions. The algorithm presented tackles this task in two stages; a contact localization analysis is followed by a force domain contact detection analysis. In the former, the Cartesian endpoint velocities are used for each link to obtain an estimate of the location of a hypothetical contact point on the link surface. A second observer, based on the displacement between two consecutive postures of the manipulator, provides an estimate of the error associated with this location. This data is fused over time by tracking the contact location using a linear observer and results in a hypothetical contact location, an associated uncertainty region, and a surface normal estimate. The detection phase uses torque domain evidence and the location estimates to verify the existence of each of the contacts. This process allows the detection of one contact per link and provides estimates of contact location, velocity, surface normal, and contact force.>

A peg-in-hole insertion task is used as an example to illustrate the utility of direct associative reinforcement learning methods for learning control under real-world conditions of uncertainty and noise. An associative reinforcement learning system has to learn appropriate actions in various situations through a search guided by evaluative performance feedback The authors used such a learning system, implemented as a connectionist network, to learn active compliant control for peg-in-hole insertion. The results indicated that direct reinforcement learning can be used to learn a reactive control strategy that works well even in the presence of a high degree of noise and uncertainty.>

The problem of planning the interaction of multifingered robot hand with objects in its environment is considered. A sensor-based planning methodology is required in this domain, and consequently, it is concluded that incremental planning mechanisms are required to permit frequent sensory registration. The formulation of incremental planning techniques or behaviors that generate grasping strategies is addressed. Finger, hand, and grasp quality metrics are discussed, followed by a detailed description of an important grasping behavior: wrench closure and registration. A means of constructing composite grasping behaviors that address general manipulation tasks is proposed. Examples illustrating its performance on simple grasping tasks are included.>

Multifingered manipulators are mechanically redundant and so may address a broad array of object geometries, more general force domain tasks, and nested grasp objectives. Many complex robot systems, such as multifingered manipulators, legged platforms, and teams of kinematically independent robots, can benefit from techniques which determine how to use environmental geometry to advantage in interaction tasks. A multiple-resolution object representation in the force domain which facilitates grasp synthesis is presented. A means for describing grasp objectives and a control strategy for designing the geometry of contact mediated by force domain goals are presented.>

The authors propose an approach to three-dimensional object recognition using computer-aided design (CAD)-based geometry models for freeform surfaces. Geometry is modeled with rational B-splines by defining surface patches and then combining these into a volumetric model of the object. Characteristic features are then extracted and subjected to a battery of tests to select an 'optimal' subset of surface features which are robust with respect to the sensor being used (e.g., laser range finder versus passive stereo) and permit recognition of the object from any viewing position. These features are then organized into a strategy tree that defines the order in which the features are sought, and any corroboration required to justify issuing a hypothesis. The authors propose the use of geometric sensor data integration techniques as a means for formally selecting surface features on free-form objects in order to build recognition strategies.>

The authors' goal is the creation of a framework within which independent objectives of a manipulation process can be used to direct a manipulation strategy. The set of contacts that are applied to a task can be partitioned into subsets with independent objectives. A system of this sort is flexible enough to manage large numbers of contacts and to address manipulation tasks that require the removal and replacement of fingers. A simulator has been constructed, and results of its application to position synthesis for initial grasps is presented. The manipulation testbed under construction at the University of Utah using the Utah/MIT dextrous hand are discussed.>

We describe an approach which facilitates and makes explicit the organization of the knowledge necessary to map robotic system requirements onto an appropriate assembly of algorlthms, processors, sensors, and actuators. In order to achieve this mapping several kinds of knowledge are needed. In this paper, we describe a system under development which exploits the Computer Aided Design (CAD) database in order to synthesize. •recognition Code for vision systems (both 2-D and 3-D), •grasping sites for simple parallel grippers, and •manipulation strategies for dextrous manipulation. We use an object-based approach and give an example application of the system to CAD-based 2-D vision.