Advancing robotic grasping and manipulation requires the ability to test algorithms and/or train learning models on large numbers of grasps. Towards the goal of more advanced grasping, we present the Grasp Reset Mechanism (GRM), a fully automated apparatus for conducting large-scale grasping trials. The GRM automates the process of resetting a grasping environment, repeatably placing an object in a fixed location and controllable 1-D orientation. It also collects data and swaps between multiple objects enabling robust dataset collection with no human intervention. We also present a standardized state machine interface for control, which allows for integration of most manipulators with minimal effort. In addition to the physical design and corresponding software, we include a dataset of 1,020 grasps. The grasps were created with a Kinova Gen3 robot arm and Robotiq 2F-85 Adaptive Gripper to enable training of learning models and to demonstrate the capabilities of the GRM. The dataset includes ranges of grasps conducted across four objects and a variety of orientations. Manipulator states, object pose, video, and grasp success data are provided for every trial.

Robotic in-hand manipulation increases the capability of robotic hands to interact with the world. The amount of manipulation that a robot is capable of is highly dependent on the design of the robot hand, and previous works have shown success in designing hands to improve performance for different types of grasping and manipulation. In this paper we present a method for designing a fully-actuated planar manipulator that optimizes for specific in-hand motions. We demonstrate that, with the Asterisk Benchmark and a light-weight IK controller, we can translate our results from simulation to the real world with minimal effort and high-fidelity. Using the simulated data (over 4,000 simulated hand-designs) we begin to analyze which features contribute to improved planar manipulation.

We present a method for classifying the quality of near-contact grasps using spatial metrics that are recoverable from sensor data. Current methods often rely on calculating precise contact points, which are difficult to calculate in real life, or on tactile sensors or image data, which may be unavailable for some applications. Our method, in contrast, uses a mix of spatial metrics that do not depend on the fingers being in contact with the object, such as the object’s approximate size and location. The grasp quality can be calculated before the fingers actually contact the object, enabling near-grasp quality prediction. Using a random forest classifier, the resulting system is able to predict grasp quality with 96% accuracy using spatial metrics based on the locations of the robot palm, fingers and object. Furthermore, it can maintain an accuracy of 90% when exposed to 10% noise across all its inputs.

In this paper we define two feature representations for grasping. These representations capture hand-object geometric relationships at the near-contact stage - before the fingers close around the object. Their benefits are: 1) They are stable under noise in both joint and pose variation. 2) They are largely hand and object agnostic, enabling direct comparison across different hand morphologies. 3) Their format makes them suitable for direct application of machine learning techniques developed for images. We validate the representations by: 1) Demonstrating that they can accurately predict the distribution of ε-metric values generated by kinematic noise. I.e., they capture much of the information inherent in contact points and force vectors without the corresponding instabilities. 2) Training a binary grasp success classifier on a real-world data set consisting of 588 grasps.

Grasping a simple object from the side is easy - unless the object is almost as big as the hand or space constraints require positioning the robot hand awkwardly with respect to the object. We show that humans - when faced with this challenge - adopt coordinated finger movements which enable them to successfully grasp objects even from these awkward poses. We also show that it is relatively straight forward to implement these strategies autonomously. Our human-studies approach asks participants to perform grasping task by either “puppetteering” a robotic manipulator that is identical (geometrically and kinematically) to a popular underactuated robotic manipulator (the Barrett hand), or using sliders to control the original Barrett hand. Unlike previous studies, this enables us to directly capture and compare human manipulation strategies with robotic ones. Our observation is that, while humans employ underactuation, how they use it is fundamentally different (and more effective) than that found in existing hardware.

This paper presents an online data collection method that captures human intuition about what grasp types are preferred for different fundamental object shapes and sizes. Survey questions are based on an adopted taxonomy that combines grasp pre-shape, approach, wrist orientation, object shape, orientation and size which covers a large swathe of common grasps. For example, the survey identifies at what object height or width dimension (normalized by robot hand size) the human prefers to use a two finger precision grasp versus a three-finger power grasp. This information is represented as a confidence-interval based polytope in the object shape space. The result is a database that can be used to quickly find potential pre-grasps that are likely to work, given an estimate of the object shape and size.

We present a human-subjects study approach that supports the analysis of the manipulation performance of robotic hands that have the same morphology but different actuation and compliance. Specifically, we use this approach to analyze three different types of hands (one underactuated, one fully actuated, one fully actuated with compliant distal joints) as they are used to perform two manipulation tasks. The first task uses a power grasp (spraying with a spray bottle), the second a precision grasp (tracing a line on a bowl with a pen). We show that compliance in the distal joints significantly improves performance and task completion. We also show that humans choose significantly different poses for the same task when using a fully-actuated versus underactuated hand, which also results in superior task performance. Our results suggest that humans use a combination of under-actuated and fully-actuated techniques, which when used on robotic systems would also improve their performance on manipulation tasks.

Perception and gaze are an integral part of determining where and how to grasp an object. In this study we analyze how gaze patterns differ when participants are asked to manipulate a robotic hand to perform a grasping task when compared with using their own. We have three findings. First, while gaze patterns for the object are similar in both conditions, participants spent substantially more time gazing at the robotic hand then their own, particularly the wrist and finger positions. Second, We provide evidence that for complex objects (eg, a toy airplane) participants essentially treated the object as a collection of sub-objects. Third, we performed a follow-up study that shows that choosing camera angles that clearly display the features participants spend time gazing at are more effective for determining the effectiveness of a grasp from images. Our findings are relevant both for automated algorithms (where visual cues are important for analyzing objects for potential grasps) and for designing tele-operation interfaces (how best to present the visual data to the remote operator).

We analyze visual cues people used to evaluate a robot grasp. Participants were presented with two (front and side) orthogonal views of a robot hand grasping an object and asked how successful the grasp would be on a scale of 1-5; they were eye-tracked while completing this survey. Ground truth of the success of the grasps is known. Our primary observations were that (1) Most of the failed grasp predictions were false positives, and this was exacerbated for grasps that were ranked as human-like. (2) Two visual cues from human-grasp research (object center-line and top) were used, but not contact points. Instead, participants gazed at robot finger, wrist, and arm locations. (3) There was a difference in the visual patterns between the left and right images, indicating that the second image was primarily used to verify the locations of fingers and wrist while the first was used to establish the object's location and shape. Finally, we generate transition matrices to model the temporal aspect of the gaze patterns.

As an alternative to the laborious process of collecting training data from physical robotic platforms for learning robotic grasp quality prediction, we explore the use of surrogate training data from crowd-sourced evaluations of images of robotic grasps. We show that in certain regions of the grasp feature space, grasp predictors trained with this surrogate data were almost as accurate as predictors built using data from physical testing with robots.

With the goal of advancing the state of automatic robotic grasping, we present a novel approach that combines machine learning techniques and rigorous validation on a physical robotic platform in order to develop an algorithm that predicts the quality of a robotic grasp before execution. After collecting a large grasp sample set (522 grasps), we first conduct a thorough statistical analysis of the ability of grasp metrics that are commonly used in the robotics literature to discriminate between good and bad grasps. We then apply Principal Component Analysis and Gaussian Process algorithms on the discriminative grasp metrics to build a classifier that predicts grasp quality. The key findings are as follows: (i) several of the grasp metrics in the literature are weak predictors of grasp quality when implemented on a physical robotic platform; (ii) the Gaussian Process-based classifier significantly improves grasp prediction techniques by providing an absolute grasp quality prediction score from combining multiple grasp metrics. Specifically, the GP classifier showed a 66% percent improvement in the True Positive classification rate at a low False Positive rate of 5% when compared with classification based on thresholding of individual grasp metrics.

This paper investigates the possibility of increasing the robustness of underactuated grasping through the use of variable transmission ratios. We propose a 4-step procedure to investigate and improve the robustness of an underactuated finger on a fixed object. This procedure maximizes the robustness against random force disturbances to the maximum obtainable value under given circumstances. A simulation study is presented that analyzes the disturbance robustness, followed by an experimental study to confirm the effect. The variable transmission ratio is a promising means to increase grasp robustness and has great application potential.

We present a novel and simple experimental method called physical human interactive guidance to study human-planned grasping. Instead of studying how the human uses his/her own biological hand or how a human teleoperates a robot hand in a grasping task, the method involves a human interacting physically with a robot arm and hand, carefully moving and guiding the robot into the grasping pose, while the robot's configuration is recorded. Analysis of the grasps from this simple method has produced two interesting results. First, the grasps produced by this method perform better than grasps generated through a state-of-the-art automated grasp planner. Second, this method when combined with a detailed statistical analysis using a variety of grasp measures (physics-based heuristics considered critical for a good grasp) offered insights into how the human grasping method is similar or different from automated grasping synthesis techniques. Specifically, data from the physical human interactive guidance method showed that the human-planned grasping method provides grasps that are similar to grasps from a state-of-the-art automated grasp planner, but differed in one key aspect. The robot wrists were aligned with the object's principal axes in the human-planned grasps (termed low skewness in this paper), while the automated grasps used arbitrary wrist orientation. Preliminary tests show that grasps with low skewness were significantly more robust than grasps with high skewness (77-93%). We conclude with a detailed discussion of how the physical human interactive guidance method relates to existing methods to extract the human principles for physical interaction.

We propose a novel approach to study the ability of an underactuated mechanism, or a mechanism that has fewer actuators than degrees of freedom, to passively adapt to environmental constraints. While prior work in underactuated robotic hands has primarily focused on the mechanism's ability to curl its distal degrees of freedom inward even after the proximal degrees of freedom are constrained by contact with the environment, this paper explores the mechanism's adaptability in terms of both motion and force-application capabilities in the presence of external constraints. Specifically, using four different transmissions for a novel singly-actuated linear three degree-of-freedom mechanism, this paper analyzes how the system's ability to reconfigure joints and apply new contact forces varies as a function of the transmission configuration and object geometry. We show that with more extensive re routing of a single actuator to multiple joints, the mechanism exhibits greater motion and force adaptability at the cost of decreased maximum joint travel and contact forces.

The compliance of an underactuated robotic hand, or a robotic hand with fewer actuators than degrees of freedom, is a function of the mechanism type, the design parameters, and the operational control mode. The transmissions used in underactuated mechanisms can be divided into two main classes based on the self adaptive transmission used to route actuation to the various degrees of freedom, namely the single-acting transmission and the double-acting transmission. While both transmission classes can be represented using a kinematic constraint equation that defines the relationship between actuator and joint motion, the main difference between the two transmission classes is that the kinematic constraint is always active in double-acting mechanisms while there are specific combinations of external disturbances and mechanism parmeters that render the constraint inactive in single-acting mechanisms. While previous studies have only explored the performance of underactuated mechanisms with the constraint always active, this paper identifies the benefits for robotic grasping (such as better disturbance rejection) that arise when the constraint becomes inactive in single-acting mechanisms.

While underactuated mechanisms have become popular in robot-hand designs because of their passive adaptability, existing systems utilize only one actuator to produce motion in the multiple degrees of freedom in the serial chain of each finger. In this paper, we explore how the performance of an underactuated serial link chain changes as more actuators are added. The fundamental question of what extra capability an additional actuator provides to an underactuated system and how best to implement it has not yet been quantified in the literature. Using a simple linear underactuated mechanism, we show that the performance of a single-actuator system (measured as the average number of contacts made with the environment) quickly plateaus as the number of degrees of freedom of the mechanism is increased. Also, we show that as the number of actuators is increased, the system's passive adaptability improves as the mechanism implementation spreads the actuators across the joints.

Humans are adept at grasping different objects robustly for different tasks. Robotic grasping has made significant progress, but still has not reached the level of robustness or versatility shown by human grasping. It would be useful to understand what parameters (called grasp measures) humans optimize as they grasp objects, how these grasp measures are varied for different tasks, and whether they can be applied to physical robots to improve their robustness and versatility. This paper demonstrates a new way to gather human-guided grasp measures from a human interacting haptically with a robotic arm and hand. The results revealed that a human-guided strategy provided grasps with higher robustness on a physical robot even under a vigorous shaking test (91%) when compared with a state-of-the-art automated grasp synthesis algorithm (77%). Furthermore, orthogonality of wrist orientation was identified as a key human-guided grasp measure, and using it along with an automated grasp synthesis algorithm improved the automated algorithm's results dramatically (77% to 93%).

With the goal of developing human-like dextrous manipulation, we investigate how the central nervous system uses the redundant control space of the human hand to perform tasks with force-stiffness requirements. Specifically, while the human hand is actuated by several muscles with varying mechanical advantage (called the moment arm), it is unclear how each muscle is used. Using the anatomically correct testbed (ACT) robotic hand to compute the control solution space and human-subject experiments with surface electromyography to measure biological control strategy, we identified that there is significant redundancy in the control spaces of both muscles with large moment arms and muscles with small moment arms. However, the central nervous system was selective about the solution for muscles with large moment arms, while it chose to span large regions of the available control space for muscles with small moment arms. Furthermore, the biological solution used low-moment-arm muscles at relatively high actuation levels. We summarize by making inferences on why the central nervous system chooses such a strategy and how this can help robotic manipulation.

With the goal of developing biologically inspired manipulation strategies for an anthropomorphic hand, we investigated how the human central nervous system utilizes the hands redundant neuromusculoskeletal biomechanics to transition between conditions. Using a experiment protocol where subjects were asked to transit between control states with equal end-effector force but different stiffness requirements, we observed that (1) some subjects used the same muscle synergy for both conditions by maintaining the same synergy throughout the transition, and (2) other subjects used two different muscle synergies to execute two conditions by transiting from one synergy to another rapidly. We hypothesize that humans typically try to use the same muscle synergy to execute two tasks when it is possible to optimize on the simplicity and speed over energy. This is a different control strategy from the way robots have been controlled in the past, and it provides a new direction in controlling an anthropomorphic robotic hand.

Multi-path routing (MPR) is an effective strategy to achieve robustness, load balancing, congestion reduction, and increased throughput by transmitting data over multiple paths. Disjoint multi-path routing (DMPR) requires the multiple paths to be link- or node-disjoint. Implementation of both MPR and DMPR poses significant challenges in obtaining loop-free multiple (disjoint) paths and effectively forwarding the data over the multiple paths, the latter being significant in data-gram networks. In this paper, we develop a disjoint multipath routing strategy using colored trees with an objective to minimize the total cost of the routing paths in a network. Two trees, namely red and blue, rooted at a given drain is formed. We demonstrate through extensive simulations that the developed technique is extremely effective in optimizing the average cost of the paths. In addition, we also observe that the developed approach minimizes the average minimum (minimum of the two paths) cost, which is lower than that obtained by earlier algorithms. The colored tree approach simply doubles the size of the routing table when two link- or node-disjoint paths to a specific node is needed.

Motivated by an error-recovery locomotion problem, we propose a control technique for a complex mechanical system by decomposing the system dynamics into a collection of simplified models. The robot considered, The Rocking and Rolling Robot (RRRobot), is a high-centered round-bodied robot that locomotes on a plane by swinging its legs and rocking on its shell. We identify the elements contributing to locomotion through two steps: 1) decoupling the leg-body rotation dynamics from the body-plane contact kinematics, and 2) decoupling the body rotational dynamics into dynamics along each rotational axis. We show, using simulation, that such decoupling provides a good approximation to RRRobot's locomotion and use these models to find an approximate control solution for RRRobot: a mapping between planar translation and leg motions

We study the problem of coverage of a given area for a maximum duration using a set of battery-operated sensors. Each sensor has a fixed sensing range and a limited lifetime due to the finite battery capacity. Sensors can be activated and deactivated at any time. The goal of this paper is to a find a schedule, determining when to activate and deactivate each sensor, to maximize the time for which every point in the area is covered by at least one sensor. We present several algorithms for this problem and show experimental and theoretical evidences to their efficiency. We also present an algorithm for a new model of coverage, called weak coverage, that does not require each point of the region to be covered all times, as long as the regions that are not covered are small

We show through experiment and simulation that a high-centered round-bodied legged robot can locomote by generating out-of-phase motions of reaction masses attached to its legs. These leg motions create body attitude oscillations which, when coupled with the slip-free contact constraints, locomote the robot. By varying the mean position of the leg oscillations, the robot can move in different directions in the plane. We also present some simplified models, where body attitude dynamics and contact kinematics are decoupled, to explain this form of legless locomotion.

Motivated by finding locomotion primitives for a legged robot, we present controllability results and a technique for kinematic reduction for a variable inertia mechanical system. We demonstrate configuration controllability for the system under consideration and use the symmetry resulting from angular momentum conservation to develop a kinematic representation of the mechanical system. We also show through simulation how plans for the kinematic representation can be implemented on the full dynamical mechanical system. It is hoped that this technique leads us to a general procedure for solving the gait synthesis problem.

We propose a locomotion technique for a legged robot that is high-centered, i.e., a robot stuck on a block with its legs dangling in air. By using its legs as reaction masses, the robot might be able to rock and roll on its stomach and incrementally move forward off the block, a form of legless locomotion using halteres. With locomotion of high-centered robots using body attitude oscillations as motivation, this paper focuses on studying the interplay between leg motions and body roll-pitch-yaw dynamics. We present results from simulation of two simplified models in which body motion is restricted to the roll and roll-yaw space respectively.