Achieving controlled sliding of objects on finger surfaces is a significant challenge for robots, substantially constraining their ability to perform complex in-hand manipulation tasks. In this work, we investigate the role of surface vibration in modulating the effective friction at the object-finger contact locations to facilitate controlled sliding. We demonstrate that friction at contact points can be reduced by applying targeted vibrations at specific locations on a robotic finger, creating regions that are suitable for sliding. In this way, we create sticking/sliding regions on finger surfaces on demand and can easily switch between sliding and rolling contacts. To investigate this phenomenon, we embedded an array of vibration modules into robotic fingers. We first analyzed the velocity fields created by surface vibrations on a single finger. Then, we developed a method to select the appropriate activation states of the modules that achieve the desired velocity field at a given object location. Utilizing these fingers and the vibration selection method, we formed a two-finger robotic hand and demonstrated controlled sliding and rotation of a held object within the hand. To the best of our knowledge, this is the first work that utilizes vibration-induced friction modulation for in-hand manipulation that can achieve combinations of object sliding and rolling actions.

Contact can be conceptualized as a set of constraints imposed on two bodies that are interacting with one another in some way. The nature of a contact, whether a point, line, or surface, dictates how these bodies are able to move with respect to one another given a force, and a set of contacts can provide either partial or full constraint on a body's motion. Decades of work have explored how to explicitly estimate the location of a contact and its dynamics, e.g., frictional properties, but investigated methods have been computationally expensive and there often exists significant uncertainty in the final calculation. This has affected further advancements in contact-rich tasks that are seemingly simple to humans, such as generalized peg-in-hole insertions. In this work, instead of explicitly estimating the individual contact dynamics between an object and its hole, we approach this problem by investigating compliance-enabled contact formations. More formally, contact formations are defined according to the constraints imposed on an object's available degrees-of-freedom. Rather than estimating individual contact positions, we abstract out this calculation to an implicit representation, allowing the robot to either acquire, maintain, or release constraints on the object during the insertion process, by monitoring forces enacted on the end effector through time. Using a compliant robot, our method is desirable in that we are able to complete industry-relevant insertion tasks of tolerances <0.25mm without prior knowledge of the exact hole location or its orientation. We showcase our method on more generalized insertion tasks, such as commercially available non-cylindrical objects and open world plug tasks.

Substantial advancements to model-based reinforcement learning algorithms have been impeded by the model-bias induced by the collected data, which generally hurts performance. Meanwhile, their inherent sample efficiency warrants utility for most robot applications, limiting potential damage to the robot and its environment during training. Inspired by information theoretic model predictive control and advances in deep reinforcement learning, we introduce Model Predictive Actor-Critic (MoPAC)†, a hybrid model-based/model-free method that combines model predictive rollouts with policy optimization as to mitigate model bias. MoPAC leverages optimal trajectories to guide policy learning, but explores via its model-free method, allowing the algorithm to learn more expressive dynamics models. This combination guarantees optimal skill learning up to an approximation error and reduces necessary physical interaction with the environment, making it suitable for real-robot training. We provide extensive results showcasing how our proposed method generally outperforms current state-of-the-art and conclude by evaluating MoPAC for learning on a physical robotic hand performing valve rotation and finger gaiting–a task that requires grasping, manipulation, and then regrasping of an object.

Learning accurate representations of robot models remains a challenging problem, and is typically approached though large, system-specific feature sets. This method inherently introduces practical shortcomings, as interpretability and transferability of the learned model typically decreases as more features are introduced into the learning framework in order to handle increasing task complexity. In this article, we examine the problem of developing transferable learned models for dexterous manipulation that are able to accurately predict the behavior of physically distinct systems without retraining. We introduce the notion of learning from visually-extracted grasp mechanics-based features, which are formulated by combining geometrically-inspired, analytical representations of the gripper into the feature set to more holistically represent the state of varied systems performing manipulation. We characterize the added utility of using such features through simulation and incorporate them into a classifier to predict specific phenomena, or modes of manipulation, that occur during prehensile within-hand movement. Four modes of manipulation—normal (rolling contact), drop, stuck, and sliding—are defined, collected physically, and trained via a self-supervised learning approach. The classifier is first trained on a single sensorless underactuated hand variant for all four modes. We, then, investigate the transferability of the learned classifier on five different planar gripper variants—analyzing applicability of this approach with both online and offline evaluation.

Grasping can be conceptualized as the ability of an end-effector to temporarily attach or fixture an object to a manipulator-constraining all motion of the workpiece with respect to the end-effector's base frame. This seemingly simplistic action often requires excessive sensing, computation, or control to achieve with multi-fingered hands, which can be mitigated with underactuated mechanisms. In this work, we present the analysis of radial graspers for automated part fixturing and grasping in the plane with a design implementation of a single-actuator, 8-finger gripper. By leveraging a passively adaptable mechanism that is under-constrained pre-contact, the gripper conforms to arbitrary object geometries and locks post-contact as to provide form closure around the object. We also justify that 8 radially symmetric fingers with passive locking are sufficient to create robust form closure grasps on arbitrary planar objects. The underlying mechanism of the gripper is described in detail, with analysis of its highly underactuated nature, and the resulting form closure ability. We show with a wide variety of objects that the gripper is able to acquire robust grasps on all of them, and maintain maximal quality form closure on most objects, with each finger exerting equal grasp force within ±2.48 N.

In this work, we present a within-hand manipulation approach that leverages a simple energy model based on caging grasps made by underactuated hands. Instead of explicitly modeling the contacts and dynamics in manipulation, we can calculate a map to describe the energy states of different hand-object configurations under an actuation input. Since the system intrinsically steers towards low energy states, the object's movement is uniquely described by the gradient of the energy map if the corresponding actuation is applied. Such maps are pre-calculated for a range of actuation inputs to represent the system's energy profile. We discretize the workspace into a grid and construct an energy gradient-based graph by locally exploring the gradients of the stored energy profile. Given a goal configuration of a simple cylindrical object, a sequence of actuation inputs can be calculated to manipulate it towards the goal by exploiting the connectivity in the graph. The proposed approach is experimentally implemented on a Yale T42 hand. Our evaluation results show that parts of the graph are well connected, explaining our ability to successfully plan and execute trajectories within the gripper's workspace.

In this work, we investigate a mechanics-inspired framework for describing fingertip-based planar within-hand manipulation with an underactuated robotic gripper. In particular, this framework leverages fundamental mechanics properties of the hand-object system, including basic terms such as local contact curvature as well as more complex features including the grasp matrix and manipulability metrics. These are extracted using a simple visual approach and then in real-time used for predicting planar manipulation modes: namely rolling, dropped, stuck, and sliding. Given a desired cartesian motion for the object, a supervised learning model predicts these four manipulation modes before they occur, allowing us to either avoid or trigger these different behaviors. Since we utilize strictly fundamental properties of the grasp matrix, finger Jacobians, and contact curvatures, we are able to demonstrate prediction transferability between different grippers using our original classifier. In particular, a Random Forests classifier trained on one gripper successfully predicts manipulation modes for grippers with different fingers with 84% accuracy, compared to just 56% from an approach in previous work. Overall, we find that the features designed in our approach better describes fingertip manipulation when precise gripper models are not available.

Current robotic terrain mapping techniques require expensive sensor suites to construct an environmental representation. In this work, we present a cube-shaped robot that can roll through unstructured terrain and construct a detailed topographic map of the surface that it traverses in real time with low computational and monetary expense. Our approach devolves many of the complexities of locomotion and mapping to passive mechanical features. Namely, rolling movement is achieved by sequentially inflating latex bladders that are located on four sides of the robot to destabilize and tip it. Sensing is achieved via arrays of fine plastic pins that passively conform to the geometry of underlying terrain, retracting into the cube. We developed a topography by shade algorithm to process images of the displaced pins to reconstruct terrain contours and elevation. We experimentally validated the efficacy of the proposed robot through object mapping and terrain locomotion tasks.

We address the problem of developing precision, quasi-static control strategies for fingertip manipulation in robot hands. In general, analytically specifying useful object transition maps, or hand-object Jacobians, for scenarios in which there is uncertainty in some key aspect of the hand-object system is difficult or impossible. This could be in scenarios with standard fully-actuated hands where, for instance, there is not an accurate model of the contact conditions, or in scenarios with fewer control inputs than mechanical degrees of freedom (such as underactuated hands or those that are controlled by synergies or impedance controlled frameworks), since the output space is of higher dimension than the input space. In this work, we develop a method for extracting object transition maps by tracking the state of the grasp frame. We begin by modeling a compliant, underactuated hand and its mechanical properties through an energy-based approach. From this energy model, we provide controlled actuation inputs to change the state of the grasp frame. We observe the response from these actions and develop a regression map of the action-reaction pairs, where the map is subject to our intent for grasp frame movement and the regional relationship between the contacts. Once the regression model is developed, we perform within-hand planning of the grasp frame with newly introduced objects. This approach is agnostic to the global geometry of the object and is able to adapt when undesirable contact conditions, such as sliding, occur. The learning-based methodology estimates the non-linearities representative in the properties of the system. We test our framework physically on an adapted Yale Openhand Model O. By transferring the learned model from simulation to the physical hand without adaptation, we show that this energy modeling approach is robust to inaccuracies in parameter estimation. We demonstrate its efficacy in a handwriting task.

In this work, we investigate methods to detect four phenomena (modes) that occur during prehensile fingertip-based within-hand manipulation without the use of tactile sensors. By using actuator states and visual data, we aim to recognize different modes of operation such as interpreting if the hand is about to drop the object, if the object will begin to slide on the fingers, or if the system is at or near a singularity. For this purpose, we utilize supervised learning techniques, which allow us to detect the modes without the use of a mechanical model of the system. We analyze the individual roles of specific features available through both the actuator and visual data, and identify the ones that have the most significance for detecting the operation modes. Our results show classification performance of 96% (using either Extra Trees, Gradient Boosting, or SVM) when using combined actuator and visual features. Interestingly, we were able to achieve a 94%classification rate using only actuator information, and 93 % using only visual information. Overall, the classifiers identified actuator positions, actuator loads, and commanded velocities as the most important features for detecting a mode. These results have implications for enabling the control of within-hand manipulation movements utilizing a minimal amount of sensory information without a model of the hand/object system.