We create a virtual 2D tactile array for soft pneumatic actuators using embedded audio components. We detect contact-specific changes in sound modulation to infer tactile information. We evaluate different sound representations and learning methods to detect even small contact variations. We demonstrate the acoustic tactile sensor array by the example of a PneuFlex actuator and use a Braille display to individually control the contact of 29 x 4 pins with the actuator's 90 x 10 mm palmar surface. Evaluating the spatial resolution, the acoustic sensor localizes edges in x- and y-direction with a root-mean-square regression error of 1.67 mm and 0.0 mm, respectively. Even light contacts of a single Braille pin with a lifting force of 0.17 N are measured with high accuracy. Finally, we demonstrate the sensor's sensitivity to complex contact shapes by successfully reading the 26 letters of the Braille alphabet from a single display cell with a classification rate of 88 %.

We present an active acoustic sensor that turns soft pneumatic actuators into contact sensors. The whole surface of the actuator becomes a sensor, rendering the question of where best to place a contact sensor unnecessary. At the same time, the compliance of the soft actuator remains unaffected. A small, embedded speaker emits a frequency sweep which travels through the actuator before it is recorded with an embedded microphone. The specific contact state of the actuator affects how the sound is modulated while traversing the structure. We learn to recognize these changes in the sound and map them to the corresponding contact locations. We demonstrate the method on the PneuFlex actuator. The active acoustic sensor achieves a classification rate of 93% and mean regression error of 3.7mm. It is robust against background noises and different objects. Finally, we test it on a Panda robot arm and show that it is unaffected by motor noises and other active sensors.

The space of all possible deformations of soft robotic actuators is extremely large. It is impossible to explicitly measure each internal degree of freedom, regardless of the number and types of sensors. It is, however, possible to measure a smaller subset of task-relevant deformations using only a few well-placed sensors. But for a different task, the soft actuator's deformation behavior might differ significantly. Instead of finding a new sensor placement for the new task, which would result in a separate hand for every task, we propose a method that maintains the original sensors and uses prior knowledge about each task to extend the applicability of the existing sensorized actuators to new tasks. We demonstrate our approach by the example of a PneuFlex actuator of the RBO Hand 2. When sensorizing the actuator for a single task, the sensor model does not transfer well to other tasks. Using our multi-task method, we train new sensor models that use prior knowledge about the tasks. The new models improve measurement accuracy for the new tasks without having to change the sensor hardware.

We propose a novel sensing method for soft pneumatic actuators. The method uses a single microphone, embedded into the actuator's air chamber. Contact with the environment induces sound (vibration) in the actuator. The materials and the shape of the actuator reflect, refract, and attenuate the sound as it propagates inside the actuator. This produces a unique sound signature for different types of events, enabling the sensing of contact locations, contact force, and the type of contacted material. Sensing is insensitive to the inflation state of the actuator and to background noise. We demonstrate the robustness and versatility of the microphone-based sensor solution in experiments with a PneuFlex actuator. The proposed sensorization avoids the fundamental challenges of sensorizing soft pneumatic actuators, because the placement of a microphone does not negatively affect the compliance of the actuator and because a single microphone suffices for sensorization of the entire actuator, eliminating the need for an application-specific sensor layout.

The compliance of soft actuators makes manipulation safer and simplifies control. But their high flexibility also makes sensorization challenging. From the large space of possible deformations not all are equally important. We present a method for sensorization of soft actuators that, for a given application, finds an effective layout from a set of sensors. It starts from a redundant sensor layout and iteratively reduces the number of sensors. Applying the method to the PneuFlex actuators of the RBO Hand 2, we identify a layout of four liquid metal strain sensors and one pressure sensor to predict actuator deformation in three dimensions: flexional, lateral, and twist. Finally, the layout is used to build a sensorized RBO Hand 2. It can detect passive shape adaptation while grasping and reveals failure cases during manipulation, e.g. slipping fingers while opening a door.

We describe the winning entry to the Amazon Picking Challenge 2015. From the experience of building this system and competing, we derive several conclusions: (1) We suggest to characterize robotic system building along four key aspects, each of them spanning a spectrum of solutions - modularity vs. integration, generality vs. assumptions, computation vs. embodiment, and planning vs. feedback. (2) To understand which region of each spectrum most adequately addresses which robotic problem, we must explore the full spectrum of possible approaches. (3) For manipulation problems in unstructured environments, certain regions of each spectrum match the problem most adequately, and should be exploited further. This is supported by the fact that our solution deviated from the majority of the other challenge entries along each of the spectra. This is an abridged version of a conference publication.

To leverage soft hands to their full potential for grasping, we propose to design their morphology and control signals together. Considering both parameter domains makes it easier and faster to find solutions compared to fixing parameters of either domain. Additionally, the approach scales well to high-dimensional parameter spaces, which is a precondition to make automated co-design useful for soft hands. We further present an efficient simulator for simulating grasps with soft hands which is based on the SOFA framework and enables us to simulate more than a million grasps per day. These two complementary improvements promise a boost in the development of competent soft hands and their control in the future.

In many robotic applications, softness leads to improved performance, robustness, and safety, while lowering manufacturing cost, increasing versatility, and simplifying control. The advantages of soft robots derive from the fact that their behavior partially results from interactions of the robot's morphology with its environment, which is commonly referred to as morphological computation (MC). But not all MC is good in the sense that it supports the desired behavior. One of the challenges in soft robotics is to build systems that exploit the morphology (good MC) while avoiding body-environment interactions that are harmful with respect to the desired functionality (bad MC). Up to this point, constructing a competent soft robot design requires experience and intuition from the designer. This work is the first to propose a systematic approach that can be used in an automated design process. It is based on calculating a low-dimensional representation of an observed behavior, which can be used to distinguish between good and bad MC. We evaluate our method based on a set of grasping experiments, with variations in hand design, controller, and objects. Finally, we show that the information contained in the low-dimensional representation is comprehensive in the sense that it can be used to guide an automated design process.

Handshakes are common greetings, and humans therefore have strong priors of what a handshake should feel like. This makes it challenging to create compelling and realistic human-robot handshakes, necessitating the consideration of human haptic perception in the design of robot hands. At its most basic level, haptic perception is encoded by contact points and contact pressure distributions on the skin. This motivates our work on measuring the contact area and contact pressure in human handshaking interactions. We present two benchmarking experiments in this regard, measuring the contact locations in human-human/human-robot handshaking and the contact pressure distribution for handshakes with a sensorized palm. We present results from human studies with the benchmarking experiments, providing a baseline for comparison with robot hands as well as presenting new insights into human handshaking. We also show initial work in using these results for the evaluation of robot hands, and progressing towards iterative design of robot hands optimized for social hand interactions.

The ability to selectively stiffen otherwise compliant soft actuators increases their versatility and dexterity. We investigate granular jamming and layer jamming as two possible methods to achieve stiffening with PneuFlex actuators, a type of soft continuum actuator. The paper details five designs of jamming compartments that can be attached to an actuator. We evaluate the stiffening of the five different prototypes, achieving an up to 8-fold increase in stiffness. The strength of the most effective prototype based on layer jamming is also validated in the context of pushing buttons, resulting in an 2.23-fold increase in pushing force.