In this work, we propose a new estimation method for second-order kinematics for floating-base robots, based on highly redundant distributed inertial feedback. The linear acceleration of each robot link is measured at multiple points using a multimodal, self-configuring and self-calibrating artificial skin. The proposed algorithm is two-fold: i) the skin acceleration data is fused at the link level for state dimensionality reduction; ii) the estimated values are then fused limb-wise with data from the joint encoders and the main inertial measurement unit (IMU), using a Sigma-point Kalman filter. In this manner, it is possible to estimate the joint velocities and accelerations while avoiding the lag and noise amplification phenomena associated with conventional numerical derivation approaches. Experiments performed on the right arm and torso of a REEM-C humanoid robot, demonstrate the consistency of the proposed estimation method.

Multiple approaches to the estimation of high-order motion derivatives for innovative control applications now rely on the data collected by redundant arrays of inertial sensors mounted on robots, with promising results. However, most of these works suffer scalability issues induced by the considerable amount of data generated by such large-scale distributed sensor systems. In this article, we propose a new adaptive sensor-selection algorithm, for distributed inertial measurements. Our approach consists in using the data of a subset of sensors, selected among a larger collection of inertial sensing elements covering a rigid robot link. The sensor selection process is formulated as an optimization problem, and solved using a projected gradient heuristics. The proposed method can run online on a robot and be used to recalculate the selected sensor arrangement on the fly when physical interaction or potential sensor failure is detected. The tests performed on a simulated UR5 industrial manipulator covered with a multimodal artificial skin, demonstrate the consistency and performance of the proposed sensor-selection algorithm.

Humanoid robots are expected to interact in human environments, where physical interactions are unavoidable. Therefore, whole-body control methods that include multi-contact interactions are required. The new emerging technologies in touch sensing are fundamental to acquire online and rich information about these physical interactions with the environment. These technologies lead to the design of novel control systems that can profit from the tactile sensor information in an efficient form, thus producing reactive and compliant robots capable of interacting with their environment. In this paper, we present a novel control framework to integrate the multi-modal tactile information of a robot skin with different control strategies, producing dynamic behaviours suitable for Human-Robot Interactions (HRI). The control framework was experimentally evaluated on a full-size humanoid robot covered with more than 1260 skin cells distributed in the whole robot body. The results show that multi-modal tactile information can be fused hierarchically with multiple control strategies, producing active compliance in a position-controlled stiff humanoid robot.

This article presents a holistic approach to the engineering of an artificial robot skin for robots. An example of a multimodal skin cell is given, one that supports multiple human-like sensing modalities, and support for skin cell network is also provided; this is essential to form large-area skin patches in order to cover the surfaces of robots. The essential elements of efficiently handling a large amount of tactile data are explained. A general control framework, which supports robots commanded in position, velocity, and torque, is provided and validated. Several applications of this robot skin will be presented, demonstrating the effectiveness and efficiency of our artificial robot skin to support a wide number of robotic platforms as well as its ease of use across different domains.

In this paper, we propose a control method, providing mobile robots with whole-body compliance capabilities, in response to multicontact physical interactions with their environment. The external forces applied to the robot, as well as their localization on its kinematic tree, are measured using a multimodal, self-configuring, and self-calibrating artificial skin. We formulate a compliance control law in Cartesian space, as a set of quadratic optimization problems, solved in parallel for each limb involved in the interaction process. This specific formulation makes it possible to determine the torque commands required to generate the desired reactive behaviors, while taking the robot kinematic and dynamic constraints into account. When a given limb fails to produce the desired compliant behavior, the generalized force residual at the considered contact points is propagated to a parent limb in order to be adequately compensated. Hence, the robot's compliance range can be extended in a both robust and easily adjustable manner. The experiments performed on a dual-arm velocity-controlled mobile manipulator show that our methodology is robust to nullspace interactions and robot physical constraints.

Artificial robotic skins are continuously in contact with their environment, and therefore highly rely on proper connections in their skin cells' network. With a static network protocol approach, the affected skin area is unusable after a connection failure. Therefore, we developed a dynamic network protocol for large-scale artificial robotic skins, which re-routes the network upon connection failures to keep the whole skin in operation. Furthermore, the protocol balances the load for driving larger skins without packet loss. For verification, we validated the protocol on a large artificial robot skin we have developed and analyzed its performance with a skin network consisting of up to 204 cells. The failure recovery of the protocol converges in at most 50ms. We showed that the balancing method achieves a packet loss reduction of over 30% compared to the previously used protocol.

The realization of a kinesthetic robot behavior using robot skin requires a reactive skin torque controller, which fuses skin information and robot information to an appropriate skin joint torque in real-time. This fusion of information in real-time is challenging when deploying large scale skin. In this paper, we present a system which efficiently computes the torque of distributed skin cells locally at the point of contacts, completely removing this complex computations from the real-time loop. We demonstrate the feasibility of realizing the skin joint torque computations on the local micro-controllers of the skin cells. Conducting experiments with a real robot, we compare the accuracy of the distributed skin joint torque computation with the computation on the control PC. We also show that the novel distributed approach completely eliminates the computational delay of computing skin joint torques in the robot's real-time control loop. As a result, this approach removes any limits for the maximum number of skin cells in control.

The automation of production lines in industrial scenarios implies solving different problems, such as the flexibility to deploy robotic solutions to different production lines, usability to allow non-robotics expert users to teach robots different tasks, and safety to enable operators to physically interact with robots without the need of fences. In this paper, we present a system that integrates three novel technologies to address the above mentioned problems. We use an autocalibrated multi-modal robot skin , a general robot control framework to generate dynamic behaviors fusing multiple sensor signals, and an intuitive and fast teaching by demonstration method based on semantic reasoning. We validate the proposed technologies with a wheeled humanoid robot in an industrial set-up. The benefits of our system are the transferability of the learned tasks to different robots, the reusability of the models when new objects are introduced in the production line, the capability of detecting and recovering from errors, and the reliable detection of collisions and pre-collisions to provide a fast reactive robot that improves the physical human-robot interaction.

In this work we present a novel efficient event-driven reactive skin controller for large scale robot skin. The novel event-driven controller derives from a standard Jacobian torque controller and fully takes advantage of our multi-modal event-driven robot skin. Event-driven systems only sample, transmit and process information when the novelty of the information is guaranteed. This increases their efficiency in comparison to synchronous systems. We also use the new event-driven controller formulation to design a new synchronous reactive skin controller. We compare both controllers in a comprehensive performance evaluation with our robot TOMM. TOMM has two UR5 robot arms, each covered with 253 multi-modal skin cells. Each skin cell samples 4 different modalities and supports data mode and event mode. The results show that the event-driven reactive skin controller always outperforms the synchronous reference controller while both controllers show exactly the same response. When the robot is not moving then the event-driven controller reduces the CPU usage by 78% in comparison to the synchronous reference controller. When the robot is responding to contacts then the CPU usage reduces by 66%.

In this paper, we present the mechatronic design of our Tactile Omnidirectional Robot Manipulator (TOMM), which is a dual arm wheeled humanoid robot with 6DoF on each arm, 4 omnidirectional wheels and 2 switchable end-effectors (1 DoF grippers and 12 DoF Hands). The main feature of TOMM is its arms and hands which are covered with robot skin. We exploit the multi-modal tactile information of our robot skin to provide a rich tactile interaction system for robots. In particular, for the robot TOMM, we provide a general control framework, capable of modifying the dynamic behavior of the entire robot, e.g., producing compliance in a non-compliant system. We present the hardware, software and middleware components of the robot and provide a compendium of the base technologies deployed in it. Furthermore, we show some applications and results that we have obtained using this robot.

The development of breakthrough technologies helps the deployment of robotic systems in the industry. The implementation and integration of such technologies will improve productivity, flexibility and competitiveness, in diverse industrial settings specially for small and medium enterprises. In this paper we present a framework that integrates three novel technologies, namely safe robot arms with multi-modal and auto-calibrated sensing skin, a robot control framework to generate dynamic behaviors fusing multiple sensor signals, and an intuitive and fast teaching by demonstration method that segments and recognizes the robot activities on-line based on re-usable semantic descriptions. In order to validate our framework, these technologies are integrated in a industrial setting to sort and pack fruits. We demonstrate that our presented framework enables a standard industrial robotic system to be flexible, modular and adaptable to different production requirements.

In this paper we describe how we realized event-based signaling for large scale artificial robotic skin. We developed a new algorithm for the event generation on multi-modal skin cells. The skin cells have two modes, the conventional data sampling mode and the event mode. A comprehensive performance evaluation and comparison of these two modes is presented. We perform different experiments on our robot TOMM which has two UR5 robot arms, each covered with 260 multi-modal skin cells. Each skin cell samples 9 signals of 4 different modalities. Finally we derive models for extrapolating CPU usage and network traffic for larger numbers of skin cells and higher sample rates. The results show that the event-based system has superior performance and its performance edge increases with larger numbers of skin cells and higher sample rates. Experimental validation on our real robot system shows that in reactive control the event-based system reduces in comparison to the conventional system the packet rate by 48.2% and the CPU usage by 17.79%. We extrapolate the worst case for 5000 cells and show that the event-based system can at least reduce the packet rate by 21.2% and the CPU usage by 17.46%.

In this paper we propose event-based signaling for large-scale artificial robotic skin to reduce bandwidth requirements on data transmission and processing power. We use the send-on-delta principle to trigger the event generation only when tactile sensors are stimulated and transduce novel information. To compare the standard non-event based method with the proposed event-based method we present a comprehensive analysis of large-scale artificial skin systems for different test applications. For this purpose we collect data of 260 CellulARSkin cells on an UR-5 arm and calculate the events off-line. We determine the optimal packet size for event-based signaling and we show that the event-based system reduces the data rate with respect to the non-event based system for an unstimulated skin cell network to 16.45% and for a heavily stimulated skin cell network to 47.69%. The obtained results show that the event-based system reduces the data redundancy and the required transmission rates without loosing information.