To enable precise grasping and control in complex environments, robotic manipulation increasingly relies on the integration of multiple sensory modalities. This paper introduces an advanced multi-modal sensor that combines a refined matrix of optoelectronic elements with an inertial measurement unit in a compact design. The improved tactile resolution provides reliable force feedback, allowing for delicate and controlled handling of fragile or perishable objects. A novel approach has been proposed that utilizes the multimodal sensing capabilities of our sensor suite to integrate data from optoelectronic elements and an inertial measurement unit (IMU) to enhance robustness and precision of manipulation. The designed sensor integrated into a Robotiq 3-fingers adaptive gripper influences advanced sensor fusion techniques to optimize the performance of deployed Proportional-Integral (PI) control for accurate positioning of fingers, enabling precise grasping. Furthermore, the innovative contribution of our work is the enhancement of Human-Robot Interaction (HRI) capabilities, in which the gripper adeptly responds to external forces exerted by the human in either longitudinal or lateral directions, thereby facilitating the controlled release or holding of objects. Hence, this manner develops safety and collaboration in distributed workstations. The experimental results of our system validate its effectiveness in handling delicate objects, and the study aligns with the premise of human-friendly robots by prioritizing insightful interaction and adaptive control.

This paper proposes a robotic based system, equipped with suitably developed mechatronic tools, able to fine manipulate medical hoses directly from the production line and during the execution of quality check tasks. The system combines suitably developed technologies (i.e., tactile sensors and cutting machine) with methodological approaches (i.e., tactile indicator, blind search methods, image processing) in order to automatize the manipulation of very thin and soft linear deformable objects, where the main challenge is the very small forces involved during manipulation. The work presents the system testing during the execution of two real quality check scenarios: verification of the hose internal diameter and checking of the hose section shape and dimensions. For the internal diameter check, a suitably defined contact indicator based on tactile signals is proposed, and it is combined with a blind search method, based on the use of a spiral path, in order to generalize the task execution, by overcoming limitations due to the possible misalignment among grasped area and hose end. For the inspection of the hose section, a specific cutting tool has been developed and image processing methods have been implemented, by means of microscope feedback, for the centering and focus of the hose sample to be analyzed.

This paper proposes a novel method to refine the 6D pose estimation inferred by an instance-level deep neural network which processes a single RGB image and that has been trained on synthetic images only. The proposed optimization algorithm usefully exploits the depth measurement of a standard RGB-D camera to estimate the dimensions of the considered object, even though the network is trained on a single CAD model of the same object with given dimensions. The improved accuracy in the pose estimation allows a robot to grasp apples of various types and significantly different dimensions successfully; this was not possible using the standard pose estimation algorithm, except for the fruits with dimensions very close to those of the CAD drawing used in the training process. Grasping fresh fruits without damaging each item also demands a suitable grasp force control. A parallel gripper equipped with special force/tactile sensors is thus adopted to achieve safe grasps with the minimum force necessary to lift the fruits without any slippage and any deformation at the same time, with no knowledge of their weight.

Despite modern industries are becoming increasingly automated, wire harness manufacturing processes still rely on manual assembly. One of the reasons behind the difficulty in making the process automatic is that wire harnesses and cable assemblies are highly customized products depending on the application. Hence, realizing an industrial automatic machine for the production of a specific wire harness is not affordable, and manual production remains the most cost-effective. A step towards the automation of wire harness manufacturing is the realization of a system that can be easily adapted to produce different types of cable assemblies. This paper proposes a system composed of a robotic arm, a gripper, and sensorized fingers for executing a wire harness manipulation task. The system can be easily adapted to produce different products by updating the order of operations, the trajectories, and the dimensions/positioning of some low cost mechanical parts.

In this paper, a tactile-based robotic system to perform the insertion of a Deformable Linear Object (DLO) in a hole is proposed. This is a typical application in manufacturing processes involving assembly of electric cables in connectors or electromechanical components. A Recurrent Neural Network (RNN) with Long Short Term Memory (LSTM) cells is adopted in this work to predict the external forces acting on the DLO from the tactile data. The tactile sensor, mounted on the finger, provides 16 signals and it is the only sensor required during the effective insertion task. In a real environment, the tight spaces very often prevent the possibility to use the vision system, also when the same task is performed by a human being. Force/Torque sensors instead increase the system price and provide signals that might be affected by inertia disturbance or other undesired effect, that are difficult to manage. The control law design is based on the RNN outputs and the distance between end-effector and target hole. In particular, it leads the plastically deformed DLO inside the hole while it adjusts the tool pose guided by the control errors in order to prevent buckling. The contribute of this work is dual: first a tactile feedback integrating a RNN to estimate contact forces on the grasped object is presented; second, a control system is developed to perform a challenging insertion of a DLO in a hole. Experimental works are presented to validate the proposed algorithms.

This paper reports the development of a manipulation system for electric wires, implemented by means of a commercial gripper installed on an industrial manipulator and equipped with cameras and suitably designed tactile sensors. The purpose of this system is the execution of wire insertion on commercial electromechanical components. The synergy between computer vision and tactile sensing is necessary because, in a real environment, the tight spaces very often prevent the possibility to use the vision system, also when the same task is performed by a human being. A novel technique to speed up the generation of training data sets for convolutional neural networks (CNNs) is proposed. Therefore, this technique is used to train a CNN in order to detect small objects (such as wire terminals). Moreover, aiming to prevent faults during the task and to interact with the environment safely, several machine learning approaches are used to produce an affordable output from the tactile sensor. The proposed approach shows how a cheap sensor embedded with suitable intelligence can provide information comparable to a more expensive force sensor.

In this paper, we introduce the problem of cross-modal visuo-tactile object recognition with robotic active exploration. With this term, we mean that the robot observes a set of objects with visual perception, and later on, it is able to recognize such objects only with tactile exploration, without having touched any object before. Using a machine learning terminology, in our application, we have a visual training set and a tactile test set, or vice versa. To tackle this problem, we propose an approach constituted by four steps: finding a visuo-tactile common representation, defining a suitable set of features, transferring the features across the domains, and classifying the objects. We show the results of our approach using a set of 15 objects, collecting 40 visual examples and five tactile examples for each object. The proposed approach achieves an accuracy of 94.7%, which is comparable with the accuracy of the monomodal case, i.e., when using visual data both as training set and test set. Moreover, it performs well compared to the human ability, which we have roughly estimated carrying out an experiment with ten participants.

The work implements a new real-time flexible motion planning method used for reactive object manipulation in pick and place tasks typical of in-store logistics scenarios such as shelf replenishment of retail stores.This method uses a new hybrid pipeline to recognize and localize an object observed through a depth camera, by integrating and optimizing state of the art techniques.The proposed algorithm guarantees recognition robustness and localization accuracy.The desired object is then manipulated.The motion planner, based on the obstacles detected in the scene, plans a collision-free path towards the target pose.The planned trajectory optimizes a cost function that reflects the best solution among those available and produces natural and smooth path through a smart IK constrained solution which avoids robot unnecessary reconfigurations.A reactive control based on distributed proximity sensors is finally adopted to locally modify the planned trajectory in real time to avoid collisions with uncertain or dynamic obstacles.Experimental results in a supermarket scenario populated with cluttered obstacles demonstrate smoothness of the robot motions and reactive capabilities in a typical fetch and carry task.

In-hand manipulation is certainly one of the most challenging problems in robotic manipulation. Solutions to this problem depend on the specific device used to grab the object, but nowadays, the trend is to exploit not only the gripper but also external constraints, such as other objects in the environment or external forces, like gravity. This allows a robot to manipulate an object even with very simple grippers, like a parallel gripper. Nevertheless, even for a simple grasping task, which aims at grabbing the object with a given fixed orientation or for executing a controlled slip, information on the contact between the fingers of the gripper and the object is relevant. In these cases, both linear and rotational slipping should be controlled during the grasping phase and during the motion phase. The present paper proposes a control strategy for the first objective, namely slipping avoidance. The strategy is based on contact information provided by a six-axis force/tactile sensor, able to measure contact force and torque as well as able to provide information on the contact geometry, that means orientation of the object with respect to the gripper. Experiments on a parallel gripper sensorized with a new force/tactile sensor and mounted on a Kuka iiwa show how the strategy successfully allows the robot to safely manipulate a rigid object in various friction conditions of its surface.

In this work, we propose a framework to deal with cross-modal visuo-tactile object recognition. By cross-modal visuo-tactile object recognition, we mean that the object recognition algorithm is trained only with visual data and is able to recognize objects leveraging only tactile perception. The proposed cross-modal framework is constituted by three main elements. The first is a unified representation of visual and tactile data, which is suitable for cross-modal perception. The second is a set of features able to encode the chosen representation for classification applications. The third is a supervised learning algorithm, which takes advantage of the chosen descriptor. In order to show the results of our approach, we performed experiments with 15 objects common in domestic and industrial environments. Moreover, we compare the performance of the proposed framework with the performance of 10 humans in a simple cross-modal recognition task.

The main objective of this paper is to propose an experimental modal analysis procedure, based on the use of a gray-box model for flexible structures. The described approach presents interesting advantages with respect to commercial solutions: ease of use due to the low number of parameters to set for an identification session; no need for expert users, even in the presence of particular cases such as double modes, since it does not use a stabilization diagram to be elaborated; use of a gray-box model whose unknown parameters have a clear physical meaning. All these characteristics are discussed in the paper, and the performance of the proposed procedure has been evaluated by using experimental data available from a non-trivial standard benchmark. The results have been compared with those obtained by using a commercial tool.

This paper proposes an experimental study of slipping avoidance algorithms based on force/tactile perception data. The claim is that contact force measurements alone or tactile data alone are not sufficient for an effective slipping avoidance strategy in real world conditions. Integrated force/tactile sensors able to provide measurements of both the contact force vector and spatially distributed tactile maps are the key enabling technology for efficient slipping avoidance control algorithms that can actually work with real world objects under no restricting assumption on the contact geometry or with unknown physical properties of the objects. The paper proposes a new slipping avoidance control scheme, which usefully exploits an integrated force/tactile sensor mounted on the parallel gripper of a Kuka youBot. The results show how the strategy successfully allows the robot to safely manipulate real-world objects, both rigid and compliant, in various friction conditions of their surface, both stable and slippery.

The paper presents a novel slipping control algorithm based on the exploitation of the tactile sensor integrated into the DEXMART anthropomorphic robotic hand. The Extended Kalman Filter (EKF) is used to solve in real-time the nonlinear model of the sensor, allowing to estimate the contact geometry variables and the friction coefficient. The innovative proposed slipping control is based on the use of the estimation error of the EKF as an indicator of incipient slipping events. The control algorithm computes the suitable grip force based on the estimation error. The effectiveness of the proposed approach is shown with experimental results.

This paper presents the design of an optical position sensor integrated into a miniaturized tendon-driven robotic joint. The sensor exploits the modulation of the light power flux that goes from an infrared Light Emitting Diode (LED) to a PhotoDiode (PD) by means of a variable-thickness canal integrated into the joint itself to detect the joint position. The LED and the PD are fixed on one of the two links that compose the robotic joint, while the canal is integrated into the other link. The paper reports the basic sensor working principle, the integrated design of the miniaturized robotic joint with embedded position sensor and the experimental evaluation of the proposed device on a force/position control loop. Finally, a preliminary prototype of the UBH-IV finger in which the proposed joint with embedded position sensor are used is presented.

In this paper, an innovative sensor based on optoelectronic components and compliant frames for the measurement of the tendon tension is presented. With respect to conventional solutions for force sensing, like strain-gauge or Bragg-grating based force sensors, this sensor presents several advantages, mainly in terms of compactness, simplicity of the implementation and conditioning electronics. The proposed sensor exploits the properties of optoelectronic components with a narrow angle of view to measure the very small deformation of a compliant frame caused by the tendon tension. The sensor can be placed at the tendon ends as such as in any position along the tendon. The paper reports the basic working principle and a simplified procedure for the design of the sensor frame together with the results of an experimental testbench where a couple of the proposed sensors are use for the feedback control of a tendon-driven robotic joint.

The purpose of this paper is to present preliminary results on the use of a torque sensor based on a Bragg grating for torque control applications of tendon-driven mechanisms. Owing to the minimally invasive nature of optical fibres, one of the most promising applications can be the integration of the sensor into anthropomorphic robotic hands for accurate impedance or compliance control. In fact, the sensing element of the proposed torque sensor can be easily bonded directly to the tendon following its natural routing with a significantly reduced invasiveness with respect to conventional sensors. These are usually based on strain gauges, which are cumbersome and require additional mechanical components and interfaces in order to provide the necessary measurement, while the Bragg sensor is embedded into the optical fibre whose typical diameter is about 125 mum which allows its integration into the tendon itself. The experimental results presented here have been obtained on a simple test-bench realized by using off-the-shelf and cheap components conceived to demonstrate the potentiality of the sensor and its effectiveness in an actual compliance control scheme.