In robot-assisted minimally invasive surgery (RMIS), the absence of haptic feedback presents a significant challenge for surgeons in accurately gauging the forces applied during procedures. However, obtaining precise force/torque (F/T) information at the surgical site is challenging. One key obstacle is distinguishing external forces from those induced by the cable-actuated kinematics of the surgical tool. We present a novel method to eliminate this interference by employing differential F/T measurement. We utilize two miniature 6-axis F/T sensors, positioned proximally and distally, to counterbalance the undesired forces and torques generated by the cable-driven system. To demonstrate the efficacy of this approach, we developed an experimental cable-actuated forceps with two degrees of freedom. We conducted a series of dynamic tests, attaching various weight configurations to the gripper to simulate external forces ranging from 0.5 N to 1.5 N. Subsequently, we evaluated three measurement methods: raw distal sensor readings, differential compensation, and a multilayer perceptron (MLP) that processes a sliding window of inputs from both sensors and actuators. Differential compensation improves performance by 70% over the distal sensor alone, achieving a root-mean-square error (RMSE) of 0.15 N and 3 mNm across the entire dataset. The MLP yields a further improvement of 90% lower RMSE relative to the distal sensor, achieving 0.05 N and 0.5 mNm on a test subset of the data not used for training.

The rising demand for customized products necessitates the integration of multiple robotic systems, underscoring the need for advanced production planning and scheduling. This paper introduces an ontology-based, artificial intelligence-enhanced method for dynamic task planning and scheduling, aimed at improving the efficiency of production process, reducing machine downtime, and consequently increasing throughput in assembly operations. Designed to generate and execute feasible production plans dynamically, this method minimizes manual planning and scheduling efforts. We evaluate its effectiveness using two gear assembly use cases with various robot skills, highlighting its flexibility in planning and scheduling and its contributions to the evolution of smart manufacturing. The method’s adaptability suggests its applicability across diverse smart factory environments.

Both robot-assisted surgery (RAS) and future manufacturing systems use telemanipulation setups to enable remote control by surgeons in the operating room and assembly technicians. Precision, intuitive handling, as well as short task execution times have the highest priority. In this paper, we present a novel force-sensing stick and velocity-based online motion generator for a robotic telemanipulator. This custom rigid stick with 6 degrees of freedom (DoF) force/torque (F/T) sensing capabilities is considered for telemanipulation. In a first study, 24 subjects perform three tasks which mimic relevant manipulation maneuvers for industrial assembly and RAS: (1) picking and placing objects, (2) inserting a peg into a hole, and (3) moving the end-effector precisely along a specific pathway. In a second study, three subjects repeat the tasks over a longer period of time in order to assess the learning characteristics and long-term effects on task performance and execution times. For comparison, the same tests are carried out using an off-the-shelf 3 DoF motion-based device. Our results show, that both devices lead to similar performance rates and task execution times over all trials. For the force-sensing stick, subjects report an intuitive and natural response to their force input with no perceptible latency. Conclusions of the long-term study are particularly interesting: within only a few days, task execution times for both input devices can be significantly reduced by 53-69%. The present study builds on previous works of the authors presented at the World Haptics Conference 2023 in Delft [1].

The automatic execution of contact-rich assembly tasks such as peg-in-hole insertion still remains a challenge in industrial manufacturing automation. Deep reinforcement learning (RL) enables agents to learn complex robotic skills, but requires extensive data collection and relatively long execution times when trained online on the physical hardware. In this paper, a robotic setup and RL implementation are presented, which can learn how to successfully perform the peg-in-hole insertion task. The state vector for our force-based learning approach only consists of force/torque (F/T) signals from the robot tooltip without using any position information. We introduce a deep Q-learning (DQN) framework adapted to the task at hand and gather a training data set with a total of 984 peg insertion attempts and 7,884 experiences on the physical setup. Based on this, we apply an offline learning process to improve efficiency by training a large number of policies with different parameter configurations in short time. Finally, the best model configurations are deployed and evaluated on the physical setup reaching a 100% success rate for the insertion task with 0.2 mm clearance.

We propose a framework for the measurement of static and dynamic physical properties of manipulation objects using both robotic tactile and kinesthetic sensing - in particular data from fingertip force/torque (F/T) and robot joint torque sensors. It completes the manipulation-relevant information about new objects that cannot be estimated from a passive camera observation. The system allows to balance the accuracy and complexity of the estimation system against the costs and complexity of the approach. We evaluate methods that allow improving robustness against noise and model errors in the manipulation system used for the estimation. The approach is validated on experimental results using data from a torque-controlled robot manipulator and precision F/T sensors.

For the manufacturing of miniature force/torque sensors, extreme accuracy is required due to the tiny size of the strain gauges inside the sensors (2×2.5 mm). The current method of manually assembling them by hand is difficult, time-intensive, and error-prone. To improve this, a system to pick up the tiny objects from a plate and place them on elementary cells is being devised using a 6-axis robot arm with custom end-effector and a camera with magnification lens. This paper focuses on the perception module by evaluating methods for detecting tiny and transparent objects and obtaining spatial information from 2D images. Additionally, it considers aspects of the camera-to-robot calibration process, which are necessary to transfer the accuracy of image recognition into the real world. An approach using image segmentation and blob detection is taken, precluding the need for machine learning models. This is possible due to the superb image quality achieved by the sufficiently advanced camera and lighting setup. As a conclusion, we propose a perception module, which is capable of pinpointing strain gauge positions within ±0.1 mm and can also recognize different types of components based on physical dimensions. Our end-to-end approach for automatic pick-and-place operations integrates the perception module, camera-to-robot calibration, and a last-minute correction routine, which ultimately leads to an overall positioning accuracy of ± 0.3 mm.

In minimally invasive surgery (MIS), the reliable detection of hard inclusions in soft tissue is crucial for the success of the intervention. In robot-assisted surgery (RAS) however, limited technologies are available for intracorporeal tissue stiffness assessment due to the lack of force and tactile feedback from the robot tool tip. This paper investigates both, human haptic perception and robotic F/T sensing in similar experimental setups to draw conclusions about the usage of a haptic sensor for teleoperation in RAS. We use a novel 6-axis F/T sensor compact enough to be moved through trocars during RAS interventions and experimentally analyze its performance in a simulated robotic palpation task. Furthermore, we carry out a comprehensive user study$(n=30)$and collect fingertip interaction data to investigate human haptic perception. Results show, that both approaches detect larger bead diameters of 19 mm and 15 mm with high precision and show similar accuracy rates. With regards to interaction forces, subjects on average apply more than 10 times the amount of normal force$(F_{z}=28.8\pm 4.9\mathrm{N})$, which leads to higher accuracy particularly for smaller embedded nodules. The robotic sensing technique, on the contrary, offers distinct advantages by providing more gentle treatments and reducing the risk of tissue damage.

The human sense of touch allows recognizing a wide set of properties of a grasped object such as weight, shape, hardness, temperature or surface texture. Despite the great importance of haptic sensing for humans, mechatronic end-effectors of humanoid robots and industrial manipulators are rarely endowed with tactile feedback. This is due to a lack of robust force/torque sensors which are compact enough to be integrated in the robot's fingertips. This paper leverages a novel 6-axis force/torque sensor and investigates, how local force/torque sensing at the end-effector fingertip best enables the robot to classify different surface textures. Fingertip measurements of reaction forces and torques are recorded for a total of 21 textures as the robot performs sliding movements similar to those that humans make when exploring textures. After data collection and signal processing, the extracted features are used for texture recognition, utilizing k-nearest neighbor (kNN), decision tree, random forest as well as multi-layer perceptron (MLP) classifiers. Our experimental results show that the concatenated power spectral densities extracted from the force and torque time series are the most discriminative input features enabling the random forest to achieve an average recognition accuracy of 98.8±0.4%.