In robot-assisted minimally invasive surgery (RMIS), the absence of haptic feedback presents a significant challenge for surgeons in accurately gauging the forces applied during procedures. However, obtaining precise force/torque (F/T) information at the surgical site is challenging. One key obstacle is distinguishing external forces from those induced by the cable-actuated kinematics of the surgical tool. We present a novel method to eliminate this interference by employing differential F/T measurement. We utilize two miniature 6-axis F/T sensors, positioned proximally and distally, to counterbalance the undesired forces and torques generated by the cable-driven system. To demonstrate the efficacy of this approach, we developed an experimental cable-actuated forceps with two degrees of freedom. We conducted a series of dynamic tests, attaching various weight configurations to the gripper to simulate external forces ranging from 0.5 N to 1.5 N. Subsequently, we evaluated three measurement methods: raw distal sensor readings, differential compensation, and a multilayer perceptron (MLP) that processes a sliding window of inputs from both sensors and actuators. Differential compensation improves performance by 70% over the distal sensor alone, achieving a root-mean-square error (RMSE) of 0.15 N and 3 mNm across the entire dataset. The MLP yields a further improvement of 90% lower RMSE relative to the distal sensor, achieving 0.05 N and 0.5 mNm on a test subset of the data not used for training.

Both robot-assisted surgery (RAS) and future manufacturing systems use telemanipulation setups to enable remote control by surgeons in the operating room and assembly technicians. Precision, intuitive handling, as well as short task execution times have the highest priority. In this paper, we present a novel force-sensing stick and velocity-based online motion generator for a robotic telemanipulator. This custom rigid stick with 6 degrees of freedom (DoF) force/torque (F/T) sensing capabilities is considered for telemanipulation. In a first study, 24 subjects perform three tasks which mimic relevant manipulation maneuvers for industrial assembly and RAS: (1) picking and placing objects, (2) inserting a peg into a hole, and (3) moving the end-effector precisely along a specific pathway. In a second study, three subjects repeat the tasks over a longer period of time in order to assess the learning characteristics and long-term effects on task performance and execution times. For comparison, the same tests are carried out using an off-the-shelf 3 DoF motion-based device. Our results show, that both devices lead to similar performance rates and task execution times over all trials. For the force-sensing stick, subjects report an intuitive and natural response to their force input with no perceptible latency. Conclusions of the long-term study are particularly interesting: within only a few days, task execution times for both input devices can be significantly reduced by 53-69%. The present study builds on previous works of the authors presented at the World Haptics Conference 2023 in Delft [1].

The automatic execution of contact-rich assembly tasks such as peg-in-hole insertion still remains a challenge in industrial manufacturing automation. Deep reinforcement learning (RL) enables agents to learn complex robotic skills, but requires extensive data collection and relatively long execution times when trained online on the physical hardware. In this paper, a robotic setup and RL implementation are presented, which can learn how to successfully perform the peg-in-hole insertion task. The state vector for our force-based learning approach only consists of force/torque (F/T) signals from the robot tooltip without using any position information. We introduce a deep Q-learning (DQN) framework adapted to the task at hand and gather a training data set with a total of 984 peg insertion attempts and 7,884 experiences on the physical setup. Based on this, we apply an offline learning process to improve efficiency by training a large number of policies with different parameter configurations in short time. Finally, the best model configurations are deployed and evaluated on the physical setup reaching a 100% success rate for the insertion task with 0.2 mm clearance.

We propose a framework for the measurement of static and dynamic physical properties of manipulation objects using both robotic tactile and kinesthetic sensing - in particular data from fingertip force/torque (F/T) and robot joint torque sensors. It completes the manipulation-relevant information about new objects that cannot be estimated from a passive camera observation. The system allows to balance the accuracy and complexity of the estimation system against the costs and complexity of the approach. We evaluate methods that allow improving robustness against noise and model errors in the manipulation system used for the estimation. The approach is validated on experimental results using data from a torque-controlled robot manipulator and precision F/T sensors.

For the manufacturing of miniature force/torque sensors, extreme accuracy is required due to the tiny size of the strain gauges inside the sensors (2×2.5 mm). The current method of manually assembling them by hand is difficult, time-intensive, and error-prone. To improve this, a system to pick up the tiny objects from a plate and place them on elementary cells is being devised using a 6-axis robot arm with custom end-effector and a camera with magnification lens. This paper focuses on the perception module by evaluating methods for detecting tiny and transparent objects and obtaining spatial information from 2D images. Additionally, it considers aspects of the camera-to-robot calibration process, which are necessary to transfer the accuracy of image recognition into the real world. An approach using image segmentation and blob detection is taken, precluding the need for machine learning models. This is possible due to the superb image quality achieved by the sufficiently advanced camera and lighting setup. As a conclusion, we propose a perception module, which is capable of pinpointing strain gauge positions within ±0.1 mm and can also recognize different types of components based on physical dimensions. Our end-to-end approach for automatic pick-and-place operations integrates the perception module, camera-to-robot calibration, and a last-minute correction routine, which ultimately leads to an overall positioning accuracy of ± 0.3 mm.

In minimally invasive surgery (MIS), the reliable detection of hard inclusions in soft tissue is crucial for the success of the intervention. In robot-assisted surgery (RAS) however, limited technologies are available for intracorporeal tissue stiffness assessment due to the lack of force and tactile feedback from the robot tool tip. This paper investigates both, human haptic perception and robotic F/T sensing in similar experimental setups to draw conclusions about the usage of a haptic sensor for teleoperation in RAS. We use a novel 6-axis F/T sensor compact enough to be moved through trocars during RAS interventions and experimentally analyze its performance in a simulated robotic palpation task. Furthermore, we carry out a comprehensive user study$(n=30)$and collect fingertip interaction data to investigate human haptic perception. Results show, that both approaches detect larger bead diameters of 19 mm and 15 mm with high precision and show similar accuracy rates. With regards to interaction forces, subjects on average apply more than 10 times the amount of normal force$(F_{z}=28.8\pm 4.9\mathrm{N})$, which leads to higher accuracy particularly for smaller embedded nodules. The robotic sensing technique, on the contrary, offers distinct advantages by providing more gentle treatments and reducing the risk of tissue damage.

The human sense of touch allows recognizing a wide set of properties of a grasped object such as weight, shape, hardness, temperature or surface texture. Despite the great importance of haptic sensing for humans, mechatronic end-effectors of humanoid robots and industrial manipulators are rarely endowed with tactile feedback. This is due to a lack of robust force/torque sensors which are compact enough to be integrated in the robot's fingertips. This paper leverages a novel 6-axis force/torque sensor and investigates, how local force/torque sensing at the end-effector fingertip best enables the robot to classify different surface textures. Fingertip measurements of reaction forces and torques are recorded for a total of 21 textures as the robot performs sliding movements similar to those that humans make when exploring textures. After data collection and signal processing, the extracted features are used for texture recognition, utilizing k-nearest neighbor (kNN), decision tree, random forest as well as multi-layer perceptron (MLP) classifiers. Our experimental results show that the concatenated power spectral densities extracted from the force and torque time series are the most discriminative input features enabling the random forest to achieve an average recognition accuracy of 98.8±0.4%.

Haptic feedback for minimally invasive surgery is a widely debated topic, especially force sensors for this kind of application are thoroughly researched. The current state-of-the-art includes several realization options, but none of them are suitable to be integrated in small diameter instruments. In this work, we present a new approach for the generation of haptic feedback by using two separate feedback mechanisms for spatial kinesthetic and unidirectional tactile feedback in a teleoperation system with a parallel kinematic slave robot and the design of the corresponding user interfaces. Kinesthetic forces are measured with proximal sensors in the robot kinematic and displayed with a delta-type kinematic mechanism. Tactile signals are acquired with an acceleration sensor placed closely at the tip of the robot manipulator and conveyed open-loop by a vibrating cylinder to the user's palm. Kinesthetic forces up to 15 N can be displayed up to a frequency of 50 Hz. The tactile interface displays accelerations up to 15m/s for frequencies from 50 Hz to 1000 Hz. The combination of the two interfaces allows the display of high-fidelity haptic feedback with greatly reduced requirements on the interface components.

Single port surgery is an innovative approach in the field of minimally invasive surgery. Although several telemanipulators exist to perform operations through only a single incision they all suffer from the lack of haptic feedback. To generate kinesthetic feedback the intracorporeal forces need to be measured, which is a challenging task. To overcome these limitations we investigate in this paper the force sensing capability of a new single port robot that has two parallelkinematic manipulators. Because of the rigidity and excellent controllability the tip force is measured using proximally arranged force sensors. To minimize friction effects a longitudinal vibration of 6 Hz with an amplitude of 20 µm is applied to the manipulator while the tip force is varied in the three main coordinate axes. Within the analysis 32320 values are investigated showing that a vibration has a significant impact and dramatically improves the signal quality by minimizing the crosstalk by a factor of up to 12.7 enabling to measure the tip force exactly.

Pseudo haptic sensation is an illusion based on visual stimuli. In virtual environments the principle of pseudo haptic feedback is used to simulate material properties such as stiffness, mass and friction. Transfering the principle of pseudo haptic feedback to real haptic teleoperation systems can provide a haptic sensation for properties of material, manipulated without active haptic feedback. Hence, the implementation of pseudo haptic feedback to teleoperation systems reduces the requirements for kinematic structures and actuators of the used haptic user interface .In this work, we discuss the usability of pseudo haptic feedback for its application in teleoperation systems. The mechanisms of pseudo-haptic feedback are theoretically explained for an exemplary teleoperation task. Options to vary the intensity of the users pseudo haptic sensation are derived. The effect of pseudo haptic feedback for teleoperation is based on changing the input-output ratio by actual measured interaction forces at the end-effector. A one degree of freedom teleoperation system is used for first experiments with pseudo haptic feedback in teleoperation. The used teleoperation system couples users input force to a defined displacement of an end-effector. Our results prove the validity of pseudo-haptic feedback in teleoperation by a compliance discrimination experiment.

In this paper we present a new single port robotic system particularly designed for single port access surgery in the field of colon cancer. In contrast to most state of the art systems the two intracorporeal manipulator arms are set up using a parallel kinematic structure with 5DOF, which are integrated into a shaft of 38mm in diameter. The parallel kinematic structure and its workspace are discussed in detail. As a unique feature, the kinematics uses the translation and rotation of three rods per manipulation arm. Using a parallel kinematic approach enables dynamic, stiff and precise movements which are needed to generate a realistic haptic feedback. The results of several performed circle tests are presented showing the ability of performing precise and dynamic movements with speeds of up to 327 mm/s and under load condition of 4 N.