Endovascular interventions require precise, cooperative control of multiple instruments, such as guidewires and catheters, to navigate complex vascular anatomies. Current robotic systems, reliant on leader-follower control, depend heavily on operator expertise and lack intelligence. Learning-based methods, often limited to single-instrument control, fall short in complex clinical scenarios requiring multi-instrument coordination. This study proposes a Multi-Agent Fuzzy Reinforcement Learning (MAFRL) framework, guided by large language models (LLMs), for task-level autonomous, cooperative navigation in endovascular robotics. LLMs provide procedural priors and context-aware policy guidance, enabling adaptive decision-making for collaborative guidewire and catheter agents. Central to the framework, fuzzy reinforcement learning mitigates LLM-induced uncertainties by adaptively embedding clinical constraints into reward functions, ensuring strict adherence to procedural safety and precise alignment with the complexities of real-world endovascular interventions. Validated in a 3D vascular simulation, this approach achieves superior navigation performance and procedural efficiency compared to conventional methods, underscoring the transformative potential of fuzzy reinforcement learning in advancing LLM-guided MARL for endovascular robotics.

Endowing the curved surfaces of rounded vision-based tactile fingers is essential for dexterous robotic manipulation, as they offer more sufficient contact with the environment. However, current rounded designs are constrained by a low sensing frequency (30–60 Hz) and the need for recalibration when adapting to new sensors due to the reliance on multi-channel captures, which hinders their performance in dynamic robotic tasks and large-scale deployment. In this work, we introduce R-Tac0, a low-cost rounded VBTS engineered for high-resolution and high-speed perception. The key innovation is a monochrome vision-based sensing principle: utilizing a black-and-white camera to capture the reflection properties of the compound rounded elastomer under monochromatic illumination. This single-channel imaging significantly reduces data volume and simplifies computational complexity, enabling 120 Hz tactile perception. A lightweight neural network can calibrate the sensor to achieve a depth reconstruction accuracy of 0.169 mm per pixel, while exhibiting surprisingly good transferability to new sensors. In experiments, we demonstrate the advantages of R-Tac0’s rounded design by evaluating its performance under different contact angles, its high-frequency perception in slip detection, and its effectiveness in robotic dynamic pose estimation.

THE sense of touch is an indispensable requirement for humans to effectively interact with the physical world around them and perform dexterous tasks. Similarly, this should be no different for robots. Imagine, for example, a robot that can open a bottle of medicine and dispense pills to an elderly person. Although this might seem a straightforward task for a human, it remains a significant challenge for a robot. Critically, the completion of the task depends on tactile sensing: the robot needs to receive and interpret the feedback from interacting with the bottle, determine the appropriate force based on the size and hardness of the pills, and adjust its pose to safely dispense them. Each step involves contact-rich interactions that can only be effectively deciphered through tactile sensing. Typically, tactile sensing works in conjunction with other modalities, such as vision, enabling the robot to adjust its actions dynamically and complete the task. In response to this vision of robots interacting with the physical world through touch, tactile robotics has now emerged as a key research area. Tactile robots can be defined as intelligent systems equipped with tactile sensors that can extract and process tactile data to guide their operations and interactions. The development of tactile robots presents scientific challenges, ranging from the design and fabrication of tactile sensors to methodologies for processing tactile data, integrating tactile feedback into task execution, and combining it with other sensory modalities to improve robot perception. As a result, tactile robotics demands collaborative efforts across several disciplines, involving material and data scientists …

Integrating robots into human-centric environments such as homes, necessitates advanced manipulation skills as robotic devices will need to engage with articulated objects such as doors and drawers. Key challenges in robotic manipulation of articulated objects are the unpredictability and diversity of these objects' internal structures, which render models based on object kinematics priors, both explicit and implicit, and inadequate. Their reliability is significantly diminished by pre-interaction ambiguities, imperfect structural parameters, encounters with unknown objects, and unforeseen disturbances. Here, we present aprior-freestrategy, Tac-Man, focusing on maintaining stable robot-object contact during manipulation. Without relying on object priors, Tac-Man leverages tactile feedback to enable robots to proficiently handle a variety of articulated objects, including those with complex joints, even when influenced by unexpected disturbances. Demonstrated in both real-world experiments and extensive simulations, it consistently achieves near-perfect success in dynamic and varied settings, outperforming existing methods. Our results indicate that tactile sensing alone suffices for managing diverse articulated objects, offering greater robustness and generalization than prior-based approaches. This underscores the importance of detailed contact modeling in complex manipulation tasks, especially with articulated objects. Advancements in tactile-informed approaches significantly expand the scope of robotic applications in human-centric environments, particularly where accurate models are difficult to obtain.

Soft growing vine robots show great potential for navigation and decontamination tasks in the nuclear industry. This paper introduces a novel hybrid continuum-eversion robot designed to address certain challenges in relation to navigating and operating within pipe networks and enclosed remote vessels. The hybrid robot combines the flexibility of a soft eversion robot with the precision of a continuum robot at its tip, allowing for controlled steering and movement in hard to access and/or complex environments. The design enables the delivery of sensors, liquids, and aerosols to remote areas, supporting remote decontamination activities.This paper outlines the design and construction of the robot and the methods by which it achieves selective steering. We also include a comprehensive review of current related work in eversion robotics, as well as other steering devices and actuators currently under research, which underpin this novel active steering approach. This is followed by an experimental evaluation that demonstrates the robot’s real-world capabilities in delivering liquids and aerosols to remote locations. The experiments reveal successful outcomes, with over 95% success in precision spraying tests. The paper concludes by discussing future work alongside limitations in the current design, ultimately showcasing its potential as a solution for remote decontamination operations in the nuclear industry.

Stretchable optical waveguides have been explored as a route to enhancing the sensing capabilities of soft actuators. Certain properties and qualities they possess recommend them for this task – their biological plausibility, compliance, low power consumption, and heightened responsiveness to external stimuli. Though well regarded for their efficiency, their practical application warrants a more detailed examination as regards their sensitivity, robustness, and resilience when integrated with various manipulators. There is a dearth of comprehensive, wide-ranging studies that investigate the relationship between soft sensors and actuators, both in terms of integration and sensor performance – the present study endeavours to fill this void. Here we present a series of findings as to the interdependent relationship at the nexus of soft actuator sensorisation, sensitivity and responsiveness. Building on our previous work and prior waveguide designs, we examine the influence of sensor location and placement along the deformation axis on responsiveness, repeatability, and longevity. Location is key as, during bending, one side experiences tension, while the other compression. Placement is identified as ’straight’ or ’loose’. Three PneuNet-based actuators were used in three design configurations: one without any additional modifications, one with a few rigid exoskeleton reinforcements, and one covered fully with rigid exoskeleton reinforcements. The purpose of applying the exoskeletons is to hold the waveguide-based sensors and to suppress any bubble formation. Each design enables the straightforward integration of sensors, so that the relationship between soft actuator design and sensor performance is easy to assess when applying various pressure intakes (from 0 to 7.1 psi) to actuate the bending motion.

Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations has shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel supervised learning framework that addresses long-horizon dexterous manipulation tasks using primitive skills. DexSkills is trained to recognize and replicate a select set of skills using human demonstration data, which can then segment a demonstrated long-horizon dexterous manipulation task into a sequence of primitive skills to achieve one-shot execution by the robot directly. Significantly, DexSkills operates solely on proprioceptive and tactile data, i.e., haptic data. Our real-world robotic experiments show that DexSkills can accurately segment skills, thereby enabling autonomous robot execution of a diverse range of tasks.

Robotic grasping in cluttered and real-world human environments is a challenging task. It requires unique kinematic capabilities to deal with spatial constraints as well as compliance and softness to offer collision safety and safe manipulation of sensitive objects. To address this challenge, we propose a novel robotic gripper with two steerable fingers whose lengths can be adjusted by way of a soft eversion mechanism. This enables the gripper to work in confined spaces while interacting safely with the environment. We also developed a new Electroadhesion (EA) pad design with a multilayer structure and a single insulating layer that can be safely integrated with the evertable fingers avoiding short-circuiting or dielectric breakdown to enhance the gripper payload. The resulting gripper can retrieve an object in a confined space and partially occluded by a barrier. It exhibits remarkable versatility in terms of object sizes, grasping objects with varying widths at least ranging from 70 mm to 600 mm. These results provide a promising avenue for new robotic applications in real-world environments.

Vision-based Tactile Sensors (VBTSs) show significant promise in that they can leverage image measurements to provide high-spatial-resolution human-like performance. However, current VBTS designs, typically confined to the fingertips of robotic grippers, prove somewhat inadequate, as many grasping and manipulation tasks require multiple contact points with the object. With an end goal of enabling large-scale, multi-surface tactile sensing via VBTSs, our research (i) develops a synchronized image acquisition system with minimal latency, (ii) proposes a modularized VBTS design for easy integration into finger phalanges, and (iii) devises a zero-shot calibration approach to improve data efficiency in the simultaneous calibration of multiple VBTSs. In validating the system within a miniature 3-fingered robotic gripper equipped with 7 VBTSs we demonstrate improved tactile perception performance by covering the contact surfaces of both gripper fingers and palm. Additionally, we show that our VBTS design can be seamlessly integrated into various end-effector morphologies significantly reducing the data requirements for calibration.

In conjunction with huge recent progress in cam-era and computer vision technology, camera-based sensors have increasingly shown considerable promise in relation to tactile sensing. In comparison to competing technologies (be they resistive, capacitive or magnetic based), they offer super-high-resolution, while suffering from fewer wiring problems. The human tactile system is composed of various types of mechanoreceptors, each able to perceive and process distinct information such as force, pressure, texture, etc. Camera-based tactile sensors such as GelSight mainly focus on high-resolution geometric sensing on a flat surface, and their force measurement capabilities are limited by the hysteresis and non-linearity of the silicone material. In this paper, we present a miniaturised dome-shaped camera-based tactile sensor that allows accurate force and tactile sensing in a single coherent system. The key novelty of the sensor design is as follows. First, we demonstrate how to build a smooth silicone hemispheric sensing medium with uniform markers on its curved surface. Second, we enhance the illumination of the rounded silicone with diffused LEDs. Third, we construct a force-sensitive mechanical structure in a compact form factor with usage of springs to accurately perceive forces. Our multi-modal sensor is able to acquire tactile information from multi-axis forces, local force distribution, and contact geometry, all in real-time. We apply an end-to-end deep learning method to process all the information.

Distributed sensor arrays capable of detecting multiple spatially distributed stimuli are considered an important element in the realisation of exteroceptive and proprioceptive soft robots. This paper expands upon the previously presented idea of decoupling the measurements of pressure and location of a local indentation from global deformation, using the overall stretch experienced by a soft capacitive e-skin. We employed machine learning methods to decouple and predict these highly coupled deformation stimuli, collecting data from a soft sensor e-skin which was then fed to a machine learning system comprising of linear regressor, gaussian process regressor, SVM and random forest classifier for stretch, force, detection and localisation respectively. We also studied how the localisation and forces are affected when two forces are applied simultaneously. Soft sensor arrays aided by appropriately chosen machine learning techniques can pave the way to e-skins capable of deciphering multi-modal stimuli in soft robots.

One of the biggest problems with soft robots is precisely the fact that they are soft. Indeed the softer they are, the less force they can exert on the environment. Researchers have proposed a number of stiffening methods, but all of them have drawbacks, such as locking the shape of the device in a way that precludes further adjustments. In this paper we propose a stiffening method inspired by the internal structure of the mammalian penis. The soft actuation chamber is divided into small compartments that trap the actuation fluid, leading to locally amplified pressure increase under certain conditions. At the same time, the proposed solution does not affect the actuation mechanism, allowing the actuator to be adjusted in one direction just as if it was in non-stiffened mode, while offering a stiff response in the opposite direction. Our prototype achieves an increase in stiffening of approximately a factor of two. The paper describes the concept, the mathematical justification of the working principle, the prototype design, its implementation and our experimental results.

Soft grippers have a number of advantages over their conventional stiff-bodied counterparts; not only do they surpass them in ease of fabrication and safety but also, in many cases, require less complex control strategies - due to natural compliance and form-fitting plasticity. Pneumatically actuated soft grippers made from non-extensible fabrics or polyethylene sheets have been shown to outperform soft silicone grippers capable of applying greater forces to the environment. Despite progress in the field, grasping a given object within a confined space proves challenging for both soft and conventional robotic grippers. A key issue is that most grippers use rotary or lateral translational motion when grasping an object, hence other objects in the scene may impede the closing motion as the gripper attempts to reach the target. In this study, we present a novel design for a soft robotic gripper equipped with a brace of fabric-based fingers capable, by way of eversion, of longitudinal extension, bending, and retraction, i.e. returning to a stowed state. Our experiments show that from the retracted to fully-extended state, the gripper fingers extend by up to 200% in length. To test the performance of the design, force characterisation experiments and grasping operations were carried out, demonstrating that each finger is capable of as much as 30 N of maximum tip force and can bend to an angle of 127°• At a bending pressure of 82.7 kPa (the maximum tested pressure in the bending chamber), a maximum (pullout) force of 16 N is needed to release an object that has been grasped by the finger. The experimental scenario detailed features all three mechanisms (eversion, bending and retraction) discussed.

Growing robots based on the eversion principle are known for their ability to extend rapidly, from within, along their longitudinal axis, and, in doing so, reach deep into hitherto inaccessible, remote spaces. Despite many advantages, vine robots also present significant challenges, one of which is maintaining sensory payload at the tip without restricting the eversion process. A variety of tip mechanisms have been proposed by the robotics community, among them rounded caps of relatively complex construction that are not always compatible with functional hardware, such as sensors or navigation pouches, integrated with the main eversion structure. Moreover, many tip designs incorporate rigid materials, reducing the robot's flexibility and consequent ability to navigate through narrow openings. Here, we address these shortcomings and propose a design to overcome them: a soft, entirely fabric based, cylindrical cap that can easily be slipped onto the tip of vine robots. Having created a series of caps of different sizes and materials, an experimental study was conducted to evaluate our new design in terms of four key aspects: vine robot made from multiple layers of everting material, solid objects protruding from the vine robot, squeezability, and navigability. In all scenarios, we can show that our soft, flexible cap is robust in its ability to maintain its position and is capable of transporting payloads such as a camera across long distances. We also demonstrate that the robot's ability to move through restricted aperture openings and indeed its overall flexibility is virtually unhindered by the addition of our cap. The paper discusses the advantages of this design and gives further recommendations in relation to aspects of its engineering.

This work presents a method for tactile classification of materials for virtual reality (VR) based robot teleoperation. In our system, a human-operator uses a remotely controlled robot-manipulator with an optical fibre-based tactile and proximity sensor to scan surfaces of objects in a remote environment. Tactile and proximity data and the robot's end-effector state feedback are used for the classification of objects' materials which are then visualized in the VR reconstruction of the remote environment for each object. Machine learning techniques such as random forest, convolutional neural and multi-modal convolutional neural networks were used for material classification. The proposed system and methods were tested with five different materials and classification accuracy of 90 % and more was achieved. The results of material classification were successfully exploited for visualising the remote scene in the VR interface to provide more information to the human-operator.

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry’s appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery.

Despite their soft and compliant bodies, most of today’s soft robots have limitations when it comes to elongation or extension of their main structure. In contrast to this, a new type of soft robot called the eversion robot can grow longitudinally, exploiting the principle of eversion. Eversion robots can squeeze through narrow openings, giving the possibility to access places that are inaccessible by conventional robots. The main drawback of these types of robots is their limited bending capability due to the tendency to move along a straight line. In this paper, we propose a novel way to fuse bending actuation with the robot’s structure. We devise an eversion robot whose body forms both the central chamber that acts as the backbone as well as the actuators that cause bending and manoeuvre the manipulator. The proposed technique shows a significantly improved bending capability compared to externally attaching actuators to an eversion robot showing a 133% improvement in bending angle. Due to the increased manoeuvrability, the proposed solution is a step towards the employment of eversion robots in remote and difficult-to-access environments.

We explored rate mode control for virtual reality (VR) based robot teleoperation with constant and variable mapping of the human-operator’s joystick position to the speed (rate) of the robot’s end-effector. The variable mapping depended on the visual scale of the virtual reconstruction of the remote environment to the scale of the real remote environment. We demonstrated how the rate mode control and variable scaling based on the VR reconstruction scale can be efficiently used for seated VR based robot teleoperation when the operator’s arms are supported to reduce tiredness. The experimental study with five human participants demonstrated that variable mapping allowed participants to teleoperate the robot more effectively, by adjusting the VR visual scale albeit at a cost of increased perceived workload.

Remote characterisation of the environment during physical robot-environment interaction is an important task commonly accomplished in telerobotics. This paper demonstrates how tactile and proximity sensing can be efficiently used to perform automatic crack detection. A custom-designed integrated tactile and proximity sensor is implemented. It measures the deformation of its body when interacting with the physical environment and distance to the environment's objects with the help of fibre optics. This sensor was used to slide across different surfaces and the data recorded during the experiments was used to detect and classify cracks, bumps and undulations. The proposed method uses machine learning techniques (mean absolute value as feature and random forest as classifier) to detect cracks and determine their width. An average crack detection accuracy of 86.46% and width classification accuracy of 57.30% is achieved. Kruskal-Wallis results (p<; 0.001) indicate statistically significant differences among results obtained when analysing only force data, only proximity data and both force and proximity data. In contrast to previous techniques, which mainly rely on visual modality, the proposed approach based on optical fibres is suitable for operation in extreme environments, such as nuclear facilities in which nuclear radiation may damage the electronic components of video cameras.

In the last decade, soft robots have been at the forefront of a robotic revolution. Due to the flexibility of the soft materials employed, soft robots are equipped with a capability to execute new tasks in new application areas -beyond what can be achieved using classical rigid-link robots. Despite these promising properties, many soft robots nowadays lack the capability to exert sufficient force to perform various real-life tasks. This has led to the development of stiffness-controllable inflatable robots instilled with the ability to modify their stiffness during motion. This new capability, however, poses an even greater challenge for robot control. In this paper, we propose a model-based kinematic control strategy to guide the tip of an inflatable robot arm in its environment. The bending of the robot is modelled using an Euler-Bernoulli beam theory which takes into account the variation of the robot's structural stiffness. The parameters of the model are estimated online using an observer based on the Extended Kalman Filter (EKF). The parameters' estimates are used to approximate the Jacobian matrix online and used to control the robot's tip considering also variations in the robot's stiffness. Simulation results and experiments using a fabric-based planar 3-degree-of-freedom (DOF) inflatable manipulators demonstrate the promising performance of the proposed control algorithm.

Thin and imperceptible soft skins that can detect internal deformations as well as external forces, can go a long way to address perception and control challenges in soft robots. However, decoupling proprioceptive and exteroceptive stimuli is a challenging task. In this paper, we present a silicone-based, capacitive E-skin for exteroception and proprioception (SCEEP). This soft and stretchable sensor can perceive stretch as along with touch at 100 different points via its 100 tactels. In this paper, we present a novel algorithm that decouples global strain from local indentations due to external forces. The soft skin is 10.1cm in length and 10cm in width and can be used to accurately measure the global strain of up to 25% with an error of under 3%; while at the same time, can determine the amplitude and position of local indentations. This is a step towards a fully soft electronic skin that can act as a proprioceptive sensor to measure internal states while measuring external forces.

This paper presents a novel approach for developing robotic grippers with variable stiffness hinges for dexterous grasps. This approach for the first time uses pneumatically actuated pouch actuators to fold and unfold morphable flaps of flexure hinges thus change stiffness of the hinge. By varying the air pressure in pouch actuators, the flexure hinge morphs into a beam with various open sections while the flaps bend, enabling stiffness variation of the flexure hinge. This design allows 3D printing of the flexure hinge using printable soft filaments. Utilizing the variable stiffness flexure hinges as the joints of robotic fingers, a light-weight and low-cost two-fingered tendon driven robotic gripper is developed. The stiffness variation caused due to the shape morphing of flexure hinges is studied by conducting static tests on fabricated hinges with different flap angles and on a flexure hinge with flaps that are bent by pouch actuators subjected to various pressures. Multiple grasp modes of the two-fingered gripper are demonstrated by grasping objects with various geometric shapes. The gripper is then integrated with a robot manipulator in a teleoperation setup for conducting a pick-and-place operation in a confined environment.

This work describes a virtual reality (VR) based robot teleoperation framework which relies on scene visualization from depth cameras and implements human-robot and human-scene interaction gestures. We suggest that mounting a camera on a slave robot's end-effector (an in-hand camera) allows the operator to achieve better visualization of the remote scene and improve task performance. We compared experimentally the operator's ability to understand the remote environment in different visualization modes: single external static camera, in-hand camera, in-hand and external static camera, in-hand camera with OctoMap occupancy mapping. The latter option provided the operator with a better understanding of the remote environment whilst requiring relatively small communication bandwidth. Consequently, we propose suitable grasping methods compatible with the VR based teleoperation with the in-hand camera. Video demonstration: https://youtu.be/3vZaEykMS_E.

In this paper, a magnetic-field-inspired robot navigation is used to navigate an under-actuated quad-copter towards the desired position amidst previously-unknown arbitrary-shaped convex obstacles. Taking inspiration from the phenomena of magnetic field interaction with charged particles observed in nature, the algorithm outperforms previous reactive navigation algorithms for flying robots found in the literature as it is able to reactively generate motion commands relying only on a local sensory information without prior knowledge of the obstacles' shape or location and without getting trapped in local minima configurations. The application of the algorithm in a dynamic model of quadcopter system and in the realistic model of the commercial AscTec Pelican micro-aerial vehicle confirm the superior performance of the algorithm.

This paper describes a novel approach in human-robot interaction driven by ergonomics. With a clear focus on optimising ergonomics, the approach proposed here continuously observes a human user’s posture and by invoking appropriate cooperative robot movements, the user’s posture is, whenever required, brought back to an ergonomic optimum. Effectively, the new protocol optimises the human-robot relative position and orientation as a function of human ergonomics. An RGB-D camera is used to calculate and monitor human joint angles in real-time and to determine the current ergonomics state. A total of 6 main causes of low ergonomic states are identified, leading to 6 universal robot responses to allow the human to return to an optimal ergonomics state. The algorithmic framework identifies these 6 causes and controls the cooperating robot to always adapt the environment (e.g. change the pose of the workpiece) in a way that is ergonomically most comfortable for the interacting user. Hence, human-robot interaction is continuously re-evaluated optimizing ergonomics states. The approach is validated through an experimental study, based on established ergonomic methods and their adaptation for real-time application. The study confirms improved ergonomics using the new approach.

In this paper, we present a reactive robot navigation method for a non-holonomic mobile robot taking inspiration from the phenomena observed in magnetic fields. The algorithm is shown to be able to guide mobile robots in arbitrary-shaped convex environment without being trapped in local minima by exploiting the local sensory information without priori knowledge about the environment. A preliminary validation study involving simulation of and experiments with a TurtleBot mobile robot platform show the advantage of the proposed method over existing ones.

Many modern roboticists take inspiration from biology to create novel robotic structures, including those that are modeled after the octopus. This paper advances this trend by creating soft robots modeling the complex motion patterns of octopus tentacles employing a bio-mimetic approach. The proposed octopus robot is entirely made from soft material and uses a novel fluidic actuation mechanism that allows the robot to advance forward, change directions and rotate around its primary axis. The paper presents the robot's design and fabrication process. An experimental study is conducted showing the feasibility of the proposed robot and actuation mechanism.

Taking inspiration from the properties of magnetic fields, we propose a reactive navigation method for soft continuum manipulators operating in unknown environments. The proposed navigation method outperforms previous works since it is able to successfully achieve collision-free movements towards the goal in environments with convex obstacles without relying on a priori information of the obstacles' shapes and locations. Simulations for the kinematic model of a soft continuum manipulator and preliminary experiments with a 2-segments soft continuum arm are performed, showing promising results and the potential for our approach to be applied widely.

The human hand is a complex structure. It is strong but precise. It consists of a very complex mechanical structure that enables the hand to adapt and efficiently handle objects of various shapes, weights and textures. Today's prosthetic devices, struggling to provide similar functions, become overly complex and expensive. They are composed of multiple, precise parts, including miniaturised actuators and sensors as well as complex control, to satisfy the manipulation tasks required. In this paper we propose a soft pneumatic hand that adapts passively to the handled object due to its mechanical compliance. It is pressure driven and enables individual fingers to be controlled independently for dexterity or in groups when a synergistic finger movement is needed. The hand has a truly anatomical shape, is easy to replace and cheap in production. The design can be easily adjusted in terms of shape and size in order to fit each individual user. The paper presents the design, manufacturing technology, current control system and preliminary tests of the hand's capabilities.

Soft robotic structures might play a major role in the 4thindustrial revolution. Researchers have successfully demonstrated advantages of soft robotics over traditional robots made of rigid links and joints in several application areas including manufacturing, healthcare and surgical interventions. However, soft robots have limited ability to exert higher forces when it comes to interaction with the environment, hence, change their stiffness on demand over a wide range. One stiffness mechanism embodies tendon-driven and pneumatic air actuation in an antagonistic way achieving variable stiffness values. In this paper, we apply a beam-mechanics-based model to this type of soft stiffness controllable robot. This mathematical model takes into account the various stiffness levels of the soft robotic manipulator as well as interaction forces with the environment at the tip of the manipulator. The analytical model is implemented into a robotic actuation system made of motorised linear rails with load cells (obtaining applied forces to the tendons) and a pressure regulator. Here, we present and analyse the performance and limitations of our model.

Nowadays, the field of industrial robotics focuses particularly on collaborative robots that are able to work closely together with a human worker in an inherently safe way. To detect and prevent harmful collisions, a number of solutions both from the actuation and sensing sides have been suggested. However, due to the rigid body structures of the majority of systems, the risk of harmful collisions with human operators in a collaborative environment remains. In this paper, we propose a novel concept for a collaborative robot made of Variable Stiffness Links (VSLs). The idea is to use a combination of silicone based structures and fabric materials to create stiffness-controllable links that are pneumatically actuated. According to the application, it is possible to change the stiffness of the links by varying the value of pressure inside their structure. Moreover, the pressure readings from the pressure sensors inside the regulators can be utilised to detect collisions between the manipulator body and a human worker, for instance. A set of experiments are performed with the aim to assess the performance of the VSL when embedded in a robotic manipulator. The effects of different loads and pressures on the workspace of the manipulator are evaluated together with the efficiency of the collision detection control system and hardware.

The constantly increasing amount of machines operating in the vicinity of humans makes it necessary to rethink the design approach for such machines to ensure that they are safe when interacting with humans. Traditional mechanisms are rigid and heavy and as such considered unsuitable, even dangerous when a controlled physical contact with humans is desired. A huge improvement in terms of safe human-robot interaction has been achieved by a radically new approach to robotics - soft material robotics. These new robots are made of compliant materials that render them safe when compared to the conventional rigid-link robots. This undeniable advantage of compliance and softness is paired with a number of drawbacks. One of them is that a complex and sophisticated controller is required to move a soft robot into the desired positions or along a desired trajectory, especially with external forces being present. In this paper we propose an improved soft fluidic rotary actuator composed of silicone rubber and fiber-based reinforcement. The actuator is cheap and easily manufactured providing near linear actuation properties when compared to pneumatic actuators presented elsewhere. The paper presents the actuator design, manufacturing process and a mathematical model of the actuator behavior as well as an experimental validation of the model. Four different actuator types are compared including a square-shaped and three differently reinforced cylindrical actuators.

It is well acknowledged that the opposing thumb granted humans advanced manipulation capabilities. However, such a feature is not statistically quantified, and its representation is not formally addressed in robotics yet. This paper studies whether the displacement of the opposing thumb in humans is a determining factor for shaping the grip. Using statistical analysis of the variability of motion capture data from the GRASP database, we found that the displacement of the thumb plays a leading role on the shaping of the grip, independently from the specific object being grasped. Furthermore, we map and compare the reachability spaces of the human thumb and two state-of-the-art robotic thumbs: (1) the shadow and (2) the iCub hands. We conclude that the kinematics of robotic thumbs does not evenly span the reachability space of the human thumb, favoring precision grasping motions. Hence, our findings contribute to the discussion of the optimal modeling of robotic hands.

Since December 2014, FourByThree Project (“Highly customizable robotic solutions for effective and safe human robot collaboration in manufacturing applications”) is developing a new generation of modular industrial robotic solutions that are suitable for efficient task execution in collaboration with humans in a safe way and are easy to use and program by factory workers. This paper summarizes the key technologies that are used to achieve this goal.

In this paper, a potential-field-based real-time path planning algorithm for a multi-segment continuum manipulator is proposed. This planner is employed to enable a continuum-style manipulator to move autonomously in dynamic environments in real-time. The classic potential field method is modified to make it applicable for a kinematics model based on the constant-curvature assumption. The contribution of this paper lies in the design of a novel potential field in the actuator space satisfying the mechanical constraints of the manipulator. The planning algorithm is tested and validated in real-time simulation for a 3 segments continuum manipulator. Preliminary tests for a tendon-driven single-segment continuum manipulator prototype confirm the performance of the proposed planner.

The complexity of soft continuum manipulators with hybrid and tuneable structures poses a challenging task to achieve an inverse kinematics model which is both precise and computationally efficient for control and optimization purposes. In this paper, a new method based on the principle of virtual work and a geometry deformation approach is presented for the inverse kinematics model of the STIFF-FLOP arm which is a pneumatically actuated continuum manipulator. We propose a novel simplified and computationally efficient yet accurate analytical solution to analyse the static behaviour of a compound soft manipulator in the presence of external and body forces which is verified against experimental data, showing promising agreement with 10% mean error for planar movements. In the process, we present a new modelling approach for braided soft extensor actuators with no braid-surface relative slip constraint. For the first time, our model predicts a simple analytical solution for the cross section deformation which is essential to control soft manipulators with regional tunable stiffness structure.

Muscle activity monitoring or Electromyography (EMG) is useful in gait analysis, injury prevention, computer or robot interfaces and assisting patients with communication difficulties. However, EMG is typically invasive or obtrusive, expensive and difficult to use for untrained users. A possible solution is textile-based surface EMG (sEMG) integrated into clothing and used as a wearable device. This is, however, challenging due to (i) uncertainties in the electrical properties of conductive threads used to construct electrodes, (ii) imprecise fabrication technologies (e.g., embroidery, sewing), and (iii) a lack of standardisation in the choice of design variables. This paper, for the first time, provides a design guide for such sensors by performing a thorough examination of the effect of design variables on sEMG quality. Electrical characterisation and sEMG measurements are performed, considering the effects of manufacturing imprecision. Results show that the imprecisions in digital embroidery lead to a trade-off between low electrode resistance and high consistency. An optimum set of variables for this trade-off is identified and tested with sEMG during a variable force isometric grip exercise with n=6 participants and compared with traditional gel-based electrodes. Results show that thread-based electrodes provide a similar level of sensitivity to force variation as gel-based electrodes with about 90% correlation with expected linear behaviour.

In this paper, we present a novel pose estimation and obstacle avoidance approach for tendon-driven multi-segment continuum manipulators moving in dynamic environments. A novel multi-stage implementation of an Extended Kalman Filter is used to estimate the pose of every point along the manipulator's body using only the position information of each segment tip. Combined with a potential field, the overall algorithm will guide the manipulator tip to a desired target location and, at the same time, keep the manipulator body safe from collisions with obstacles. The results show that the approach works well in a real-time simulation environment that contains moving obstacles in the vicinity of the manipulator.

Contact force play is a significant role in success of the cardiac ablation. However, it is still challenging to estimate contact force when a catheter is under large bending and multiple contacts. This paper develops a new multi-section static model of the tendon-driven catheters for both real-time intrinsic force sensing and interaction control. The model allows estimating the catheter shape by the external force at arbitrary location. Also, an algorithm is developed for the contact force estimation using the shape estimation with the catheter end-position tracking and tension feedback. In this study, we validated the contact force and shape estimation using a robotic platform, which steers a catheter consisting of 4 tendons with tension feedback. The shape estimation results show that the model can accurately predict the catheter shape; the position difference between measured and estimated was 2.5mm. The results of the contact force estimation show that 3-dimensional contact forces can be estimated accurately using the proposed method. The magnitude of contact force error was 0.0117N with 350Hz update rate.

Proximity and distance estimation sensors are broadly used in robotic hands to enhance the quality of grasping during grasp planning, grasp correction and in-hand manipulation. This paper presents a fiber optical proximity sensor that is integrated with a tactile sensing fingertip of a robotic hand of a mobile robot. The distance estimation of proximity sensors are typically influenced by the reflective properties of an object, such as color or surface roughness. With the approach proposed in this paper, the accuracy of the proximity sensor is enhanced using the information collected by the vision system of the robot. A camera is employed to obtain RGB values of the object to be grasped. Further on, the data obtained from the camera is used to obtain the correct calibration for the proximity sensor. Based on the experimental evidence, it is shown that our approach can be effectively used to reduce the distance estimation error.

This work investigates the applicability of stiffness-based modelling in soft robotics manipulation. The methodology is introduced and applied to model a soft robotics manipulator as single 3d Timoshenko beam element. The model is then utilized to solve the forward kinematics problem for the manipulator. The algorithm is validated comparing the simulated deflection with the deflection of the physical manipulator for two defined pressure sequences. It is shown that the model behaves in a highly similar fashion in comparison to the manipulator. For both trajectories the maximum position error is close to 6 mm while the error in orientation not more than 18°. The methodology as described in this work reveals great applicability to the field of soft robots being limited only by the stiffness matrix assembly for the given system. Implementations of inverse kinematics and the effects of external force applications are effectively integrable in the described theory.

Tactile data and kinesthetic cues are two important sensing sources in robot object recognition and are complementary to each other. In this paper, we propose a novel algorithm named Iterative Closest Labeled Point (iCLAP) to recognize objects using both tactile and kinesthetic information. The iCLAP first assigns different local tactile features with distinct label numbers. The label numbers of the tactile features together with their associated 3D positions form a 4D point cloud of the object. In this manner, the two sensing modalities are merged to form a synthesized perception of the touched object. To recognize an object, the partial 4D point cloud obtained from a number of touches iteratively matches with all the reference cloud models to identify the best fit. An extensive evaluation study with 20 real objects shows that our proposed iCLAP approach outperforms those using either of the separate sensing modalities, with a substantial recognition rate improvement of up to 18%.

This paper presents a methodology for the development of a multi-axis force/torque sensor based on optoelectronic technology. The advantages of using this sensing principle are the low manufacturing costs, the simple fabrication, and the immunity to electrical noise. The force/ torque sensor makes use of six optical sensors: each sensor measures the displacement of a reflective surface that moves integrally with a simply-supported beam. The proposed mechanical structure allows for a variety of shapes on the mechanical structure to be easily adaptable to many robot applications. In this paper, we present a five-axis force/torque sensor based on this optoelectronic principle. To measure force/torque components, two identical three-DoF force/torque sensor structures (comprised of three beams) are mounted on top of each other. Photo sensors and mirrors are fixed inside the structure to measure the six beam deflections. In this paper, we describe the sensor structure, design, fabrication, calibration, and verify our sensor development methodology.

The production and sale of counterfeit and substandard pharmaceutical products, such as essential medicines, is an important global public health problem. We describe a chemometric passport-based approach to improve the security of the pharmaceutical supply chain. Our method is based on applying nuclear quadrupole resonance (NQR) spectroscopy to authenticate the contents of medicine packets. NQR is a non-invasive, non-destructive, and quantitative radio frequency (RF) spectroscopic technique. It is sensitive to subtle features of the solid-state chemical environment and thus generates unique chemical fingerprints that are intrinsically difficult to replicate. We describe several advanced NQR techniques, including two-dimensional measurements, polarization enhancement, and spin density imaging, that further improve the security of our authentication approach. We also present experimental results that confirm the specificity and sensitivity of NQR and its ability to detect counterfeit medicines.

This paper presents salient features of human-human interaction where one person with limited auditory and visual perception of the environment (a follower) is guided by an agent with full perceptual capabilities (a guider) via a hard rein along a given path. We investigate several salient features of the interaction between the guider and follower such as: 1) the order of an autoregressive (AR) control policy that maps states of the follower to actions of the guider; 2) how the guider may modulate the pulling force in response to the trust level of the follower; and 3) how learning may successively apportion the responsibility of control across different muscles of the guider. Based on experimental systems identification on human demonstrations from ten pairs of naive subjects, we show that guiders tend to adopt a third-order AR predictive control policy and followers tend to adopt second-order reactive control policy. Moreover, the extracted guider's control policy was implemented and validated by human-robot interaction experiments. By modeling the follower's dynamics with a time varying virtual damped inertial system, we found that it is the coefficient of virtual damping which is most sensitive to the trust level of the follower. We used these experimental insights to derive a novel controller that integrates an optimal order control policy with a push/pull force modulator in response to the trust level of the follower monitored using a time varying virtual damped inertial model.

Humans can hold a live animal like a hamster without overly squeezing despite the fact that its soft body undergoes impedance and size variations due to breathing and wiggling. Although the exact nature of such biological motor controllers is not known, existing literature suggests that they maintain metastable interactions with dynamic objects based on prediction rather than reaction. Most robotic gripper controllers find such tasks very challenging mainly due to hard constraints imposed on the stability of closed-loop control and inadequate rates of convergence of adaptive controller parameters. This paper presents experimental and numerical simulation results of a control law based on a relaxed stability criterion of reducing the probability of failure to maintain a stable grip on a soft object that undergoes temporal variations in its internal impedance. The proposed controller uses only three parameters to interpret the probability of failure estimated using a history of grip forces to adjust the grip on the dynamic object. Here, we demonstrate that the proposed controller can maintain smooth and stable grip tightening and relaxing when the object undergoes random impedance variations, compared with a reactive controller that involves a similar number of controller parameters.

This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. It is then treated as a probabilistic estimation problem solved in a framework of recursive Bayesian filtering. Feature-based measurement model and Gaussian based motion model are thus built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and the results have proven the feasibility of our proposed framework.

This paper proposes a soft, inflatable manipulator that is antagonistically actuated by tendons and pneumatics. The combination of the two actuation mechanisms in this antagonistic robot structure is inspired by the octopus which uses its longitudinal and transversal muscles to steer, elongate, shrink and also stiffen its continuum arms. By “activating” its antagonistic muscle groups at the same time, the octopus can achieve multiple motion patterns as well as stiffen their arms. Being organized in a similar fashion, our robot manipulator uses, on the one hand, pneumatic actuation and, on the other hand, tendon-based actuation - one opposing the other, achieving an overall antagonistic actuation framework. Controlling the pressure inside the robot while at the same time controlling the tendons' displacements, the robot can be moved into a wide range of configurations while simultaneously controlling the arm's stiffness. This paper builds on earlier work by the authors: Here, we present a new conic-shaped manipulator structure and the control architecture. Using a constant curvature model, we have derived an approach suitable for controlling the robot manipulator. The manipulator's reachable workspace is analyzed and proof-of-concept experiments were conducted to show the robot's stiffness control and motion abilities.

The absence of touch of sense is a widely known drawback of robotic minimally invasive surgery (MIS). This paper proposes a design of optic soft tactile arrays which is promising to be adapted for MIS. The proposed design consists of multiple soft material channels. Each channel is designed using the Bernoulli pipe structure to amplify the sensor's sensitivity through input and output diameter difference. A multi-core optic fiber cable and a camera are used to capture the change of light intensity caused by the contact forces applied onto the individual soft material channels. The proposed sensor has the following advantages: 1) making use of 3D printing and soft material casting, it is suitable for designing sensors with high density of tactile elements; 2) it also allows the sensor to be designed in an arbitrary shape to fit various MIS applications; 3) compared to other light-intensity based tactile sensor, it is easy to fabricate and miniaturize; it avoids the complexity of attaching reflectors to individual sensing elements; 4) it is immune to electromagnetic interference. In this paper, a prototype which has 3×3 tactile elements in an area of 9.5 × 11 mm2has been developed and test for feasibility study. Also, a noise-filtering algorithm is developed to reduce the imaging noise. Validation experiments were carried out and results show that the average measurable force range for a single tactile element is 0 to 1.622N with an average accuracy of 97%. The sensor has low crosstalk-to-signal ratio, 1.8% on average, and has no signal drift over time.

This paper proposes a novel Cosserat rod model based method to estimate contact forces based on the shape analysis of the catheter. We simplify the original Cosserat rod model to achieve real-time estimation of the force magnitude and directions applied to the catheter tip. The simplified model contains a set of arithmetic equations, which can be rapidly solved using an iterative optimization algorithm. Experimental evaluation shows that the computational frequency of the force estimation was found to be 33.7Hz. Both the magnitude and the direction of the contact force were accurately estimated. The accuracy of the estimations of magnitude was 89.50%, and for the contact direction was 88.13%. Mean errors of the contact force and the contact angle are 7 × 10-4Nand 0.931 degree respectively. To prove the concept, the catheter shape is detected through a RGB camera. However, the proposed method can be easily applied to existing catheter gating and detecting methods using the medical imaging environments such as X-ray fluoroscopy, CT, ultrasound, magnetic field and MRI. The results of evaluation experiments demonstrate what the proposed method is promising for force estimation without the need of a physical force sensor in various types of catheter tips.

This paper presents an innovative hand-held device able to compute stiffness when interacting with a soft object. The device is composed of four linear indenters and a USB camera. The stiffness is computed in real-time, tracking the movements of spherical features in the image of the camera. Those movements relate to the movements of the four indenters when interacting with a soft surface. Since the indenters are connected to springs with different spring constants, the displacement of the indenters varies when interacting with a soft object. The proposed multi-indenting device allows measuring the object's stiffness as well as the pan and tilt angles between the sensor and the surface of the soft object. Tests were performed to evaluate the accuracy of the proposed palpation mechanism against commercial springs of known stiffness. Results show that the accuracy and sensitivity of the proposed device increases with the softness of the examined object. Preliminary tests with silicon show the ability of the sensing mechanism to characterize phantom soft tissue for small indentation. It is noted that the results are not affected by the orientation of the device when probing the surface. The proposed sensing device can be used in different applications, such as external palpation for diagnosis or, if miniaturized, embedded on an endoscopic camera and used in Minimally Invasive Surgery (MIS).

It is well known that tactile information can be used to enhance the quality of grasping. Therefore, new technological solutions for sensing in grasping are needed. This paper presents an optical based fingertip sensor that measures both interaction forces and proximity between fingertip and environment. The combination of multiple sensing modalities in the tip of a finger can significantly improve grasping and manipulation capabilities. In this work we present the design and the required calibration of individual sensing elements, and of the integrated fingertip sensor developed for a 3-fingered metamorphic robotic hand. Emulated grasping experiments, using a pinch grip, were performed to illustrate the concept and validate the performance of the developed sensing system. As a result, it was possible to determine the sensor position with respect to an object during approach, contact and grasp.

This paper presents a highly compact fibre-optic system based on light intensity modulation for multi-segment flex sensing in pliable robot arms, e.g., articulated surgical instruments. This fibre-optic arrangement is 7.5 mm in diameter and is comprised of a two-segment flexible and stretchable Steiner chain arm section with twelve housings at the distal side which accommodates passive cables. The displacement of each cable will be used to determine the bending. This Steiner chain section is followed by a basal rigid fibre-optic sensing unit integrated with a low-friction retractable distance modulation array which couples the motion of the passive cables with light-emitting optical fibres. The low-friction retractable distance modulation array uses steel spring-needle double sliders to reduce the hysteresis and to recover reference sensor values when the arm returns to its original straight configuration. The U-shape loopback design of the optical fibres allows integration of all electronics away from the sensing site. The experimental results indicate a maximum bending angle error of 6° in one individual segment of the two-segment arm with respect to reference angle values calculated from camera images.

To efficiently explore a surface using the sense of touch, a novel contact sensing finger was created and a surface following control algorithm for the finger was devised. Based on the accurate estimation of contact locations, and the direction and magnitude of the normal and tangential forces, the finger can robustly and rapidly follow surfaces with large change in curvature while maintaining a desired constant normal force. In this paper, the design and testing of the contact sensing finger are presented and the control algorithm for surface contour following is proposed and validated using objects with different shapes and surface materials. The results demonstrate that using the developed finger and the control algorithm, a surface can be efficiently explored with rapid sliding speed. To demonstrate the potential applications of the proposed approach, the friction properties of an explored object surface are computed and, for a known object, its pose is estimated.

There has been a growing enthusiasm to use anthropomorphic hands of humanoid robots to manipulate every-day objects and tools designed for humans. However, multi-fingered grasping imposes a formidable control challenge due to the high dimensionality of the joint space and the difficulty to form a functional grip on objects. We propose a hybrid technique based on grasping synergies extracted from kinaesthetic demonstrations on a given object with a primitive geometry - a cuboid in this case - and passive kinematic enveloping as a generalization technique. Experiments were carried out on an iCub humanoid robot using everyday objects such as a telephone receiver, a computer mouse, three white board markers bundled together, a fencing handle, a compact disc keep case, and a drinking glass. We prove that the primitives extracted from kinaesthetic demonstrations on a cuboid can be generalized across a majority of the above real world objects.

This paper presents an innovative approach of utilising visual feedback to determine physical interaction forces with soft tissue during Minimally Invasive Surgery (MIS). This novel force sensing device is composed of a linear retractable mechanism and a spherical visual feature. The sensor mechanism can be adapted to endoscopic cameras used in MIS. As the distance between the camera and feature varies due to the sliding joint, interaction forces with anatomical surfaces can be computed based on the visual appearance of the feature in the image. Hence, this device allows the measurement of forces without introducing new stand-alone sensors. A mathematical model was derived based on validation data tests and preliminary experiments were conducted to verify the model's accuracy. Experimental results confirm the effectiveness of our vision based approach.

Current surgical tele-manipulators do not provide explicit haptic feedback during soft tissue palpation. Haptic information could improve the clinical outcomes significantly and help to detect hard inclusions within soft-tissue organs indicating potential abnormalities. However, system instability is often caught by direct force feedback. In this paper, a new approach to intra-operative tumor localization is introduced. A virtual-environment tissue model is created based on the reconstructed surface of a soft-tissue organ using a Kinect depth sensor and the organ's stiffness distribution acquired during rolling indentation measurements. Palpation applied to this tissue model is haptically fed back to the user. In contrast to previous work, our method avoids the control issues inherent to systems that provide direct force feedback. We demonstrate the feasibility of this method by evaluating the performance of our tumor localization method on a soft tissue phantom containing buried stiff nodules. Results show that participants can identify the embedded tumors; the proposed method performed nearly as well as manual palpation.

This paper introduces an optical based three axis force sensor which can be integrated with the robot arm of the EU project STIFF-FLOP (STIFFness controllable Flexible and Learnable Manipulator for Surgical Operations) in order to measure applied external forces. The structure of the STIFF-FLOP arm is free of metal components and electric circuits and, hence, is inherently safe near patients during surgical operations. In addition, this feature makes the performance of this sensing system immune against strong magnetic fields inside magnetic resonance (MR) imaging scanners. The hollow structure of the sensor allows the implementation of distributed actuation and sensing along the body of the manipulator. In this paper, we describe the design and calibration procedure of the proposed three axis optics-based force sensor. The experimental results confirm the effectiveness of our optical sensing approach and its applicability to determine the force and momentum components during the physical interaction of the robot arm with its environment.

Robotic manipulators for Robot-assisted Minimally Invasive Surgery (RMIS) pass through small incisions into the patient's body and interact with soft internal organs. The performance of traditional robotic manipulators such as the da Vinci Robotic System is limited due to insufficient flexibility of the manipulator and lack of haptic feedback. Modern surgical manipulators have taken inspiration from biology e.g. snakes or the octopus. In order for such soft and flexible arms to reconfigure itself and to control its pose with respect to organs as well as to provide haptic feedback to the surgeon, tactile sensors can be integrated with the robot's flexible structure. The work presented here takes inspiration from another area of biology: cucumber tendrils have shown to be ideal tactile sensors for the plant that they are associated with providing useful environmental information during the plant's growth. Incorporating the sensing principles of cucumber tendrils, we have created miniature sensing elements that can be distributed across the surface of soft manipulators to form a sensor network capable of acquire tactile information. Each sensing element is a retractable hemispherical tactile measuring applied pressure. The actual sensing principle chosen for each tactile makes use of optic fibres that transfer light signals modulated by the applied pressure from the sensing element to the proximal end of the robot arm. In this paper, we describe the design and structure of the sensor system, the results of an analysis using Finite Element Modeling in ABAQUS as well as sensor calibration and experimental results. Due to the simple structure of the proposed tactile sensor element, it is miniaturisable and suitable for MIS. An important contribution of this work is that the developed sensor system can be ”loosely” integrated with a soft arm effectively operating independently of the arm and without affecting the arm's motion during bending or elongation.

This paper presents a novel technique for estimating stiffness distribution of a soft tissue using a prototype of air-float stiffness probe. The air-float stiffness probe uses an indentation technique to estimate tissue stiffness. It consists of a spherical indenter and an indentation depth sensing mechanism that operates under a supply of compressed air. The probe has the ability to estimate tissue stiffness in non-planner tissue profiles. A novel technique to estimate indentation force, using supply air pressure is described and validated using both experimental procedures and finite element analysis (FEA) techniques. FEA package, ANSYS CFX was used for analyzing in 2D the solid-fluid interactions within the probe to estimate force available at the indenter at different supply air pressure settings. Both the experimental results and numerical simulations suggest that there is a very strong linear correlation between the indentation force and the supply air pressure. This relationship is used to estimate the indentation force in real time during an indentation test. Verification tests carried out on simulated silicon samples showed that the probe is capable of estimating tissue stiffness values with high accuracy and repeatability.

This paper introduces a novel continuum-style robot that integrates multiple layers of compliant modules. Its essential features lie in that its bending is not based on natural compliance of a continuous backbone element or soft skeletal elements but instead is based on the compliance of each structured planar module. This structure provides several important advantages. First, it demonstrates a large linear bending motion, whilst avoiding joint friction. Second, its contraction and bending motion are decoupled. Third, it possesses ideal back-drivability and a low hysteresis. We further provide an analytical method to study the compliance characteristics of the planar module and derive the statics and kinematics of the robot. The paper provides an overview of experiments validating the design and analysis.

This paper explores a new hybrid actuation principle combining pneumatic and tendon-driven actuators for a soft robotic manipulator. The fusion of these two actuation principles leads to an overall antagonistic actuation mechanism whereby pneumatic actuation opposes tendon actuation - a mechanism commonly found in animals where muscles can oppose each other to vary joint stiffness. We are taking especially inspiration from the octopus who belongs to the class of Cephalopoda; the octopus uses its longitudinal and transversal muscles in its arms to achieve varied motion patterns; activating both sets of muscles, the octopus can control the arm stiffness over a wide range. Our approach mimics this behavior and achieves comparable motion patterns, including bending, elongation and stiffening. The proposed method combines the advantages of tendon-driven and pneumatic actuated systems and goes beyond what current soft, flexible robots can achieve: because the new robot structure is effectively an inflatable, sleeve, it can be pumped up to its fully inflated volume and, also, completely deflated and shrunk. Since, in the deflated state, it comprises just its outer “skin” and tendons, the robot can be compressed to a very small size, many times smaller when compared to its fully-inflated state. In this paper, we describe the mechanical structure of the soft manipulator. Proof-of-concept experiments focus on the robot's ability to bend, to morph from completely shrunk to entirely inflated as well as to vary its stiffness.

The break-away friction ratio (BF-ratio), which is the ratio between friction force and the normal force at slip occurrence, is important for the prediction of incipient slip and the determination of optimal grasping forces. Conventionally, this ratio is assumed constant and approximated as the static friction coefficient. However, this ratio varies with acceleration rates and force rates applied to the grasped object and the object material, which lead to difficulties in determining optimal grasping forces that avoid slip. In this paper, we propose a novel approach based on the interactive forces to allow a robotic hand to predict object slip before its occurrence. The approach only requires the robotic hand to have a short haptic surface exploration over the object surface before manipulating it. Then, the frictional properties of the finger-object contact can be efficiently identified, and the BF-ratio can be real-time predicted to predict slip occurrence under dynamic grasping conditions. Using the predicted BF-ratio as a slip, threshold is demonstrated to be more accurate than using the static/Coulomb friction coefficient. The presented approach has been experimentally evaluated on different object surfaces, showing good performance in terms of prediction accuracy, robustness, and computational efficiency.

In the wind turbine manufacturing industry, the bolt-nut joint tightening process is one of the core processes in the full production chain and concerned with assembling the hub body, the pitch system and the bearing unit. This operation is currently executed manually with the aid of different tools and gauges; the main disadvantages are a relatively high degree of variability and the necessity to repeat this task several times during a production run to achieve a satisfactory, final tightening torque within a specified angle range. Moreover, the bolt tightening process includes various uncertainties such as the presence of friction forces and the use of different bolt sizes with different stiffness values which make it highly nonlinear and uncertain resulting in a challenging control problem. To facilitate the development of an effective control strategy, we study the bolt tightening process and propose 4 tightening stages, namely, bolt-nut alignment, partial and full engagement and final bolt tightening. Based on the characteristics of each stage, a fuzzy controller is designed for each stage to realize the respective control objectives. A fuzzy error detector incorporating the knowledge of each stage is proposed for early error detection, making use of the input from a torque and encoder (angular position) sensor. Errors can be detected in each stage to interrupt the process and prevent any damage to the system.

Flexible manipulators have promising applications in minimally invasive surgery as it allows the surgical tools reach targets which are prohibited by conventional rigid surgical instrument. However one of the technical difficulties of implementing the flexible manipulator is to measure the bending curvature. This paper proposes the design of a novel optical sensor for measuring the bending curvature of a flexible manipulator based on light intensity modulation. The sensor is low cost and is temperature independent. A theoretical model of using the sensor design to deduce the curvature of a flexible robot has been created. Implementing the proposed theoretical model, the developed sensor has been used to measure the bend of a section of a flexible segment. Validation tests have been carried out; the results demonstrate that the developed sensor has good accuracy in measuring the bending angles, the orientation of the bending and the bending radius.

Robot grasping and manipulation relies mainly on two types of sensory data: vision and tactile sensing. Localisation and recognition of the object is typically done through vision alone, while tactile sensors are commonly used for grasp control. Vision performs reliably in uncluttered environments, but its performance may deteriorate when the object is occluded, which is often the case during a manipulation task, when the object is in-hand and the robot fingers stand between the camera and the object. This paper presents a method to use the robot's sense of touch to refine the knowledge of a manipulated object's pose from an initial estimate provided by vision. The objective is to find a transformation on the object's location that is coherent with the current proprioceptive and tactile sensory data. The method was tested with different object geometries and proposes applications where this method can be used to improve the overall performance of a robotic system. Experimental results show an improvement of around 70% on the estimate of the object's location when compared to using only vision.

Advanced tactile tools in minimally invasive surgery have become a pressing need in order to reduce time and improve accuracy in localizing potential tissue abnormalities. In this regard, one of the main challenges is to be able to estimate tissue parameters in real time. In palpation, tactile information felt at a given location is identified by the viscoelastic dynamics of the neighboring tissue. Due to this reason the tissue examination behavior and the distribution of viscoelastic parameters in tissue should be considered in conjunction. This paper investigates the salient features of palpation behavior on soft tissue determining the effectiveness of localizing hard nodules. Experimental studies involving human participants, and validation tests using finite element simulations and a tele-manipulator, were carried out. Two distinctive tissue examination strategies in force-velocity modulation for the given properties of target tissue were found. Experimental results suggest that force-velocity modulations during continuous path measurements are playing an important role in the process of mechanical soft tissue examination. These behavioral insights, validated by detailed numerical models and robotic experimentations shed light on future designs of optimal robotic palpation.

In the case of human intervention in disaster response operations like indoor firefighting, where the environment perception is limited due to thick smoke, noise in the oxygen masks and clutter, not only limit the environmental perception of the human responders, but also causes distress. An intelligent agent (man/machine) with full environment perceptual capabilities is an alternative to enhance navigation in such unfavorable environments. Since haptic communication is the least affected mode of communication in such cases, we consider human demonstrations to use a hard rein to guide blindfolded followers with auditory distraction to be a good paradigm to extract salient features of guiding using hard reins. Based on numerical simulations and experimental systems identification based on demonstrations from eight pairs of human subjects, we show that, the relationship between the orientation difference between the follower and the guider, and the lateral swing patterns of the hard rein by the guider can be explained by a novel 3rdorder auto regressive predictive controller. Moreover, by modeling the two party voluntary movement dynamics using a virtual damped inertial model, we were able to model the mutual trust between two parties. In the future, the novel controller extracted based on human demonstrations can be tested on a human-robot interaction scenario to guide a visually impaired person in various applications like fire fighting, search and rescue, medical surgery, etc.

This paper presents a novel fiber optic tactile probe designed for tissue palpation during minimally invasive surgery (MIS). The probe consists of 3×4 tactile sensing elements at 2.6mm spacing with a dimension of 12×18×8 mm3allowing its application via a 25mm surgical port. Each tactile element converts the applied pressure values into a circular image pattern. The image patterns of all the sensing elements are captured by a camera attached at the proximal end of the sensor system. Processing the intensity and the area of these circular patterns allows the computation of the applied pressure across the sensing array. Validation tests show that each sensing element of the tactile probe can measure forces from 0 to 1N with a resolution of 0.05 N. The proposed sensing concept is low cost, lightweight, sterilizable, easy to be miniaturized and compatible for magnetic resonance (MR) environments. Experiments using the developed sensor for tissue abnormality detection were conducted. Results show that the proposed tactile probe can accurately and effectively detect nodules embedded inside soft tissue, demonstrating the promising application of this probe for surgical palpation during MIS.

Robot-assisted minimal invasive surgery made it possible to improve the quality of surgical procedures and to enhance clinical outcomes. However, the need to palpate soft tissue organs with the aim to localize potential sites of abnormalities in real time has been recognized. For this work, ten subjects were recruited to perform a remote palpation procedure on a silicone phantom utilizing a tele-manipulation setup, to study their behavior when remotely palpating soft tissue. The stiffness values acquired during the remote palpation of a silicone phantom were transferred to the subjects by means of haptic and visual feedback. Participating subjects were asked to detect hard nodules in the silicone tissue using two distinct strategies: a) randomly chosen movements, and b) trajectory pattern, based on manual palpation techniques for clinical breast examination. We have compared relevant parameters, defining patterns observed during manual palpation, with the counterpart patterns occurring during remote palpation. The results show the effectiveness of applying palpation trajectory pattern used during manual soft tissue examination to tele-manipulation palpation.

During open surgery, surgeons can perceive the locations of tumors inside soft-tissue organs using their fingers. Palpating an organ, surgeons acquire distributed pressure (tactile) information that can be interpreted as stiffness distribution across the organ -an important aid in detecting buried tumors in otherwise healthy tissue. Previous research has focused on haptic systems to feedback the tactile sensation experienced during palpation to the surgeon during minimally invasive. However, the control complexity and high cost of tactile actuators limits its current application. This paper describes a pneumatic multi-fingered haptic feedback system for robot-assisted minimally invasive surgery. It simulates soft tissue stiffness by changing the pressure of an air balloon and recreates the deformation of fingers as experienced during palpation. The pneumatic haptic feedback actuator is validated by using finite element analysis. The results prove that the interaction stress between the fingertip and the soft tissue as well as the deformation of fingertips during palpation can be recreated by using our pneumatic multi-fingered haptic feedback method.

The aim of this paper is to improve the optimality and accuracy of techniques to guide a human in limited visibility and auditory conditions such as in fire-fighting in warehouses or similar environments. At present, breathing apparatus (BA) wearing fire-fighters move in teams following walls. Due to limited visibility and high noise in the oxygen masks, they predominantly depend on haptic communication through reins. An intelligent agent (man/machine) with full environment perceptual capabilities is an alternative to enhance navigation in such unfavorable environments, just like a dog guiding a blind person. This paper proposes an optimal state-dependent control policy to guide a follower with limited environmental perception, by an intelligent and environmentally perceptive agent. Based on experimental systems identification and numerical simulations on human demonstrations from eight pairs of participants, we show that the guiding agent and the follower experience learning for a optimal stable state-dependent novel 3rd and 2nd order auto regressive predictive and reactive control policies respectively. Our findings provide a novel theoretical basis to design advanced human-robot interaction algorithms in a variety of cases that require the assistance of a robot to perceive the environment by a human counterpart.

This paper introduces a new and low-cost tissue stiffness simulation technique for surgical training and robot-assisted minimally invasive surgery (RMIS) with pseudo-haptic feedback based on tissue stiffness maps provided by rolling mechanical imaging. Superficial palpation and deep palpation pseudo-haptic simulation methods are presented. Although without expensive haptic interfaces users receive only visual feedback (pseudo-haptics) when maneuvering a cursor over the surface of a virtual soft-tissue organ by means of an input device such as a mouse, a joystick, or a touch-sensitive tablet, the alterations to the cursor behavior induced by the method creates the experience of actual interaction with a tumor in the users' minds. The proposed methods are experimentally evaluated for tissue abnormality identification. It is shown that users can recognize tumors with these two methods and the rate of correctly recognized tumors in deep palpation pseudo-haptic simulation is higher than superficial palpation simulation.

This paper proposes a new computationally fast algorithm for classifying the primitive shape and pose of the local contact area in real-time using a tactile array sensor attached on a robotic fingertip. The proposed approach abstracts the lower structural property of the tactile image by analyzing the covariance between pressure values and their locations on the sensor and identifies three orthogonal principal axes of the pressure distribution. Classifying contact shapes based on the principal axes allows the results to be invariant to the rotation of the contact shape. A naïve Bayes classifier is implemented to classify the shape and pose of the local contact shapes. Using an off-shelf low resolution tactile array sensor which comprises of 5×9 pressure elements, an overall accuracy of 97.5% has been achieved in classifying six primitive contact shapes. The proposed method is very computational efficient (total classifying time for a local contact shape = 576μs (1736 Hz)). The test results demonstrate that the proposed method is practical to be implemented on robotic hands equipped with tactile array sensors for conducting manipulation tasks where real-time classification is essential.

This paper presents a prototype of a force/torque sensor that uses fiber optic guided light and linear polarizer materials to obtain intensity modulated light to detect applied force and torque to the sensing structure. The sensor is also capable of measuring the contact direction between the sensor and the object. The sensor's design and operating principles are explained and experimental data is given to verify the proposed operating principle. The experimental data shows that linear polarizers can be used to measure the torque applied to a force/torque sensor.

This paper presents a novel approach to measuring the indentation depth of a stiffness sensor in real time during a soft tissue palpation activity. The proposed system is integrated into a stiffness probe and is designed to intra-operatively aid the surgeon to rapidly identify the tissue abnormalities with minimum measurement inaccuracies due to tissue surface profile variations. Stiffness probe and the associated surface profile sensors are pneumatic and the newly designed system can concurrently measure the indentation depth and surface profile variations while sliding over the soft tissues in any direction in a near frictionless manner. With the pneumatic pressure maintained constant, the displacement of the sensing element is a direct function of the stiffness of the tissue under investigation. The sensor has a tunable force range and the indentation force can be adjusted externally to match tissue limitations. The prototype of the new design of stiffness probe was calibrated and tested on silicone blocks simulating soft tissue. The results show that this sensor can measure indentation depth more accurately than air cushion probe alone. The structure, working principle, and a mathematical model for this new design are described.

Maintaining the grip on an artery with a pulsating impedance, holding the steering wheel of a vehicle on a bumpy terrain, or holding a live hamster without excessive squeezing may be trivial tasks to most humans. However, a robot will find it very difficult to maintain the grip of such uncertain objects based on real-time feedback control. This paper presents a stochastic control law to maintain the grip on an uncertain object while manipulating against external forces. The radial impedance parameters of the soft object is assumed to undergo Gaussian random variations. Here we demonstrate that the proposed model free grip controller can maintain a safe grip at two diagonally opposite points of the object merely based on the statistics of the normal force. It accomplishes this by computing a probability of grip failure to adapt the compression on the soft object. A novel optimal estimation algorithm that can concurrently estimate the unknown impedance parameters of the object and the states of the coupled dynamic system is discussed as a potential tool to be used in predictive optimal impedance control on uncertain objects. Experimental results on adaptive grip control on a cylindrical tube inflated and deflated with a Gaussian random variation has been presented to validate the algorithm.

Robotic manipulators for minimally invasive surgeries have traditionally been rigid, with a steerable end effector. While the rigidity of manipulators improve precision and controllability, it limits reachability and dexterity in constrained environments. Soft manipulators with controllable stiffness on the other hand, can be deployed in single port or natural orifice surgical applications to reach a wide range of areas inside the body, while being able to passively adapt to uncertain external forces, adapt the stiffness distribution to suit the kinematic and dynamic requirements of the task, and provide flexibility for configuration control. Here, we present the design of a snake-like laboratory made soft robot manipulator of 20 mm in average diameter, which can actuate, soften, or stiffen joints independently along the length of the manipulator by combining granular jamming with McKibben actuators. It presents a comprehensive study on the relative contributions of the granule size, material type, and membrane coupling on the range, profile, and variability of stiffness.

Object surface properties are among the most important information which a robot requires in order to effectively interact with an unknown environment. This paper presents a novel haptic exploration strategy for recognizing the physical properties of unknown object surfaces using an intelligent finger. This developed intelligent finger is capable of identifying the contact location, normal and tangential force, and the vibrations generated from the contact in real time. In the proposed strategy, this finger gently slides along the surface with a short stroke while increasing and decreasing the sliding velocity. By applying a dynamic friction model to describe this contact, rich and accurate surface physical properties can be identified within this stroke. This allows different surface materials to be easily distinguished even if when they have very similar texture. Several supervised learning algorithms have been applied and compared for surface recognition based on the obtained surface properties. It has been found that the naïve Bayes classifier is superior to radial basis function network and k-NN method, achieving an overall classification accuracy of 88.5% for distinguishing twelve different surface materials.

Slip prediction is important for maintaining the stability of object handling in robust grasping and dexterous manipulation. However, up to date a challenge still remains that how to accurately predict slip occurrence before it actually happens to allow robotic hands to conduct slip compensation in time. The concept of friction cone has been conventionally used to predict slip occurrence, where the static/kinetic friction coefficient is used as a threshold. However, this threshold, i.e. the ratio of the friction and normal forces at slip occurrence (also named as break-away friction ratio), is found not constant but varies with changes in acceleration and disturbing forces applied on the grasped object, raising difficulties when attempting to accurately predict slip. In this paper, we propose a novel approach to accurately predict varying slip thresholds in real time and compensate the predicted slip during a dynamic grasping. To achieve this, first a simple but efficient haptic surface exploration using robotic fingers is carried out to identify the friction properties of an object surface. Once the friction properties are established, the slip threshold at a given grasping condition can be predicted and the grasping forces are adjusted to prevent slip. The presented approach has been evaluated, showing good performance in terms of prediction accuracy and computational efficiency.

This paper presents a novel optical fiber-based rolling indentation probe designed to measure the stiffness distribution of a soft tissue while rolling over the tissue surface during minimally invasive surgery. By fusing the measurements along rolling paths, the probe can generalize a mechanical image to visualize the stiffness distribution within the internal tissue structure. Since tissue abnormalities are often firmer than the surrounding organ or parenchyma, a surgeon then can localize abnormalities by analyzing the image. The performance of the developed probe was validated using simulated soft tissues. Results show that the probe can measure both force and indentation depth accurately with different orientations when the probe approached and rolled on the tissue surface. In addition, experiments for tumor, identification through rolling indentation were conducted. The size and embedded depth of the tumor, as well as the stiffness ratio between the tumor and tissue, were varied during tests. Results demonstrate that the probe can effectively and accurately identify the embedded tumors.

We describe a novel approach for demonstrate of a wheel-rolling tissue deformation as well as the abnormalities tissue depth evaluation using a rolling finite element model (RFEM). Since a wheeled probe which is capable of performing rolling tissue indentation has been proven to be a promising device to rapid conduct soft tissue property identification for localization and documentation of the abnormalities within the tissue, with the aim of compensating the loss of haptic and tactile feedback experienced during robotic-assisted minimally invasive surgery (MIS). To implement such a device requires a good understanding of the dynamics of the wheel-tissue rolling interaction and relationship between the tissue internal structure and the corresponding tissue reaction force. In this paper we propose the RFEM of the dynamic interaction between a wheeled probe and a soft tissue sample using ABAQUS finite element analysis software package. The aim of this work is to more precisely locate abnormalities depth within soft tissues using RFEM and aid surgeons better in the decision of resection during MIS through the understanding of dynamics of wheel-tissue rolling interaction. The soft tissue was modelled as a nonlinear hyperelastic material with geometrical nonlinearity and the modelling parameters were calibrated using experimental data from standard tests. The purposed RFEM consists of simulations of wheel-tissue rolling indentations on a silicone phantom with varied tissue internal structure and also running on a biological tissue such as a porcine kidney. The results show that the proposed method can predicted the wheel-tissue interaction force of the rolling indentation with a good agreement results and the documentary from empirical equation of RFEM can identify the simulated tumors depth accurately.

Slip plays a vital role in traction control when a mobile robot traverses over soft soils. To estimate slip parameters, accurate measurement of robot velocity is particularly required. Previous related work done by the authors has adopted a single downward-looking single camera system for velocity and slip estimation [1][2]; however, such a single camera system is prone to lose accuracy when the distance between the camera and terrain is time-varying, such as traversing over uneven terrains [1]. To cope with the problem, this paper presents a robust downward-looking camera based velocity estimation approach, which can particularly be capable of identifying height variation and compensating for velocity estimation. A downward-looking stereo camera instead of previously used single camera is adopted. The camera-terrain distance can be estimated by matching same features in left and right frames. Robot velocity measured with height compensation can be more accurate, compared to estimates without it. The proposed approach has been validated through comprehensive experimental study on a lab-based test rig; and test results show good performance of the proposed approach. With the proposed method, slip estimation techniques given by [2][3] can be promisingly extended to non-flat terrains.

This paper presents a novel miniaturized force-indentation depth (FID) sensor designed to conduct indentation on soft tissue during minimally invasive surgery. It can intra-operatively aid the surgeon to rapidly identify the tissue abnormalities within the tissue. The FID sensor can measure the indentation depth of a semi-spherical indenter and the tissue reaction force simultaneously. It make use of with fiber optical fiber sensing method measure indentation depth and force and is small enough to fit through a standard trocar port with a diameter of 11 mm. The created FID sensor was calibrated and tested on silicone block simulating soft tissue. The results show that the sensor can measure the indentation depth accurately and also the orientation of the sensor with respect to the tissue surface whilst performing indentation.

This paper presents the prototype design and development of a miniature MR-compatible fiber optic force sensor suitable for the detection of force during MR-guided cardiac catheterization. The working principle is based on light intensity modulation where a fiber optic cable interrogates a reflective surface at a predefined distance inside a catheter shaft. When a force is applied to the tip of the catheter, a force sensitive structure varies the distance and the orientation of the reflective surface with reference to the optical fiber. The visual feedback from the MRI scanner can be used to determine whether or not the catheter tip is normal or tangential to the tissue surface. In both cases the light is modulated accordingly and the axial or lateral force can be estimated. The sensor exhibits adequate linear response, having a good working range, very good resolution and good sensitivity in both axial and lateral force directions. In addition, the use of low-cost and MR-compatible materials for its development makes the sensor safe for use inside MRI environments.

This paper describes the design and construction of a miniaturized triaxial force sensor which can be applied inside a magnetic resonance imaging (MRI) machine. The sensing principle of the sensor is based on an optical intensity modulation mechanism that utilizes bent-tip optical fibers to measure the deflection of a compliant platform when exposed to a force. By measuring the deflection of the platform using this optical approach, the magnitude and direction of three orthogonal force components (Fx, Fy, and Fz) can be determined. The sensor prototype described in this paper demonstrates that it can perform force measurements in axial and radial directions with working ranges of +/-2 N. Since the sensor is small in size and entirely made of nonmetallic materials, it is compatible with minimally invasive surgery (MIS) and safe to be deployed within magnetic resonance (MR) environments.

This paper presents a novel approach of performing artificial tactile sensing based on the deployment of prismatic-tip optical fibers. The primary principle of the sensing schemes relies on light intensity modulation for detecting the deformation of an elastic element experiencing a force load. By measuring the change of light signal intensity, the magnitude of the applied force can be determined. The force distribution over an area can be evaluated using an array of optical fibers. The tactile sensor array prototype described in this paper demonstrates its capability and feasibility in performing tactile sensing and force measurement over a range of approximately 0 to 4.8 N. Due to its simple sensing structure, it is easy to manufacture and the sensor can be miniaturized for applications in dexterous robotic handling.

This paper describes a novel design of a 3-axis force sensor which can be applied in magnetic resonance (MR) workspaces such as that of a magnetic resonance imaging (MRI) machine. The sensor operates based on an optical sensing principle to measure forces deforming a 3 degree-of-freedom (DOF) flexible structure. By detecting minute deflection of such a structure using an optical sensing scheme, the magnitude and direction of an applied force can be determined. The sensor prototype described in this document demonstrates its capability of performing force measurement in both axial and radial directions with the calibrated working ranges of +/-3 N. Because all the sensor's components are entirely fabricated from non-metallic and dielectric materials, the sensor is considered suitable for applications in MR environments.

The authors have recently proposed the method of rolling indentation over soft tissue to rapidly identify soft tissue properties for localization and detection of tissue abnormalities, with the aim of compensating for the loss of haptics information experienced during robotic-assisted minimally invasive surgery (RMIS). This paper investigates the concept of rolling indentation using finite element modeling. To obtain ground truth data, rolling indentation experiments are conducted on a silicone phantom which contains three simulated tumours. The tissue phantom is modeled as hyperelastic material using ABAQUStrade. The identification of tumours includes two parts: firstly, when the spatial location of tumour is known, identify the tumour's mechanical properties (initial shear modulus); secondly if the mechanical properties of tumour are known, identify the tumour's spatial location. The results show that the proposed method can identify information of tumours accurately and robustly. The identified tumour mechanical properties and tumour locations are in good agreement with experimental measurements.

Minimally Invasive Surgery (MIS) and robot surgery have opened new ways to perform surgical operations in a safer and simultaneously faster manner. In an effort to follow this minimally invasive trend, this paper presents the feasibility study of a novel fibre-optic catheter prototype. This prototype sensor has the ability to measure forces from the sides and tip. Classification of forces from multiple positions on a catheter provides valuable information for safe navigation inside the vasculature and heart of a patient. This sensor employs two fibre-optic schemes, one for the tip and one for the sides of the catheter; it is made entirely of plastic, making it compatible with Magnetic Resonance Imaging (MRI). A test bench was used to determine the linearity coefficients during static loading. These initial experiments on the prototype gave rise to an ideal linear force response coupled with low hysteresis. Finally, an experiment which tries to simulate the human blood vessel achieved satisfying results during dynamic sensor movement.

This paper presents a novel air-cushion tactile sensor for minimally invasive surgery that is fully MRI (magnetic resonance imaging) compatible. The proposed sensor is designed to detect tissue abnormalities within soft tissue surfaces. This is achieved by rolling over soft tissue in a virtually frictionless manner due to the design of the sensor in which the sensing element, a sphere, rests on a cushion of air. This design allows for rapid acquisition of tactile and mechanical properties of large areas of soft tissue. Laboratory experiments are carried out to show its feasibility as a tactile sensor for MIS and its behaviour under loading. The outcomes of the experiments illustrate the sensor's capability and potential as a tactile sensor for MIS. These results are discussed and future work is outlined.

This paper presents a robust slip estimation method for skid-steered mobile robots when they traverse over rough terrain. An optical flow-based visual sensor looking down the terrain surface is employed to recover motion of a mobile robot by tracking features selected from the terrain surface. The motion states of the mobile robot are initially estimated by the visual sensor, however, the estimates are prone to noise and uncertainty which degrades the accuracy and robustness of estimation. To cope with the noise and uncertainty from the visual sensor, a sliding mode observer (SMO) based on the kinematics model of the skid-steered mobile robot is delicately designed to simultaneously estimate slip parameters. The SMO scheme can give more accurate estimates than the extended Kalman filter (EKF) when the slip of the mobile robot has significant changes at abrupt steering. The complete slip estimation method is independent of terrain parameters and robust in the presence of noise and uncertainty. Experimental results show that the method has confident potential for slip estimation of skid-steered mobile robots.

Nuclear quadrupole resonance (NQR) is a radio frequency (RF) spectroscopic technique, allowing the detection of many high explosives and narcotics. In practice, NQR is restricted by the low signal-to-noise ratio of the observed signals, a problem further exacerbated by the presence of strong RF interference (RFI). The current literature focuses on the use of conventional, multiple-pulsed NQR (cNQR) to obtain signals. Here, we investigate an alternative method called stochastic NQR (sNQR), having many advantages over cNQR, one of which is the availability of signal-of-interest free samples. We exploit these samples forming a matched subspace-type detector, able to efficiently reduce the influence of RFI. Further, many of the ideas already developed for cNQR, including providing robustness to uncertainties in the assumed complex amplitudes and exploiting the temperature dependencies of the NQR spectral components, are recast for sNQR. The presented detector is evaluated on both simulated and measured trinitrotoluene (TNT) data.

This paper proposes a novel approach for the identification of the internal structure and mechanical properties of biological soft tissue using a force sensitive wheeled probe to generate a 'mechanical image' by rolling across the surface of a solid organ. Initially, a testing facility for validating the concept ex-vivo was developed. Preliminary validation tests were carried out on a silicone phantom with embedded abnormalities with the aim to link the derived 'mechanical image' with the known internal structure. Ex-vivo validation tests were also conducted on excised porcine livers. The data were analyzed in four parts: 1) the dynamic analysis of wheel-tissue interaction to validate that the measured parameters are representative of underlying tissue stiffness; 2) the development of a 'mechanical image' from the rolling data; 3) a comparison of standard 1-DOF indentation testing with 2-DOF rolling and 4) the characterization of the relationship between force and tissue deflection from the data contained within the mechanical image. The results show that the 'rolling mechanical image' is capable of capturing information relating to the underlying tissue stiffness distribution and characterizing the force-tissue deflection profile for a tissue sample. Examples of scenarios, where this information could potentially be used, include providing a surgeon with the ability to probe solid organs in-vivo during robot-assisted MIS or providing prior information for the modeling of tool-tissue interactions, such as steerable needles.

This paper describes the preliminary design and construction of an optical fiber sensor which has been developed for evaluating the feasibility of using an optical-based force sensing methodology to investigate mechanical soft tissue properties during minimally invasive surgery. This sensor applies a novel reflective light intensity modulation scheme using bent-tip optical fibers and a reflector to measure mechanical response of the soft tissue when it interacts with the sensor. By adopting such optical fibers to detect minute deflection of a flexible cylindrical structure at the reflective edge of the reflector, good sensitivity for the force detection can be obtained. The prototype described in this document has demonstrated that it can detect the tissue interaction forces in the axial direction and identify variations in tissue stiffness. The maximum force range that can be detected by the sensor is 3 N. The measurement resolution is 0.02 N.

Unmanned ground vehicles are widely used in industries where repetitive tasks or high risk missions are required. Such vehicles usually operate on soft deformable terrains and still require human supervision due to the complexity of the interaction between the vehicle and the terrain. A traversability prediction simulator has recently been developed. The simulator is used to investigate the influence of vehicle design and soil properties on the maneuverability performance of tracked vehicles on soft terrains. Results of the simulations carried out give insight into the behavior of such vehicles with different operating conditions and how they can be controlled.

This paper proposes a novel technique to estimate slips and velocities of an unmanned skid-steered vehicle. An optical flow-based visual sensor looking down the terrain surface is employed to recover the motion of the vehicle by tracking features selected from the terrain surface. The special orientation of the on-board camera is to assure high accuracy of the motion estimation. To cope with the noise and uncertainty from the visual sensor, a sliding mode observer (SMO) based on the kinematic model of the skid-steered vehicle is delicately designed to simultaneously estimate the slips and velocities. The complete non-GPS slip and velocity estimation technique is independent of terrain parameters and robust to noise and uncertainty. The SMO scheme can produce more accurate estimates than the extended Kalman filter (EKF) in the nonlinear case. Experimental results are given to show that the technique has good potential for vehicle slip and velocity estimation.

This paper proposes a novel air-cushion sensor for the acquisition of tactile and force information from soft tissue, as it could be useful during robotic-assisted minimally invasive surgery in order to provide the surgeon with tactile and haptic feedback. Advancing recent work on rolling indenters [1], the sensor proposed here makes use of a rigid sphere which is held at the end of a tubular shaft and pressed against the tissue by a stream of air. Variations in tissue stiffness result in movements of the sphere within the shaft and are picked up by an optical system. The new approach allows virtually frictionless motion of the sphere across the tissuepsilas surface and rapid acquisition of tactile information over large areas of soft tissue. The structure and working principles of this new air-cushion tactile sensor are described. Laboratory experiments are conducted to show the feasibility and illustrate the behaviour of the proposed sensor system. The outcome of the conducted experiments shows the potential of the sensor system.

This paper proposes a novel approach for the identification of tissue properties in-vivo using a force sensitive wheeled probe. The purpose of such a device is to compensate a surgeon for a portion of the loss of haptic and tactile feedback experienced during robotic-assisted minimally invasive surgery. Initially, a testing facility for validating the concept ex-vivo was developed and used to characterize two different testing modalities - static (1-DOF) tissue indentation and rolling (2-DOF) tissue indentation. As part of the static indentation experiments a mathematical model was developed to classify tissue condition based on changes in mechanical response. The purpose of the rolling indentation tests was to detect tissue abnormalities, such as tumors, which are difficult to isolate under static testing conditions. During such tests, the test-rig was capable of detecting simulated miniature buried masses at depths of 12mm. Based on these experiments a portable device capable of carrying out similar tests in-vivo was developed. The device was designed to be operated through a trocar port and its key feature is the ability to transition between static indentation and rolling indentation modalities without retracting and changing the tool.

This paper considers a tracked vehicle traversing unknown terrain, and proposes an approach based on the Generalized Newton Raphson (GNR) method for identifying all the unknown soil parameters required for tractive force prediction. For the first time, the methodology, based on measurements of track slip, i, and tractive force, F, to find unknown soil parameters is developed. The tractive force is the force generated by a tracked vehicle to drive itself forwards. This tractive force depends to a large extent on certain soil parameters, namely soil cohesion (c), soil internal friction angle (phi), and soil shear deformation modulus (K). Accurately identifying parameters of the soil on which a tracked vehicle is moving will potentially lead to accurate traversability prediction, effective traction control, and precise trajectory tracking. The soil parameter identification algorithm is validated with the experimental data from Wong [3] and from in-house track- terrain interaction test rig showing good identification accuracy and fast execution speed. It is also shown to be relatively robust to initial condition. The identified soil parameters are, in turn, used to predict the tractive forces showing good agreement with all the experimental data. The technique presented in this paper is general and can be applied to any tracked vehicle.

This paper proposes a novel nonlinear viscoelastic soft tissue model generated from ex vivo experimental results on ovine liver using a force sensitive probe. In order to study the biomechanics of soft tissue, static indentation tests were applied on ovine liver. An empirical constitutive equation was extracted from the examined data. A mechanical model combining linear viscoelasticity with a nonlinear function of strain-stress is proposed. The developed model has been evaluated both statically and dynamically with different strain rates - i.e. where the velocity of indentation is varied. By comparing simulation results and measured experimental data, it has been concluded that the proposed model is robust for modelling both static and dynamic indentation conditions. The effect of changing boundary conditions on the parameters in the proposed model has been studied by choosing test sites with different underlying tissue thicknesses. The results indicate that for small strain, the effect of the thickness condition is reasonable to be neglected.

This paper proposes a novel experimental study of the recovery of soft tissue after the removal of an external load. An optical fiber probe has been developed to measure the tissue recovery precisely through a static indentation method. Ex-vivo soft tissue recovery tests have been conducted on porcine liver. A robotic manipulator is used to control the motion of the probe and the force sensor is used to record the interaction force at the tip of the probe. An empirical mathematical model which describes the relationship of the liver recovery behavior and the holding time during which the probe keeps the tissue statically indented has been developed. The error analysis shows this model is able to predict liver recovery reasonably well. The experimental study shows that the recovery behavior of soft tissue depends on the holding time of the probe. In addition, the formula of the recovering force of the liver, which was deduced from the rate of liver recovery, is proposed.

This paper proposes a novel approach for soil estimation and classification in autonomous excavation exploiting key features from bucket velocity and energy signatures considering the interaction dynamic of the digging process. A real-time energy-based method is proposed for estimating the dynamic friction force arising during soil-tool interaction. The method relies on a novel technique for measuring the force and displacement variables which allows the on-line determination of the bucket velocity and dissipation energy along the full excavation profile. It is shown that these measurements can be effectively used for the on-line identification and classification of different types of soil encountered during the excavation process. The proposed method is insensitive to noise and can be easily implemented in practice using an excavation arm and hydraulic actuators. Various experimental results are presented supporting the practicality of the proposed method.

Closed-circuit television (CCTV) is currently used in many inspection applications, such as the inspection of nonaccessible pipe surfaces. This human-oriented approach based on offline analysis of the raw images is highly subjective and prone to error because of the exorbitant amount of data to be assessed. Laser profilers have been recently proposed to project well-defined light patterns, improving the illumination of standard CCTV systems as well as enhancing the capability of automating the assessment process. This research shows that positional (geometrical) as well as intensity information, related to potential defects, can be extracted from the acquired laser projections. While most researchers focus on the analysis of positional information obtained from the acquired profiler signals, here the intensity information contained within the reflected light is also exploited for the purpose of defect classification and visualization. This paper describes novel strategies created for the automation of defect classification in tubular structures and explores new methods to fuse intensity and positional information, achieving improved multivariable defect classification. The acquired camera/laser images are processed in order to extract signal information for the purpose of visualization and map creation for further assessment. Then, a two-stage approach based on image processing and artificial neural networks is used to classify the images. First, a binary classifier identifies defective pipe sections, and then in a second stage, the defects are classified into different types, such as holes, cracks, and protruding obstacles. Experimental results are provided. Note to Practitioners-The method presented in this paper aims to automate the inspection of nonaccessible pipe surfaces. The method was thought to be employed in the inspection of sewers; however, it could be used in many other industrial applications and could also be extended to other shapes rather than tubular structures. A laser ring profiler, consisting, for instance, of a laser diode and a ring projector, can be easily integrated into existing closed-circuit television systems. The proposed algorithm identifies defective areas and categorizes the types of defects, analyzing the successive recorded camera images that will contain the reflected ring of light. The algorithm, that can be used online, makes use of the deformation of the reflected laser ring together with its changes in intensity. The fact of combining the two kinds of data using artificial-intelligent algorithms makes the method robust enough to work in harsh environments

Nuclear quadrupole resonance (NQR) is a radio-frequency technique that can be used to detect the presence of quadrupolar nuclei, such as the$^{14}{\rm N}$nucleus prevalent in many explosives and narcotics. In a typical application, one observes trains of decaying NQR echoes, in which the decay is governed by the spin echo decay time(s) of the resonant line(s). In most detection algorithms, these echoes are simply summed to produce a single echo with a higher signal-to-noise ratio, ignoring the decaying echo structure of the signal. In this paper, after reviewing current NQR signal models, we propose a novel NQR data model of the full echo train and detail why and how these echo trains are produced. Furthermore, we refine two recently proposed approximative maximum-likelihood detectors that enable the algorithms to optimally exploit the proposed echo train model. Extensive numerical evaluations based on both simulated and measured NQR data indicate that the proposed detectors offer a significant improvement as compared to current state-of-the-art detectors.

This paper presents a novel technique for identifying soil parameters for a wheeled vehicle travelling on an unknown terrain. The identified soil parameters are required for predicting vehicle drawbar pull and wheel drive torque which can be employed for traversability prediction, traction control, and performance optimization of a wheeled vehicle on unknown terrain. The Newton Raphson method is used as the identification technique applied on the modified form of the wheel-soil interaction dynamics model using the composite Simpson's rule. This work focuses on identifying the internal friction angle, the shear deformation modulus, and the lumped pressure-sinkage coefficient. The fourth parameter, cohesion, does not influence the vehicle drawbar pull and is assigned an average value during the identification process. In an experimental study, the identified parameters are compared with known values, and shown to be in good agreement. Soil parameter identification can be carried out on-line and thus our approach is suitable for real-time applications. The robustness of the method is also shown to be relatively good. The identified soil parameters can be used to predict drawbar pull and wheel drive torque with good accuracy

Accurate estimation of slip is essential in developing autonomous navigation strategies for mobile vehicles operating in unstructured terrain. In this paper, a sliding mode observer is firstly constructed to estimate slip parameters based on the kinematics model of a skid-steering vehicle and trajectory measurement. The stability of the sliding mode observer is given in a mathematical context. Slip estimation schemes using an extended Kalman filter and direct mathematical inversion of the kinematic equations are also presented for comparison purposes. It is shown that the non-linear sliding mode observer is more accurate than the other two methods. The robustness and superior performance of the sliding mode observer is demonstrated using both simulation and experimental results. A camera based system is used to measure the vehicle trajectory during experimental validation

Online identification of soil-bucket interaction is significant for the development of an autonomous excavation strategy. This paper presents a method for identifying multiple unknown soil parameters in real-time using the novel Modified Newton Raphson Method. The new approach involving a model selection strategy based on Parallel Execution Model, Flexible Parallel Execution Model and Hybrid Execution Model consisting of the Mohr-Coulomb soil model and the Chen and Liu Upper Bound soil model is examined. The identification of unknown soil parameters is achieved by minimizing the error between the measured forces and the modeled forces computed by the soil model. The results demonstrate that the proposed estimation scheme is accurate when comparing to the measured soil parameters. In addition, the reduced processing time of convergence and high robustness with respect to initial conditions show that the proposed method has real time estimation capability. The proposed method is very promising and highly suitable for soil-bucket interaction identification for an autonomous excavation system in an unpredictable, dynamic environment.

Standard CCTV (close circuit television) is currently used in many pipe inspection applications, such as sewers. This human-based approach is prone to error because of the exorbitant amount of data to be assessed, and smaller anomalies or defects are likely to be overlooked reducing the chance of detection of faults at an early stage. Laser profilers for pipe inspection have been recently proposed to overcome CCTV problems. Positional as well as intensity information, related to potential defects, can be extracted from the laser-camera acquired images. While most of these systems are based on the geometrical analysis of pipes, here the intensity distribution of the reflected light is also exploited. This paper describes the strategies developed for the automation of defect classification in pipes and explores new methods to fuse intensity and positional information and shows how they can be used to improve multi-variable defect classification. A neural network-based classification method is presented. Experimental results are provided.

This paper presents a novel approach for the internal inspection of sewers through the use of sonar techniques, generating enhanced 3D graphs which represent the inner sewer surfaces. These graphs not only show the inner contour of the pipe but also integrate the intensity of the received echoes. The enhanced profile is generated by superimposing the peak intensity from the returning echoes at the calculated x, y and z coordinates where they are reflected from the pipe wall. These coordinates are calculated by measuring the time of flight of the first reflections, which are extracted from consecutives B-mode images generated during the ultrasonic scanning of the pipe. The proposed method has been capable of showing anomalous conditions, inside pipes filled with liquid, with dimensions smaller than the theoretical lateral and axial resolution of the transducer, in contrast to traditional methods where these kinds of defects were not detected. The proposed inspection method and its capabilities were validated through the realization of simulations and experiments. The simulations were conducted to validate the proposed method and explore its limitations. The proposed approach was particularly developed with the aim of scanning internal sections of sewers or water pipes filled with liquid using rotary ultrasonic sonars where visual methods could not be employed. It is expected that this research could also be expanded to the inspection of other submerged structures, such as water tanks, or pressurized vessels.

The work presents the results of production trials of a gap sensing system for continuous extrusion machinery. It is critical to maintain a precise pre-defined extrusion gap for the efficient running of the continuous extrusion process and to maintain product quality. A high temperature capacitive gap sensing system is designed and implemented on a copper extrusion machine in a production plant. The results from the successful gap sensing production trials are presented and the benefits of the gap sensing system are demonstrated. First it is shown that machine setup times prior to production can be reduced from 35-40 minutes to 5 minutes with active gap sensing. The sensors can be used for on-line direct gap measurement and control, and for the first time provide a detailed view of extrusion zone gap behavior during a full production cycle. The gap sensor is used to evaluate the relationship between gap size, and waste levels. It is shown that there is a linear reduction of waste levels from 20% to 2%, when the gap size is reduced from 1 mm to 0.15 mm.

Real time estimation for soil-tool interaction is a key method for the development of an autonomous excavation strategy. This paper presents a method for identifying the unknown soil parameters in real-time using a novel hybrid soil model. The hybrid models consist of the Mohr-Coulomb soil model and the Chen and Liu Upper Bound soil model. A switching mode is utilized to select a more accurate soil model to compute the failure force depending on the position of the excavator bucket. The Newton Raphson method is proposed to identify the soil parameters by minimizing the error between the experimental forces and the forces computed by the hybrid soil model. The results demonstrate that the proposed estimation scheme is accurate when comparing to the measured soil parameters and the experimental data. In addition, the high speed estimation time shows that the proposed method has a real time estimation capability. A very high robustness is achieved when compared to the least square method. The proposed method is very promising and highly suitable for the soil-tool interaction identification of an autonomous excavator in an unpredictable, dynamical and potentially hazardous environment.

A robust, fast, and simple technique for the experimental identification of the link parameters (mass, inertia, and length) and friction coefficients of a full-scale excavator arm is presented. This new technique, based on the generalized Newton method (GNM), estimates unknown individual parameters of the excavator arm dynamic equations. The technique can be used when the number of equations is different from the number of estimated variables. Using experimental data from a full-scale field Combat Engineer Excavator (CEE), the values of link parameters and friction coefficients are successfully identified. The identified parameters are compared with known values, and shown to be in agreement. The method is compared with the least square method, and shows that the GNM is better in terms of prediction accuracy and robustness to noise. Further, the joint positions predicted by the analytical model using the identified parameters are validated against different experimental trajectories, showing very good agreement. The experimental data was obtained in collaboration with QinetiQ Ltd. (Hampshire, U.K.). The technique presented in this paper is general and can be applied to any manipulator.

Pipe inspection, for many industrial applications, is commonly carried out using CCTV (closed-circuit TV) cameras and off-line human analysis via raw image examination for defect detection. Researchers have proposed the use of additional sensors and lighting systems to create ring profiles onto pipe segments. Geometrical changes in the profile are analyzed to determine pipe deformation. However the light intensity information has not been considered in such works. In this paper we propose an intensity-based profiler system. The sensor consists of a laser profiler that is attached to a standard CCTV system. The physical sensor behavior is described using reflectance theory and experimental data. Finally, an image of the pipe wall is generated by extracting the intensity information existing in the pipe pictures. Defects and anomalies can be identified using this extracted image.

This paper presents the experimental results of an automated sensor system for the inspection of tubular structures. The method is applied to the autonomous inspection of sewers overcoming the drawbacks of standard CCTV-based inspection systems. The transducer consists of a low-cost laser-based profiler attached to a standard CCTV camera. Image analysis techniques and artificial neural networks are used to automatically locate and classify the defects in the pipe using the intensity distribution in the acquired camera images. A wide range of tests using data from different types of pipes in realistic conditions have been conducted and are presented here. It is shown that the proposed inspection approach is particularly well suited to complement existing CCTV inspection systems, providing automated and reliable detection of pipe defects in the millimeter range.

This paper presents an innovative approach for the development of ultrasonic solutions for sewer inspection, through the creation of a simulation tool capable of generating simulated images from pipes filled with water. The developed tool is capable of creating two dimensional and three dimensional surface scans of pipes simulating rotational sonar scanners submerged in water. Pipe deformations and anomalous conditions can be simulated. The properties of the simulation tool and the comparison of simulations against experimental tests, to validate its capability and accuracy to generate ultrasound images, are presented. Additionally, an optimization approach used to reduce the processing time to run the simulations, and a method to create 3D surfaces from B-mode images based on an edge detection method are described. The developed simulation routines represent a valuable tool for the development and design of pipe inspection systems and operator training.

This paper presents a novel method for estimating soil properties on-line during excavation tasks such as ground leveling, digging and sheet pilling. The proposed method computes key soil parameters by measuring the forces acting on the excavator bucket whilst being in contact with the soil and minimizing the error between measured forces and estimated forces produced by a real-time capable soil model. Two soil models, the Mohr-Coulomb soil model and the Chen and Liu upper bound soil model, are implemented and researched in the context of this estimation scheme. Parameter optimization is carried out employing the Newton Raphson method. The method is evaluated using experimental data and through comparison with an approach that makes use of graphical intersection for model optimization. The results demonstrate that the proposed Newton Raphson-based method is as accurate as the graphical intersection-based approach, but up to 2000 times faster, and thus, most suitable for on-line soil parameter estimation in an automated system which provides optimized digging trajectories for a given excavation task.

A robust and fast yet simple approach for experimental identification of the link (mass, inertia and length) parameters and friction coefficients for a full-scale closed-chain excavator arm is developed. The approach is based on a generalized Newton method, an excavation arm dynamic model and measured data. The new approach can be used where the number of equations is different from the number of variables, or if the Jacobian cannot be assumed nonsingular. The parameters are needed for improving the control actions in autonomous solution for excavation tasks and contributing to excavator design evaluation. Using experimental data obtained while moving the links of an instrumented full-scale combat engineer excavator, the values of the link parameters and friction coefficients for various links were successfully identified. The identified parameters are compared with physical values, and they are in agreement. Further, the joint torques and positions computed by the developed model using the identified parameters are validated against measured data, also showing excellent agreement. The experimental data was obtained in collaboration with QinetiQ. The novel technique presented in this paper is general and can be applied to a wide range of heavy-duty closed chain hydraulic manipulators.

Threaded fastenings are a common assembly method, and account for over a quarter of all assembly operations. These operations are very difficult to automate, with threaded to align and position the screw with respect to the hole, and to apply axial torque until the appropriate tightening forces are achieved. Screw insertions are typically carried out manually with the purpose of joining one component to another. They permit easy disassembly for maintenance, repair, and relocation. There is little research reported on automating screw insertions, with most automated assembly research focusing on the peg-in-hole assembly problem. This paper presents an on-line parameter estimation strategy employing the Newton Raphson method (NRM) for threaded fastenings. The motivation for the study is to employ the estimated parameters to develop strategies for monitoring threaded fastenings. The monitoring problem deals with predicting the integrity of self-tapping screw insertion process, based on the torque vs. insertion angle curves generated during the insertions. A technique for estimating three unknown parameters (friction coefficient, hole diameter, and screw major diameter) during a typical screw insertion is presented.

An innovative inspection method to assess the condition of sewer pipes is proposed in this paper. The standard sewer inspection technique, based on closed-circuit television systems, has a relatively poor performance; a video camera is mounted on a robot and the video recording is provided off-line to an engineer who classifies any defects. The focus of this research is the automated identification and location of discontinuities in the internal surface of sewers. The transducer used is an assembly of a CCD camera and optical elements to generate a ring-shaped laser pattern. The automated inspection method consists of several stages including the segmentation of the image into characteristic geometric features and potential defect regions. Automatic recognition, rating and classification of pipe defects are carried out by means of the computation of a partial histogram based on adaptive image processing techniques. Experiments in a realistic environment have been conducted and results are presented.

The backpropagation (BP) algorithm is commonly used in many applications, including robotics, automation and weight changes of artificial neural networks (ANNs). This paper proposes the addition of an extra term, a proportional factor (PF), to the standard BP algorithm to speed up the weight adjusting process. The proposed algorithm is tested and the results show that the proposed algorithm outperforms the conventional BP algorithm in convergence speed and the ability to escape from learning stalls. The paper presents a convergence analysis of the three-term BP algorithm. It is shown that if the learning parameters of the three-term BP algorithm satisfies certain conditions given in this paper, then it is guaranteed that the system is stable and will converge to a local minimum. The paper shows that all the local minima of the cost function are stable for the three-term backpropagation algorithm.

Standard sewer inspection systems are based on closed circuit television (CCTV) cameras mounted on wheeled platforms. One of the disadvantages of camera inspection systems is that they can detect only a small part of all possible sewer damage that could conclude in collapses. The inspection outcome of standard CCTV systems relies not only on the quality of the acquired images, but also on the off-line recognition and classification conducted by human operators. The objective of this research is the development of intelligent sensor systems that will enable the automation of the pipe condition assessment. Optical techniques are proposed to complement the existing CCTV-based approach and to improve inspection results. Besides that, automated defect recognition algorithms based on Artificial Neural Networks are proposed. Experiments to test the tolerance of the automated. algorithm to artificially-generated noise have been conducted and results are presented.

This paper presents an integrated physics based model for a front end mobile excavator. The model describes the dynamic relationship between the operator input commands and the position, orientation, speed and forces of the vehicle and the excavation arm. The dynamic model has the potential to be used in advanced controller design for automated excavation systems. The dynamic model for the excavation system is validated against measured data. The validation of the model is conducted in collaboration with QinetiQ, UK. A PID controller for trajectory tracking is implemented and tested using a computer simulation study. The graphical machine model is developed in the Zed3D graphical environment and all the inputs to the graphical model are taken from the simulation model.

The automation of screw insertions is a highly desirable task. An important part of the automation process is the monitoring of the insertion. The paper presents an application of artificial neural networks for monitoring this common manufacturing procedure. The research focuses on the insertion of self-tapping screws. A radial basis artificial neural network is employed to distinguish between successful and failed insertions. The network is tested with tasks of increasing complexity using simulation data. The approach is then validated with the use of experimental data, and the tests results are presented.

The automation of screw insertions represents a highly desirable task. An important part of the automation process is the monitoring of the insertion, The paper presents an application of artificial neural networks for monitoring this common manufacturing procedure. The research focuses on the insertion of self-tapping screws. Artificial neural networks have been employed to distinguish between successful and failed insertions. The networks under investigation use radial basis functions for the computation of the data. A range of networks, differing in size, has been implemented and thoroughly tested. Results and evaluations of the networks from the experiments are presented.

A weightless neural network based intelligent monitoring strategy for automated self-tapping screw insertions is presented. Problems encountered with automated screw insertion workstations include screw jamming, thread stripping and cross threading. If such problems are not detected early, this could lead to defective assemblies. A weightless neural network is designed and trained to monitor automated screw fastenings. The network is first trained and tested using computer simulations. An experimental test rig is constructed and the weightless neural network is tested using both seen and unseen cases. Experimental results are presented to confirm the effectiveness of the approach.

This paper describes a novel navigation and obstacle avoidance system for robotic manipulators. The system is divided into separate fuzzy units which individually control the links of a manipulator. The rule base of each unit combines the repelling influence of obstacles with the attracting influence of the target position in a fuzzy way to generate actuating commands for the link. Owing to its simplicity and hence its short response time, the fuzzy navigator is especially suitable in on-line applications with strong real-time requirements. Furthermore, this approach allows obstacle avoidance in dynamic environments. The functioning of the fuzzy navigator with respect to robotic manipulators and results of real-world experiments are presented.