Over the past decade, significant advancements have been made in the research and industrialization of robotic systems for endovascular procedures, yet their clinical application remains relatively limited. Physicians commonly report that these robots lack certain intelligent assistive capabilities during procedures. There has been increasing interest and attempts to apply learning-centered algorithms to the training and enhancement of surgical robot skills. This paper proposes an autonomous navigation algorithm for interventional guidewires that is initially trained solely in a virtual simulation environment and subsequently deployed to a real-world robot. Experimental results demonstrate the feasibility of this approach for real-world applications. The proposed approach can help physicians reduce the learning curve for guidewire manipulation and elevate the robot to a higher level of autonomous operation, thereby breaking through the current bottleneck in the level of intelligence for clinical applications of interventional robots. It also holds promise for bringing intelligent transformation to future interventional procedures. Note to Practitioners—This work is motivated by the emerging need to increase the level of autonomy in robotic-assisted endovascular procedures, which has the potential to improve procedural efficiency, standardize procedures, and broaden the adoption of robotic systems in clinical practice. The proposed simulation-based reinforcement learning provides a safe and efficient method for training robotic systems, enabling them to master complex tasks in simulation environments prior to real-world application. The successful deployment of models trained in simulation onto physical robotic platforms demonstrates the feasibility of this method for real-world applications. The proposed simulation-based reinforcement learning method offers a promising and viable pathway for enhancing skill acquisition in endovascular interventional robots.

Moving object segmentation is critical to interpret scene dynamics for robotic navigation systems in challenging environments. Neuromorphic vision sensors are tailored for motion perception due to their asynchronous nature, high temporal resolution, and reduced power consumption. However, their unconventional output requires novel perception paradigms to leverage their spatially sparse and temporally dense nature. In this work, we propose a novel event-based motion segmentation algorithm using a Graph Transformer Neural Network, dubbed GTNN. Our proposed algorithm processes event streams as 3D graphs by a series of nonlinear transformations to unveil local and global spatiotemporal correlations between events. Based on these correlations, events belonging to moving objects are segmented from the background without prior knowledge of the dynamic scene geometry. The algorithm is trained on publicly available datasets including MOD, EV-IMO, and EV-IMO2 using the proposed training scheme to facilitate efficient training on extensive datasets. Moreover, we introduce the Dynamic Object Mask-aware Event Labeling (DOMEL) approach for generating approximate ground-truth labels for event-based motion segmentation datasets. We use DOMEL to label our own recorded Event dataset for Motion Segmentation (EMS-DOMEL), which we release to the public for further research and benchmarking. Rigorous experiments are conducted on several unseen publicly-available datasets where the results revealed that GTNN outperforms state-of-the-art methods in the presence of dynamic background variations, motion patterns, and multiple dynamic objects with varying sizes and velocities. GTNN achieves significant performance gains with an average increase of 9.4% and 4.5% in terms of motion segmentation accuracy (IoU%) and detection rate (DR%), respectively.

Underwater robotic vision encounters numerous challenges posed by complex environments and varying lighting conditions. Meeting these challenges requires solutions that not only deliver accuracy but also demonstrate adaptability. In this paper, we present MARS (Multi-Scale Adaptive Robotics Vision), a pioneering approach to underwater object detection that prioritizes real-time performance and resource efficiency-essential attributes for underwater robotic systems. Leveraging a well-established object detection architecture, MARS integrates Domain-Adaptive Multi-Scale Attention (DAMSA), enhancing both detection accuracy and adaptability to diverse underwater domains. During training, DAMSA employs domain class-based attention, allowing the model to learn and prioritize features specific to different underwater environments. Extensive evaluation across diverse underwater datasets underscores the effectiveness of MARS. On the original dataset, MARS achieves an impressive mean Average Precision (mAP) of58.57%, demonstrating its proficiency in detecting critical underwater objects such as echinus, starfish, holothurian, scallop, and waterweeds. This capability positions MARS as a promising solution for applications in marine robotics, marine biology research, and environmental monitoring. Moreover, MARS exhibits exceptional resilience to domain shifts. When evaluated on an augmented dataset incorporating various enhancements, MARS delivers a commendable mAP of 36.16%, showcasing its robustness and adaptability in recognizing objects across diverse underwater conditions. The source code for MARS is publicly available on GitHub at https://github.com/LyesSaadSaoud/MARS-Object-Detection/.

The difficulties posed by terrain-induced slippage for wheeled rovers traversing soft terrains are critical to ensuring safe and precise mobility. While bilateral teleoperation systems offer a promising solution to this issue, the inherent network-induced delays hinder the fidelity of the closed-loop integration, potentially compromising teleoperator system controls, and resulting in poor command-tracking performance. This work introduces a new model-free predictor framework based on deep learning designed to improve prediction performance and effectively compensate for large network delays in teleoperated wheeled rovers. Our approach employs the Recurrent Neural Network (RNN) to achieve a significant improvement in modeling complexity and prediction accuracy. Particularly, our framework consists of two distinct predictors, each tailored to the forward and backward coupling variables of the teleoperated wheeled rover. Human-in-the-loop experiments were conducted to validate the effectiveness of the developed framework in compensating for the delays encountered by teleoperated wheeled rovers coupled with terrain-induced slippage. The results confirm the improved prediction accuracy of the framework. This improvement is evidenced by improved performance and transparency metrics, which lead to better command-tracking performance. A supplementary video is available at https://youtu.be/-06UGumQ0tA.

This research addresses the pressing challenge of weed identification in agriculture, crucial for ensuring food security in anticipation of a global population exceeding 9.7 billion by 2050. Utilizing drone imagery, we collected a dataset and proposed a customized model to achieve optimal performance. Our proposed model uses strategically modified backbone, neck, and head components, leveraging elements such as Ghost Convolution, BottleNeckCSP, and ECA (Efficient Channel Attention) layers. These modifications enhance the model’s capability to discern intricate patterns in drone imagery, ultimately leading to improved precision in weed detection. We introduce a purposefully crafted dataset to complement the model’s training, and our experiments demonstrate superior performance compared to the baseline models. Our model achieves a precision of 72.5%, recall of 68.0%, and [email protected] of 73.9, showcasing the effectiveness of our approach in enhancing detection accuracy. Leveraging a unique blend of feature extraction mechanisms, our model achieves remarkable accuracy in real-time soybean detection, outperforming established models like RT-DETR (Real-Time DEtection TransfoRmer) and YOLOv10. A detailed ablation study and comparative analysis with different YOLO versions and the transformer-based RT-DETR showcase the effectiveness of the proposed enhancements. Our work signifies a significant step towards advancing the field of precision agriculture, offering a model that is not only adaptive but also robust in identifying and localizing weeds in soybean fields.

Event cameras triggered a paradigm shift in the computer vision community delineated by their asynchronous nature, low latency, and high dynamic range. Calibration of event cameras is always essential to account for the sensor intrinsic parameters and for 3D perception. However, conventional image-based calibration techniques are not applicable due to the asynchronous, binary output of the sensor. The current standard for calibrating event cameras relies on either blinking patterns or event-based image reconstruction algorithms. These approaches are difficult to deploy in factory settings and are affected by noise and artifacts degrading the calibration performance. To bridge these limitations, we present E-Calib, a novel, fast, robust, and accurate calibration toolbox for event cameras utilizing the asymmetric circle grid, for its robustness to out-of-focus scenes. E-Calib introduces an efficient reweighted least squares (eRWLS) method for feature extraction of the calibration pattern circles with sub-pixel accuracy and robustness to noise. In addition, a modified hierarchical clustering algorithm is devised to detect the calibration grid apart from the background clutter. The proposed method is tested in a variety of rigorous experiments for different event camera models, on circle grids with different geometric properties, on varying calibration trajectories and speeds, and under challenging illumination conditions. The results show that our approach outperforms the state-of-the-art in detection success rate, reprojection error, and pose estimation accuracy.

Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer vision community and is serving as a key enabler for a wide range of applications. This technology has offered significant advantages, including reduced power consumption, reduced processing needs, and communication speedups. However, neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms. In this article, we propose a novel noise filtration algorithm to eliminate events that do not represent real log-intensity variations in the observed scene. We employ a graph neural network (GNN)-driven transformer algorithm, called GNN-Transformer, to classify every active event pixel in the raw stream into real log-intensity variation or noise. Within the GNN, a message-passing framework, referred to as EventConv, is carried out to reflect the spatiotemporal correlation among the events while preserving their asynchronous nature. We also introduce the known-object ground-truth labeling (KoGTL) approach for generating approximate ground-truth labels of event streams under various illumination conditions. KoGTL is used to generate labeled datasets, from experiments recorded in challenging lighting conditions, including moon light. These datasets are used to train and extensively test our proposed algorithm. When tested on unseen datasets, the proposed algorithm outperforms state-of-the-art methods by at least 8.8% in terms of filtration accuracy. Additional tests are also conducted on publicly available datasets (ETH Zürich Color-DAVIS346 datasets) to demonstrate the generalization capabilities of the proposed algorithm in the presence of illumination variations and different motion dynamics. Compared to state-of-the-art solutions, qualitative results verified the superior capability of the proposed algorithm to eliminate noise while preserving meaningful events in the scene.

Visual perception is an important component for autonomous navigation of unmanned surface vessels (USV), particularly for the tasks related to autonomous inspection and tracking. These tasks involve vision-based navigation techniques to identify the target for navigation. Reduced visibility under extreme weather conditions in marine environments makes it difficult for vision-based approaches to work properly. To overcome these issues, this paper presents an autonomous vision-based navigation framework for tracking target objects in extreme marine conditions. The proposed framework consists of an integrated perception pipeline that uses a generative adversarial network (GAN) to remove noise and highlight the object features before passing them to the object detector (i.e., YOLOv5). The detected visual features are then used by the USV to track the target. The proposed framework has been thoroughly tested in simulation under extremely reduced visibility due to sandstorms and fog. The results are compared with state-of-the-art de-hazing methods across the benchmarked MBZIRC simulation dataset, on which the proposed scheme has outperformed the existing methods across various metrics.

Vision-based control of unmanned aerial vehicles (UAVs) has been adopted in a wide range of applications due to the availability of low-cost onboard sensors and computers. Tuning such systems to work properly requires extensive domain-specific experience, which limits the growth of emerging applications. Moreover, obtaining performance limits for UAVs performing visual servoing is difficult due to the complexity of the models used. In this article, we propose a novel noise-tolerant approach for real-time identification and tuning of visual servoing systems. This is based on the deep neural networks (DNNs) classification of the system response generated by the modified relay feedback test (MRFT). The proposed method, called DNN with noise-protected MRFT (DNN-NP-MRFT), can be used with a multitude of vision sensors and estimation algorithms despite high levels of sensor noise. The response of DNN-NP-MRFT to noise perturbations is investigated and its effect on identification and tuning performance is analyzed. The proposed DNN-NP-MRFT is able to detect performance changes induced by the use of high latency vision sensors or by integrating an inertial measurement unit sensor into the UAV states estimation pipeline. Experimental identification closely matches simulation results, which can be used to explain the system behavior and predict the closed-loop performance limits for a given hardware and software setup. We also demonstrate the ability of DNN-NP-MRFT-tuned UAVs to reject external disturbances like wind or human push and pull. Finally, we discuss the advantages of the proposed DNN-NP-MRFT visual servoing design approach compared with other approaches in the literature.

Visual object tracking is a fundamental and challenging task in many high-level vision and robotics applications. It is typically formulated by estimating the target appearance model between consecutive frames. Discriminative correlation filters (DCFs) and their variants have achieved promising speed and accuracy for visual tracking in many challenging scenarios. However, because of the unwanted boundary effects and lack of geometric constraints, these methods suffer from performance degradation. In the current work, we propose hierarchical spatiotemporal graph-regularized correlation filters for robust object tracking. The target sample is decomposed into a large number of deep channels, which are then used to construct a spatial graph such that each graph node corresponds to a particular target location across all channels. Such a graph effectively captures the spatial structure of the target object. In order to capture the temporal structure of the target object, the information in the deep channels obtained from a temporal window is compressed using the principal component analysis, and then, a temporal graph is constructed such that each graph node corresponds to a particular target location in the temporal dimension. Both spatial and temporal graphs span different subspaces such that the target and the background become linearly separable. The learned correlation filter is constrained to act as an eigenvector of the Laplacian of these spatiotemporal graphs. We propose a novel objective function that incorporates these spatiotemporal constraints into the DCFs framework. We solve the objective function using alternating direction methods of multipliers such that each subproblem has a closed-form solution. We evaluate our proposed algorithm on six challenging benchmark datasets and compare it with 33 existing state-of-the art trackers. Our results demonstrate an excellent performance of the proposed algorithm compared to the existing trackers.

Visual Object Tracking (VOT) is an essential task for many computer vision applications. VOT becomes challenging when a target object faces severe occlusion, drastic illumination changes, and scale variation problems. In the literature, Discriminative Correlation Filters (DCFs)-based tracking methods have achieved promising results in terms of accuracy and efficiency in many complex VOT scenarios. A plethora of DCFs trackers have been proposed which exploit information observed in past frames to create and update DCFs for VOT. To adapt to target appearance variations, the DCFs are enhanced by incorporating spatial and temporal consistency constraints. Nevertheless, the performance degradation is observed for these methods because of the aforementioned limitations. To address these issues, we propose a novel algorithm based on bidirectional DCFs for VOT. In this algorithm, we propose the original idea of leveraging information from both past and future frames. The proposed algorithm first tracks the target object forward in the video sequence and then its uses the predicted location of the last window frame and track the target object backward towards the current frame. We design an appearance consistency loss function by taking the$L_{2}$norm between the regression target of the forward tracking and response map of the backward tracking to obtain the resulting response map. Our proposed algorithm realizes a highly accurate DCFs because forward and backward tracking information are fused together for consistent VOT. Although, a result will be output with some small delay because information is taken from a future to the present period, our proposed algorithm has the merit of addressing the drastic appearance variations VOT challenges. We evaluate our proposed tracker using deep features on three publicly available challenging datasets. Our results demonstrate the superior performance of the proposed tracker compared to the existing state-of-the-art trackers.

Flagellated micro-organism are regarded as excellent swimmers within their size scales. This, along with the simplicity of their actuation and the richness of their dynamics makes them a valuable source of inspiration to design continuum, self-propelled underwater robots. Here we introduce a soft, flagellum-inspired system which exploits the compliance of its own body to passively attain a range of geometrical configurations from the interaction with the surrounding fluid. The spontaneous formation of stable helical waves along the length of the flagellum is responsible for the generation of positive net thrust. We investigate the relationship between actuation frequency and material elasticity in determining the steady-state configuration of the system and its thrust output. This is ultimately used to perform a parameter identification procedure of an elastodynamic model aimed at investigating the scaling laws in the propulsion of flagellated robots.

In this paper, a geometric and unified model of soft-rigid multi-body systems is presented, based on a discrete Cosserat approach of the soft-body dynamics. The model is in fact a generalization to soft and hybrid systems of the geometric theory of rigid robotics characterized by the exponential map. A generalization of the recursive Newton-Euler algorithm is also presented, able to solve inverse and forward dynamic problems with linear O(N) complexity. The proposed model provides several improvements with respect to the existing flexible multi-body models, which make it particularly suitable to study the dynamics of modern soft robots as shown for a multi-body system inspired by motile bacteria.

In this paper, we introduce a coverage path planning algorithm with adaptive viewpoint sampling to construct accurate 3D models of complex large structures using Unmanned Aerial Vehicle (UAV). The developed algorithm, Adaptive Search Space Coverage Path Planner (ASSCPP), utilizes an existing 3D reference model of the complex structure and the onboard sensors' noise models to generate paths that are evaluated based on the traveling distance and the quality of the model. The algorithm generates a set of viewpoints by performing adaptive sampling that directs the search towards areas with low accuracy and low coverage. The algorithm predicts the coverage percentage obtained by following the generated coverage path using the reference model. A set of experiments were conducted in real and simulated environments with structures of different complexities to test the validity of the proposed algorithm.

Nowadays, the most adopted model for the design and control of soft robots is the piecewise constant curvature model, with its consolidated benefits and drawbacks. In this work, an alternative model for multisection soft manipulator dynamics is presented based on a discrete Cosserat approach, in which the continuous Cosserat model is discretized by assuming a piecewise constant strain along the soft arm. As a consequence, the soft manipulator state is described by a finite set of constant strains. This approach has several advantages with respect to the existing models. First, it takes into account shear and torsional deformations, which are both essential to cope with out-of-plane external loads. Furthermore, it inherits desirable geometrical and mechanical properties of the continuous Cosserat model, such as intrinsic parameterization and greater generality. Finally, this approach allows to extend to soft manipulators, the recursive composite-rigid-body and articulated-body algorithms, whose performances are compared through a cantilever beam simulation. The soundness of the model is demonstrated through extensive simulation and experimental results.

The best way for researchers to test their algorithms and design concepts, before experimenting on real human beings, is to create simulation environment platforms. In this paper, a virtual simulator for active endoscopic capsules is proposed. The simulator intends to provide researchers with an environment to test their vision and navigation algorithms applied to endoscopic capsule applications. The proposed simulation was created using Gazebo simulator, , a robust physics engine under Robotic Operating System (ROS) environment. It consists of three main software modules: (i) capsule model, (ii) capsule control, and (iii) Gazebo customized plugins. The current version of the simulator can provide three main functions: lumen tracking, capsule tele-operation and haptic feedback for capsule navigation.

In this paper we present the design and characterization of a novel Passive Variable Stiffness Joint (pVSJ). pVSJ is the proof of concept of a passive revolute joint with controllable variable stiffness. The current design is intended to be a bench-test for future development towards applications in haptic teleoperation purposed exoskeletons. The main feature of the pVSJ is its capability of varying the stiffness with infinite range based on a simple mechanical system. Moreover, the joint can rotate freely at the zero stiffness case without any limitation. The stiffness varying mechanism consists of two torsional springs, mounted with an offset from the pVSJ rotation center and coupled with the joint shaft by an idle roller. The position of the roller between the pVSJ rotation center and the spring's center is controlled by a linear sliding actuator fitted on the chassis of the joint. The variation of the output stiffness is obtained by changing the distance from the roller-springs contact point to the joint rotation center (effective arm). If this effective arm is null, the stiffness of the joint will be zero. The stiffness increases to reach high stiffness values when the effective arm approaches its maximum value, bringing the roller close to the torsional springs' center. The experimental results matched with the physical-based modeling of the pVSJ in terms of stiffness variation curve, stiffness dependency upon the springs' elasticity, joint deflection and the spring's deflection.

Modeling and control of soft robots is an up-to-date and exciting area of research which has been tackled with complementary approaches so far. In this paper, we modify the existing continuum Cosserat approach optimizing it for soft robot arms which can be discretized in a finite number of sections and degrees of freedom. The resulting new piece-wise constant strain model extends the existing piece-wise constant curvature model by allowing torsion and shears strains which are fundamental to cope with out-of-the-plane external forces as appearing for example during ground locomotion. A first experimental comparison has been also conducted using one fluidic actuated leg of the soft crawler FASTT.

In this paper, torque reflection based coordination control algorithm is designed for network-based bilateral shared autonomous system over open communication networks. The control algorithm for master and slave manipulator is designed by combining delayed position and velocity signal with the delayed reflected torques from the interaction between human and master and between slave and environment. Robust and adaptive control technique is used to deal with uncertainty associated with the gravity, unmodeled dynamic and other external input disturbance. The convergence of the closed loop system is shown by using Lyapunov method. In contrast with existing force reflection based design, the proposed controller can deal with uncertainty associated with the gravity, unmodeled dynamic and external input disturbance. Compared with other methods, the proposed design uses reflected torques from the interaction between master and human and between slave and environment so as to improve the transparency of the bilateral shared autonomous system. Finally, evaluation results are presented to demonstrate the validity of the proposed design for real-time applications.

In this paper, we deal with the stability and tracking control problem of quadrotor unmanned aerial vehicle (UAV) in the presence of the modeling error and disturbance uncertainty. The flight tracking control system combines classical proportional-derivative (PD) like term with robust and adaptive control term. Lyapunov method is used to design and show the asymptotic behavior of the linear and angular states of the vehicle. In contrast with other existing adaptive backstepping design, the proposed design is very simple and easy to implement as it does not require multiple design steps without using augmented signals and known bound of the uncertainty. Various experimental results on quadrotor UAV system are presented to demonstrate the effectiveness of the proposed design for real-time application.

The absence of touch of sense is a widely known drawback of robotic minimally invasive surgery (MIS). This paper proposes a design of optic soft tactile arrays which is promising to be adapted for MIS. The proposed design consists of multiple soft material channels. Each channel is designed using the Bernoulli pipe structure to amplify the sensor's sensitivity through input and output diameter difference. A multi-core optic fiber cable and a camera are used to capture the change of light intensity caused by the contact forces applied onto the individual soft material channels. The proposed sensor has the following advantages: 1) making use of 3D printing and soft material casting, it is suitable for designing sensors with high density of tactile elements; 2) it also allows the sensor to be designed in an arbitrary shape to fit various MIS applications; 3) compared to other light-intensity based tactile sensor, it is easy to fabricate and miniaturize; it avoids the complexity of attaching reflectors to individual sensing elements; 4) it is immune to electromagnetic interference. In this paper, a prototype which has 3×3 tactile elements in an area of 9.5 × 11 mm2has been developed and test for feasibility study. Also, a noise-filtering algorithm is developed to reduce the imaging noise. Validation experiments were carried out and results show that the average measurable force range for a single tactile element is 0 to 1.622N with an average accuracy of 97%. The sensor has low crosstalk-to-signal ratio, 1.8% on average, and has no signal drift over time.

To efficiently explore a surface using the sense of touch, a novel contact sensing finger was created and a surface following control algorithm for the finger was devised. Based on the accurate estimation of contact locations, and the direction and magnitude of the normal and tangential forces, the finger can robustly and rapidly follow surfaces with large change in curvature while maintaining a desired constant normal force. In this paper, the design and testing of the contact sensing finger are presented and the control algorithm for surface contour following is proposed and validated using objects with different shapes and surface materials. The results demonstrate that using the developed finger and the control algorithm, a surface can be efficiently explored with rapid sliding speed. To demonstrate the potential applications of the proposed approach, the friction properties of an explored object surface are computed and, for a known object, its pose is estimated.

Current surgical tele-manipulators do not provide explicit haptic feedback during soft tissue palpation. Haptic information could improve the clinical outcomes significantly and help to detect hard inclusions within soft-tissue organs indicating potential abnormalities. However, system instability is often caught by direct force feedback. In this paper, a new approach to intra-operative tumor localization is introduced. A virtual-environment tissue model is created based on the reconstructed surface of a soft-tissue organ using a Kinect depth sensor and the organ's stiffness distribution acquired during rolling indentation measurements. Palpation applied to this tissue model is haptically fed back to the user. In contrast to previous work, our method avoids the control issues inherent to systems that provide direct force feedback. We demonstrate the feasibility of this method by evaluating the performance of our tumor localization method on a soft tissue phantom containing buried stiff nodules. Results show that participants can identify the embedded tumors; the proposed method performed nearly as well as manual palpation.

Robotic manipulators for Robot-assisted Minimally Invasive Surgery (RMIS) pass through small incisions into the patient's body and interact with soft internal organs. The performance of traditional robotic manipulators such as the da Vinci Robotic System is limited due to insufficient flexibility of the manipulator and lack of haptic feedback. Modern surgical manipulators have taken inspiration from biology e.g. snakes or the octopus. In order for such soft and flexible arms to reconfigure itself and to control its pose with respect to organs as well as to provide haptic feedback to the surgeon, tactile sensors can be integrated with the robot's flexible structure. The work presented here takes inspiration from another area of biology: cucumber tendrils have shown to be ideal tactile sensors for the plant that they are associated with providing useful environmental information during the plant's growth. Incorporating the sensing principles of cucumber tendrils, we have created miniature sensing elements that can be distributed across the surface of soft manipulators to form a sensor network capable of acquire tactile information. Each sensing element is a retractable hemispherical tactile measuring applied pressure. The actual sensing principle chosen for each tactile makes use of optic fibres that transfer light signals modulated by the applied pressure from the sensing element to the proximal end of the robot arm. In this paper, we describe the design and structure of the sensor system, the results of an analysis using Finite Element Modeling in ABAQUS as well as sensor calibration and experimental results. Due to the simple structure of the proposed tactile sensor element, it is miniaturisable and suitable for MIS. An important contribution of this work is that the developed sensor system can be ”loosely” integrated with a soft arm effectively operating independently of the arm and without affecting the arm's motion during bending or elongation.

This paper presents a novel technique for estimating stiffness distribution of a soft tissue using a prototype of air-float stiffness probe. The air-float stiffness probe uses an indentation technique to estimate tissue stiffness. It consists of a spherical indenter and an indentation depth sensing mechanism that operates under a supply of compressed air. The probe has the ability to estimate tissue stiffness in non-planner tissue profiles. A novel technique to estimate indentation force, using supply air pressure is described and validated using both experimental procedures and finite element analysis (FEA) techniques. FEA package, ANSYS CFX was used for analyzing in 2D the solid-fluid interactions within the probe to estimate force available at the indenter at different supply air pressure settings. Both the experimental results and numerical simulations suggest that there is a very strong linear correlation between the indentation force and the supply air pressure. This relationship is used to estimate the indentation force in real time during an indentation test. Verification tests carried out on simulated silicon samples showed that the probe is capable of estimating tissue stiffness values with high accuracy and repeatability.

In this paper, a complete system identification process for identifying flight dynamics model of MAVs (miniature aerial vehicles) is presented. CIFER identification toolkit, which is developed by NASA Ames research center and particularly suitable for identifying rotary-wing aircraft dynamics, is adopted. The modeling procedure is detailed by addressing the following four key steps: 1) data collection and processing based on frequency-sweep input form, 2) parameter identification in frequency domain by minimizing the cost function, 3) result analysis based on the frequency responses matching, the Cramer-Rao Bound, and the Insensitivity, and 4) model fidelity validation in time domain.

This paper introduces a novel continuum-style robot that integrates multiple layers of compliant modules. Its essential features lie in that its bending is not based on natural compliance of a continuous backbone element or soft skeletal elements but instead is based on the compliance of each structured planar module. This structure provides several important advantages. First, it demonstrates a large linear bending motion, whilst avoiding joint friction. Second, its contraction and bending motion are decoupled. Third, it possesses ideal back-drivability and a low hysteresis. We further provide an analytical method to study the compliance characteristics of the planar module and derive the statics and kinematics of the robot. The paper provides an overview of experiments validating the design and analysis.

The break-away friction ratio (BF-ratio), which is the ratio between friction force and the normal force at slip occurrence, is important for the prediction of incipient slip and the determination of optimal grasping forces. Conventionally, this ratio is assumed constant and approximated as the static friction coefficient. However, this ratio varies with acceleration rates and force rates applied to the grasped object and the object material, which lead to difficulties in determining optimal grasping forces that avoid slip. In this paper, we propose a novel approach based on the interactive forces to allow a robotic hand to predict object slip before its occurrence. The approach only requires the robotic hand to have a short haptic surface exploration over the object surface before manipulating it. Then, the frictional properties of the finger-object contact can be efficiently identified, and the BF-ratio can be real-time predicted to predict slip occurrence under dynamic grasping conditions. Using the predicted BF-ratio as a slip, threshold is demonstrated to be more accurate than using the static/Coulomb friction coefficient. The presented approach has been experimentally evaluated on different object surfaces, showing good performance in terms of prediction accuracy, robustness, and computational efficiency.

In the wind turbine manufacturing industry, the bolt-nut joint tightening process is one of the core processes in the full production chain and concerned with assembling the hub body, the pitch system and the bearing unit. This operation is currently executed manually with the aid of different tools and gauges; the main disadvantages are a relatively high degree of variability and the necessity to repeat this task several times during a production run to achieve a satisfactory, final tightening torque within a specified angle range. Moreover, the bolt tightening process includes various uncertainties such as the presence of friction forces and the use of different bolt sizes with different stiffness values which make it highly nonlinear and uncertain resulting in a challenging control problem. To facilitate the development of an effective control strategy, we study the bolt tightening process and propose 4 tightening stages, namely, bolt-nut alignment, partial and full engagement and final bolt tightening. Based on the characteristics of each stage, a fuzzy controller is designed for each stage to realize the respective control objectives. A fuzzy error detector incorporating the knowledge of each stage is proposed for early error detection, making use of the input from a torque and encoder (angular position) sensor. Errors can be detected in each stage to interrupt the process and prevent any damage to the system.

Flexible manipulators have promising applications in minimally invasive surgery as it allows the surgical tools reach targets which are prohibited by conventional rigid surgical instrument. However one of the technical difficulties of implementing the flexible manipulator is to measure the bending curvature. This paper proposes the design of a novel optical sensor for measuring the bending curvature of a flexible manipulator based on light intensity modulation. The sensor is low cost and is temperature independent. A theoretical model of using the sensor design to deduce the curvature of a flexible robot has been created. Implementing the proposed theoretical model, the developed sensor has been used to measure the bend of a section of a flexible segment. Validation tests have been carried out; the results demonstrate that the developed sensor has good accuracy in measuring the bending angles, the orientation of the bending and the bending radius.

Robot grasping and manipulation relies mainly on two types of sensory data: vision and tactile sensing. Localisation and recognition of the object is typically done through vision alone, while tactile sensors are commonly used for grasp control. Vision performs reliably in uncluttered environments, but its performance may deteriorate when the object is occluded, which is often the case during a manipulation task, when the object is in-hand and the robot fingers stand between the camera and the object. This paper presents a method to use the robot's sense of touch to refine the knowledge of a manipulated object's pose from an initial estimate provided by vision. The objective is to find a transformation on the object's location that is coherent with the current proprioceptive and tactile sensory data. The method was tested with different object geometries and proposes applications where this method can be used to improve the overall performance of a robotic system. Experimental results show an improvement of around 70% on the estimate of the object's location when compared to using only vision.

This paper presents a novel fiber optic tactile probe designed for tissue palpation during minimally invasive surgery (MIS). The probe consists of 3×4 tactile sensing elements at 2.6mm spacing with a dimension of 12×18×8 mm3allowing its application via a 25mm surgical port. Each tactile element converts the applied pressure values into a circular image pattern. The image patterns of all the sensing elements are captured by a camera attached at the proximal end of the sensor system. Processing the intensity and the area of these circular patterns allows the computation of the applied pressure across the sensing array. Validation tests show that each sensing element of the tactile probe can measure forces from 0 to 1N with a resolution of 0.05 N. The proposed sensing concept is low cost, lightweight, sterilizable, easy to be miniaturized and compatible for magnetic resonance (MR) environments. Experiments using the developed sensor for tissue abnormality detection were conducted. Results show that the proposed tactile probe can accurately and effectively detect nodules embedded inside soft tissue, demonstrating the promising application of this probe for surgical palpation during MIS.

In this paper we present a new geometrical mapping structure that captures both the geometry and the connectivity of the environment. A robot's ability to successfully complete a required task is bound by its knowledge about the operation environment. Thus, the robot must be able to collect information from its surrounding and map it accurately to create a correct and complete representation of the environment. The solution in this paper uses the Gap-Navigation Tree as the underlying structure for the proposed Triangulation-Based exploration which maps the environment using the Dynamic Triangulation Tree structure DTT developed in this study. The efficiency of the proposed strategy is validated experimentally through simulations. The DTT does not only embed the geometry of the environment but also provides a direct mapping of the connectivity of its free space. The proposed algorithm is tested in simulations using various scenarios for exhaustive validation to prove its main advantages, namely ease of construction, compactness and completeness.

During open surgery, surgeons can perceive the locations of tumors inside soft-tissue organs using their fingers. Palpating an organ, surgeons acquire distributed pressure (tactile) information that can be interpreted as stiffness distribution across the organ -an important aid in detecting buried tumors in otherwise healthy tissue. Previous research has focused on haptic systems to feedback the tactile sensation experienced during palpation to the surgeon during minimally invasive. However, the control complexity and high cost of tactile actuators limits its current application. This paper describes a pneumatic multi-fingered haptic feedback system for robot-assisted minimally invasive surgery. It simulates soft tissue stiffness by changing the pressure of an air balloon and recreates the deformation of fingers as experienced during palpation. The pneumatic haptic feedback actuator is validated by using finite element analysis. The results prove that the interaction stress between the fingertip and the soft tissue as well as the deformation of fingertips during palpation can be recreated by using our pneumatic multi-fingered haptic feedback method.

This paper introduces a new and low-cost tissue stiffness simulation technique for surgical training and robot-assisted minimally invasive surgery (RMIS) with pseudo-haptic feedback based on tissue stiffness maps provided by rolling mechanical imaging. Superficial palpation and deep palpation pseudo-haptic simulation methods are presented. Although without expensive haptic interfaces users receive only visual feedback (pseudo-haptics) when maneuvering a cursor over the surface of a virtual soft-tissue organ by means of an input device such as a mouse, a joystick, or a touch-sensitive tablet, the alterations to the cursor behavior induced by the method creates the experience of actual interaction with a tumor in the users' minds. The proposed methods are experimentally evaluated for tissue abnormality identification. It is shown that users can recognize tumors with these two methods and the rate of correctly recognized tumors in deep palpation pseudo-haptic simulation is higher than superficial palpation simulation.

This paper proposes a new computationally fast algorithm for classifying the primitive shape and pose of the local contact area in real-time using a tactile array sensor attached on a robotic fingertip. The proposed approach abstracts the lower structural property of the tactile image by analyzing the covariance between pressure values and their locations on the sensor and identifies three orthogonal principal axes of the pressure distribution. Classifying contact shapes based on the principal axes allows the results to be invariant to the rotation of the contact shape. A naïve Bayes classifier is implemented to classify the shape and pose of the local contact shapes. Using an off-shelf low resolution tactile array sensor which comprises of 5×9 pressure elements, an overall accuracy of 97.5% has been achieved in classifying six primitive contact shapes. The proposed method is very computational efficient (total classifying time for a local contact shape = 576μs (1736 Hz)). The test results demonstrate that the proposed method is practical to be implemented on robotic hands equipped with tactile array sensors for conducting manipulation tasks where real-time classification is essential.

This paper presents a prototype of a force/torque sensor that uses fiber optic guided light and linear polarizer materials to obtain intensity modulated light to detect applied force and torque to the sensing structure. The sensor is also capable of measuring the contact direction between the sensor and the object. The sensor's design and operating principles are explained and experimental data is given to verify the proposed operating principle. The experimental data shows that linear polarizers can be used to measure the torque applied to a force/torque sensor.

This paper presents a novel approach to measuring the indentation depth of a stiffness sensor in real time during a soft tissue palpation activity. The proposed system is integrated into a stiffness probe and is designed to intra-operatively aid the surgeon to rapidly identify the tissue abnormalities with minimum measurement inaccuracies due to tissue surface profile variations. Stiffness probe and the associated surface profile sensors are pneumatic and the newly designed system can concurrently measure the indentation depth and surface profile variations while sliding over the soft tissues in any direction in a near frictionless manner. With the pneumatic pressure maintained constant, the displacement of the sensing element is a direct function of the stiffness of the tissue under investigation. The sensor has a tunable force range and the indentation force can be adjusted externally to match tissue limitations. The prototype of the new design of stiffness probe was calibrated and tested on silicone blocks simulating soft tissue. The results show that this sensor can measure indentation depth more accurately than air cushion probe alone. The structure, working principle, and a mathematical model for this new design are described.

Object surface properties are among the most important information which a robot requires in order to effectively interact with an unknown environment. This paper presents a novel haptic exploration strategy for recognizing the physical properties of unknown object surfaces using an intelligent finger. This developed intelligent finger is capable of identifying the contact location, normal and tangential force, and the vibrations generated from the contact in real time. In the proposed strategy, this finger gently slides along the surface with a short stroke while increasing and decreasing the sliding velocity. By applying a dynamic friction model to describe this contact, rich and accurate surface physical properties can be identified within this stroke. This allows different surface materials to be easily distinguished even if when they have very similar texture. Several supervised learning algorithms have been applied and compared for surface recognition based on the obtained surface properties. It has been found that the naïve Bayes classifier is superior to radial basis function network and k-NN method, achieving an overall classification accuracy of 88.5% for distinguishing twelve different surface materials.

Slip prediction is important for maintaining the stability of object handling in robust grasping and dexterous manipulation. However, up to date a challenge still remains that how to accurately predict slip occurrence before it actually happens to allow robotic hands to conduct slip compensation in time. The concept of friction cone has been conventionally used to predict slip occurrence, where the static/kinetic friction coefficient is used as a threshold. However, this threshold, i.e. the ratio of the friction and normal forces at slip occurrence (also named as break-away friction ratio), is found not constant but varies with changes in acceleration and disturbing forces applied on the grasped object, raising difficulties when attempting to accurately predict slip. In this paper, we propose a novel approach to accurately predict varying slip thresholds in real time and compensate the predicted slip during a dynamic grasping. To achieve this, first a simple but efficient haptic surface exploration using robotic fingers is carried out to identify the friction properties of an object surface. Once the friction properties are established, the slip threshold at a given grasping condition can be predicted and the grasping forces are adjusted to prevent slip. The presented approach has been evaluated, showing good performance in terms of prediction accuracy and computational efficiency.

This paper investigates the stability of the interval type-2 (IT2) fuzzy-model-based (FMB) control systems. An IT2 T-S fuzzy model is developed to represent the nonlinear plant subject to parameter uncertainties, which are captured by the lower and upper membership functions. An IT2 fuzzy model is then be proposed to close the feedback loop. It is not required that the IT2 fuzzy controller shares the same premise membership functions or the same number of fuzzy rules as those of the IT2 T-S fuzzy model. Consequently, it offers a greater design flexibility to the IT2 fuzzy controllers. However, the mismatched premise membership functions are not favourable to the development of stability conditions and thus leads to a conservative stability analysis result. In this paper, with the consideration of the lower and upper membership functions, which carry the information of the nonlinearities and parameter uncertainties of the nonlinear plant, the stability of the IT2 FMB control systems is investigated based on the Lyapunov stability theory. Stability conditions in terms of linear matrix inequalities are developed to guarantee the stability of the IT2 FMB control systems and synthesize the IT2 fuzzy controller. A simulation example is given to demonstrates the effectiveness of the proposed approach.

This paper presents a novel optical fiber-based rolling indentation probe designed to measure the stiffness distribution of a soft tissue while rolling over the tissue surface during minimally invasive surgery. By fusing the measurements along rolling paths, the probe can generalize a mechanical image to visualize the stiffness distribution within the internal tissue structure. Since tissue abnormalities are often firmer than the surrounding organ or parenchyma, a surgeon then can localize abnormalities by analyzing the image. The performance of the developed probe was validated using simulated soft tissues. Results show that the probe can measure both force and indentation depth accurately with different orientations when the probe approached and rolled on the tissue surface. In addition, experiments for tumor, identification through rolling indentation were conducted. The size and embedded depth of the tumor, as well as the stiffness ratio between the tumor and tissue, were varied during tests. Results demonstrate that the probe can effectively and accurately identify the embedded tumors.

We describe a novel approach for demonstrate of a wheel-rolling tissue deformation as well as the abnormalities tissue depth evaluation using a rolling finite element model (RFEM). Since a wheeled probe which is capable of performing rolling tissue indentation has been proven to be a promising device to rapid conduct soft tissue property identification for localization and documentation of the abnormalities within the tissue, with the aim of compensating the loss of haptic and tactile feedback experienced during robotic-assisted minimally invasive surgery (MIS). To implement such a device requires a good understanding of the dynamics of the wheel-tissue rolling interaction and relationship between the tissue internal structure and the corresponding tissue reaction force. In this paper we propose the RFEM of the dynamic interaction between a wheeled probe and a soft tissue sample using ABAQUS finite element analysis software package. The aim of this work is to more precisely locate abnormalities depth within soft tissues using RFEM and aid surgeons better in the decision of resection during MIS through the understanding of dynamics of wheel-tissue rolling interaction. The soft tissue was modelled as a nonlinear hyperelastic material with geometrical nonlinearity and the modelling parameters were calibrated using experimental data from standard tests. The purposed RFEM consists of simulations of wheel-tissue rolling indentations on a silicone phantom with varied tissue internal structure and also running on a biological tissue such as a porcine kidney. The results show that the proposed method can predicted the wheel-tissue interaction force of the rolling indentation with a good agreement results and the documentary from empirical equation of RFEM can identify the simulated tumors depth accurately.

Slip plays a vital role in traction control when a mobile robot traverses over soft soils. To estimate slip parameters, accurate measurement of robot velocity is particularly required. Previous related work done by the authors has adopted a single downward-looking single camera system for velocity and slip estimation [1][2]; however, such a single camera system is prone to lose accuracy when the distance between the camera and terrain is time-varying, such as traversing over uneven terrains [1]. To cope with the problem, this paper presents a robust downward-looking camera based velocity estimation approach, which can particularly be capable of identifying height variation and compensating for velocity estimation. A downward-looking stereo camera instead of previously used single camera is adopted. The camera-terrain distance can be estimated by matching same features in left and right frames. Robot velocity measured with height compensation can be more accurate, compared to estimates without it. The proposed approach has been validated through comprehensive experimental study on a lab-based test rig; and test results show good performance of the proposed approach. With the proposed method, slip estimation techniques given by [2][3] can be promisingly extended to non-flat terrains.

This paper presents a novel miniaturized force-indentation depth (FID) sensor designed to conduct indentation on soft tissue during minimally invasive surgery. It can intra-operatively aid the surgeon to rapidly identify the tissue abnormalities within the tissue. The FID sensor can measure the indentation depth of a semi-spherical indenter and the tissue reaction force simultaneously. It make use of with fiber optical fiber sensing method measure indentation depth and force and is small enough to fit through a standard trocar port with a diameter of 11 mm. The created FID sensor was calibrated and tested on silicone block simulating soft tissue. The results show that the sensor can measure the indentation depth accurately and also the orientation of the sensor with respect to the tissue surface whilst performing indentation.

This paper presents the prototype design and development of a miniature MR-compatible fiber optic force sensor suitable for the detection of force during MR-guided cardiac catheterization. The working principle is based on light intensity modulation where a fiber optic cable interrogates a reflective surface at a predefined distance inside a catheter shaft. When a force is applied to the tip of the catheter, a force sensitive structure varies the distance and the orientation of the reflective surface with reference to the optical fiber. The visual feedback from the MRI scanner can be used to determine whether or not the catheter tip is normal or tangential to the tissue surface. In both cases the light is modulated accordingly and the axial or lateral force can be estimated. The sensor exhibits adequate linear response, having a good working range, very good resolution and good sensitivity in both axial and lateral force directions. In addition, the use of low-cost and MR-compatible materials for its development makes the sensor safe for use inside MRI environments.

This paper describes the design and construction of a miniaturized triaxial force sensor which can be applied inside a magnetic resonance imaging (MRI) machine. The sensing principle of the sensor is based on an optical intensity modulation mechanism that utilizes bent-tip optical fibers to measure the deflection of a compliant platform when exposed to a force. By measuring the deflection of the platform using this optical approach, the magnitude and direction of three orthogonal force components (Fx, Fy, and Fz) can be determined. The sensor prototype described in this paper demonstrates that it can perform force measurements in axial and radial directions with working ranges of +/-2 N. Since the sensor is small in size and entirely made of nonmetallic materials, it is compatible with minimally invasive surgery (MIS) and safe to be deployed within magnetic resonance (MR) environments.

This paper presents a novel approach of performing artificial tactile sensing based on the deployment of prismatic-tip optical fibers. The primary principle of the sensing schemes relies on light intensity modulation for detecting the deformation of an elastic element experiencing a force load. By measuring the change of light signal intensity, the magnitude of the applied force can be determined. The force distribution over an area can be evaluated using an array of optical fibers. The tactile sensor array prototype described in this paper demonstrates its capability and feasibility in performing tactile sensing and force measurement over a range of approximately 0 to 4.8 N. Due to its simple sensing structure, it is easy to manufacture and the sensor can be miniaturized for applications in dexterous robotic handling.

This paper presents the stability analysis of fuzzy-model-based control systems. Stepwise membership functions are introduced to facilitate the stability analysis. Through the stepwise membership functions approximating those of the fuzzy model and fuzzy controller, the information of the membership functions can be brought into the stability analysis. Based on the Lyapunov stability theory, stability conditions in terms of linear matrix inequalities are derived in a simple and easy-to-understand manner to guarantee the system stability. The proposed stability analysis approach offers a nice property to include the membership functions of both fuzzy model and fuzzy controller in the LMI-based stability conditions for a dedicated fuzzy-model-based control system. Furthermore, the proposed stability analysis approach can be applied to the fuzzy-model-based control systems of which the membership functions of both fuzzy model and fuzzy controller are not necessarily the same. Greater design flexibility is allowed by choosing the membership functions during the design of fuzzy controllers. By employing membership functions with simple structure, it is possible to lower the structural complexity and the implementation cost. Simulation example is given to illustrate the merits of the proposed approach.

This paper describes a novel design of a 3-axis force sensor which can be applied in magnetic resonance (MR) workspaces such as that of a magnetic resonance imaging (MRI) machine. The sensor operates based on an optical sensing principle to measure forces deforming a 3 degree-of-freedom (DOF) flexible structure. By detecting minute deflection of such a structure using an optical sensing scheme, the magnitude and direction of an applied force can be determined. The sensor prototype described in this document demonstrates its capability of performing force measurement in both axial and radial directions with the calibrated working ranges of +/-3 N. Because all the sensor's components are entirely fabricated from non-metallic and dielectric materials, the sensor is considered suitable for applications in MR environments.

The authors have recently proposed the method of rolling indentation over soft tissue to rapidly identify soft tissue properties for localization and detection of tissue abnormalities, with the aim of compensating for the loss of haptics information experienced during robotic-assisted minimally invasive surgery (RMIS). This paper investigates the concept of rolling indentation using finite element modeling. To obtain ground truth data, rolling indentation experiments are conducted on a silicone phantom which contains three simulated tumours. The tissue phantom is modeled as hyperelastic material using ABAQUStrade. The identification of tumours includes two parts: firstly, when the spatial location of tumour is known, identify the tumour's mechanical properties (initial shear modulus); secondly if the mechanical properties of tumour are known, identify the tumour's spatial location. The results show that the proposed method can identify information of tumours accurately and robustly. The identified tumour mechanical properties and tumour locations are in good agreement with experimental measurements.

Minimally Invasive Surgery (MIS) and robot surgery have opened new ways to perform surgical operations in a safer and simultaneously faster manner. In an effort to follow this minimally invasive trend, this paper presents the feasibility study of a novel fibre-optic catheter prototype. This prototype sensor has the ability to measure forces from the sides and tip. Classification of forces from multiple positions on a catheter provides valuable information for safe navigation inside the vasculature and heart of a patient. This sensor employs two fibre-optic schemes, one for the tip and one for the sides of the catheter; it is made entirely of plastic, making it compatible with Magnetic Resonance Imaging (MRI). A test bench was used to determine the linearity coefficients during static loading. These initial experiments on the prototype gave rise to an ideal linear force response coupled with low hysteresis. Finally, an experiment which tries to simulate the human blood vessel achieved satisfying results during dynamic sensor movement.

This paper presents a novel air-cushion tactile sensor for minimally invasive surgery that is fully MRI (magnetic resonance imaging) compatible. The proposed sensor is designed to detect tissue abnormalities within soft tissue surfaces. This is achieved by rolling over soft tissue in a virtually frictionless manner due to the design of the sensor in which the sensing element, a sphere, rests on a cushion of air. This design allows for rapid acquisition of tactile and mechanical properties of large areas of soft tissue. Laboratory experiments are carried out to show its feasibility as a tactile sensor for MIS and its behaviour under loading. The outcomes of the experiments illustrate the sensor's capability and potential as a tactile sensor for MIS. These results are discussed and future work is outlined.

This paper presents a robust slip estimation method for skid-steered mobile robots when they traverse over rough terrain. An optical flow-based visual sensor looking down the terrain surface is employed to recover motion of a mobile robot by tracking features selected from the terrain surface. The motion states of the mobile robot are initially estimated by the visual sensor, however, the estimates are prone to noise and uncertainty which degrades the accuracy and robustness of estimation. To cope with the noise and uncertainty from the visual sensor, a sliding mode observer (SMO) based on the kinematics model of the skid-steered mobile robot is delicately designed to simultaneously estimate slip parameters. The SMO scheme can give more accurate estimates than the extended Kalman filter (EKF) when the slip of the mobile robot has significant changes at abrupt steering. The complete slip estimation method is independent of terrain parameters and robust in the presence of noise and uncertainty. Experimental results show that the method has confident potential for slip estimation of skid-steered mobile robots.

This paper proposes a novel approach for the identification of the internal structure and mechanical properties of biological soft tissue using a force sensitive wheeled probe to generate a 'mechanical image' by rolling across the surface of a solid organ. Initially, a testing facility for validating the concept ex-vivo was developed. Preliminary validation tests were carried out on a silicone phantom with embedded abnormalities with the aim to link the derived 'mechanical image' with the known internal structure. Ex-vivo validation tests were also conducted on excised porcine livers. The data were analyzed in four parts: 1) the dynamic analysis of wheel-tissue interaction to validate that the measured parameters are representative of underlying tissue stiffness; 2) the development of a 'mechanical image' from the rolling data; 3) a comparison of standard 1-DOF indentation testing with 2-DOF rolling and 4) the characterization of the relationship between force and tissue deflection from the data contained within the mechanical image. The results show that the 'rolling mechanical image' is capable of capturing information relating to the underlying tissue stiffness distribution and characterizing the force-tissue deflection profile for a tissue sample. Examples of scenarios, where this information could potentially be used, include providing a surgeon with the ability to probe solid organs in-vivo during robot-assisted MIS or providing prior information for the modeling of tool-tissue interactions, such as steerable needles.

This paper describes the preliminary design and construction of an optical fiber sensor which has been developed for evaluating the feasibility of using an optical-based force sensing methodology to investigate mechanical soft tissue properties during minimally invasive surgery. This sensor applies a novel reflective light intensity modulation scheme using bent-tip optical fibers and a reflector to measure mechanical response of the soft tissue when it interacts with the sensor. By adopting such optical fibers to detect minute deflection of a flexible cylindrical structure at the reflective edge of the reflector, good sensitivity for the force detection can be obtained. The prototype described in this document has demonstrated that it can detect the tissue interaction forces in the axial direction and identify variations in tissue stiffness. The maximum force range that can be detected by the sensor is 3 N. The measurement resolution is 0.02 N.

Unmanned ground vehicles are widely used in industries where repetitive tasks or high risk missions are required. Such vehicles usually operate on soft deformable terrains and still require human supervision due to the complexity of the interaction between the vehicle and the terrain. A traversability prediction simulator has recently been developed. The simulator is used to investigate the influence of vehicle design and soil properties on the maneuverability performance of tracked vehicles on soft terrains. Results of the simulations carried out give insight into the behavior of such vehicles with different operating conditions and how they can be controlled.

This paper proposes a novel technique to estimate slips and velocities of an unmanned skid-steered vehicle. An optical flow-based visual sensor looking down the terrain surface is employed to recover the motion of the vehicle by tracking features selected from the terrain surface. The special orientation of the on-board camera is to assure high accuracy of the motion estimation. To cope with the noise and uncertainty from the visual sensor, a sliding mode observer (SMO) based on the kinematic model of the skid-steered vehicle is delicately designed to simultaneously estimate the slips and velocities. The complete non-GPS slip and velocity estimation technique is independent of terrain parameters and robust to noise and uncertainty. The SMO scheme can produce more accurate estimates than the extended Kalman filter (EKF) in the nonlinear case. Experimental results are given to show that the technique has good potential for vehicle slip and velocity estimation.

This paper proposes a novel air-cushion sensor for the acquisition of tactile and force information from soft tissue, as it could be useful during robotic-assisted minimally invasive surgery in order to provide the surgeon with tactile and haptic feedback. Advancing recent work on rolling indenters [1], the sensor proposed here makes use of a rigid sphere which is held at the end of a tubular shaft and pressed against the tissue by a stream of air. Variations in tissue stiffness result in movements of the sphere within the shaft and are picked up by an optical system. The new approach allows virtually frictionless motion of the sphere across the tissuepsilas surface and rapid acquisition of tactile information over large areas of soft tissue. The structure and working principles of this new air-cushion tactile sensor are described. Laboratory experiments are conducted to show the feasibility and illustrate the behaviour of the proposed sensor system. The outcome of the conducted experiments shows the potential of the sensor system.

This paper presents the stability analysis of interval type-2 fuzzy-model-based (FMB) control systems. To investigate the system stability, an interval type-2 Takagi-Sugeno (T-S) fuzzy model, which can be regarded as a collection of a number of type-1 T-S fuzzy models, is proposed to represent the nonlinear plant subject to parameter uncertainties. With the lower and upper membership functions, the parameter uncertainties can be effectively captured. Based on the interval type-2 T-S fuzzy model, an interval type-2 fuzzy controller is proposed to close the feedback loop. To facilitate the stability analysis, the information of the footprint of uncertainty is used to develop some membership function conditions, which allow the introduction of slack matrices to handle the parameter uncertainties in the stability analysis. Stability conditions in terms of linear matrix inequalities are derived using a Lyapunov-based approach. Simulation examples are given to illustrate the effectiveness of the proposed interval type-2 FMB control approach.

This paper presents an artificial neural networks approach to estimate the indicated torque of a single- cylinder diesel engine from crank shaft angular position and velocity measurements. Since these variables can be measured using low-cost sensors, the estimator may be useful in the implementation of the control or diagnostics strategies that require cylinder indicated torque, a variables that are not easily measured and need expensive sensors. The approach is to design indicated torque estimators using feedback and an artificial neural networks model as feedforward. Such an approach can offer the advantage of being amenable to real-time implementation. The estimated results of the engine indicated torque are presented, which compared with experimental data indicate a good agreement.

This paper proposes a novel approach for the identification of tissue properties in-vivo using a force sensitive wheeled probe. The purpose of such a device is to compensate a surgeon for a portion of the loss of haptic and tactile feedback experienced during robotic-assisted minimally invasive surgery. Initially, a testing facility for validating the concept ex-vivo was developed and used to characterize two different testing modalities - static (1-DOF) tissue indentation and rolling (2-DOF) tissue indentation. As part of the static indentation experiments a mathematical model was developed to classify tissue condition based on changes in mechanical response. The purpose of the rolling indentation tests was to detect tissue abnormalities, such as tumors, which are difficult to isolate under static testing conditions. During such tests, the test-rig was capable of detecting simulated miniature buried masses at depths of 12mm. Based on these experiments a portable device capable of carrying out similar tests in-vivo was developed. The device was designed to be operated through a trocar port and its key feature is the ability to transition between static indentation and rolling indentation modalities without retracting and changing the tool.

This paper considers a tracked vehicle traversing unknown terrain, and proposes an approach based on the Generalized Newton Raphson (GNR) method for identifying all the unknown soil parameters required for tractive force prediction. For the first time, the methodology, based on measurements of track slip, i, and tractive force, F, to find unknown soil parameters is developed. The tractive force is the force generated by a tracked vehicle to drive itself forwards. This tractive force depends to a large extent on certain soil parameters, namely soil cohesion (c), soil internal friction angle (phi), and soil shear deformation modulus (K). Accurately identifying parameters of the soil on which a tracked vehicle is moving will potentially lead to accurate traversability prediction, effective traction control, and precise trajectory tracking. The soil parameter identification algorithm is validated with the experimental data from Wong [3] and from in-house track- terrain interaction test rig showing good identification accuracy and fast execution speed. It is also shown to be relatively robust to initial condition. The identified soil parameters are, in turn, used to predict the tractive forces showing good agreement with all the experimental data. The technique presented in this paper is general and can be applied to any tracked vehicle.

This paper proposes a novel nonlinear viscoelastic soft tissue model generated from ex vivo experimental results on ovine liver using a force sensitive probe. In order to study the biomechanics of soft tissue, static indentation tests were applied on ovine liver. An empirical constitutive equation was extracted from the examined data. A mechanical model combining linear viscoelasticity with a nonlinear function of strain-stress is proposed. The developed model has been evaluated both statically and dynamically with different strain rates - i.e. where the velocity of indentation is varied. By comparing simulation results and measured experimental data, it has been concluded that the proposed model is robust for modelling both static and dynamic indentation conditions. The effect of changing boundary conditions on the parameters in the proposed model has been studied by choosing test sites with different underlying tissue thicknesses. The results indicate that for small strain, the effect of the thickness condition is reasonable to be neglected.

This paper proposes a novel experimental study of the recovery of soft tissue after the removal of an external load. An optical fiber probe has been developed to measure the tissue recovery precisely through a static indentation method. Ex-vivo soft tissue recovery tests have been conducted on porcine liver. A robotic manipulator is used to control the motion of the probe and the force sensor is used to record the interaction force at the tip of the probe. An empirical mathematical model which describes the relationship of the liver recovery behavior and the holding time during which the probe keeps the tissue statically indented has been developed. The error analysis shows this model is able to predict liver recovery reasonably well. The experimental study shows that the recovery behavior of soft tissue depends on the holding time of the probe. In addition, the formula of the recovering force of the liver, which was deduced from the rate of liver recovery, is proposed.

This paper proposes a novel approach for soil estimation and classification in autonomous excavation exploiting key features from bucket velocity and energy signatures considering the interaction dynamic of the digging process. A real-time energy-based method is proposed for estimating the dynamic friction force arising during soil-tool interaction. The method relies on a novel technique for measuring the force and displacement variables which allows the on-line determination of the bucket velocity and dissipation energy along the full excavation profile. It is shown that these measurements can be effectively used for the on-line identification and classification of different types of soil encountered during the excavation process. The proposed method is insensitive to noise and can be easily implemented in practice using an excavation arm and hydraulic actuators. Various experimental results are presented supporting the practicality of the proposed method.

Closed-circuit television (CCTV) is currently used in many inspection applications, such as the inspection of nonaccessible pipe surfaces. This human-oriented approach based on offline analysis of the raw images is highly subjective and prone to error because of the exorbitant amount of data to be assessed. Laser profilers have been recently proposed to project well-defined light patterns, improving the illumination of standard CCTV systems as well as enhancing the capability of automating the assessment process. This research shows that positional (geometrical) as well as intensity information, related to potential defects, can be extracted from the acquired laser projections. While most researchers focus on the analysis of positional information obtained from the acquired profiler signals, here the intensity information contained within the reflected light is also exploited for the purpose of defect classification and visualization. This paper describes novel strategies created for the automation of defect classification in tubular structures and explores new methods to fuse intensity and positional information, achieving improved multivariable defect classification. The acquired camera/laser images are processed in order to extract signal information for the purpose of visualization and map creation for further assessment. Then, a two-stage approach based on image processing and artificial neural networks is used to classify the images. First, a binary classifier identifies defective pipe sections, and then in a second stage, the defects are classified into different types, such as holes, cracks, and protruding obstacles. Experimental results are provided. Note to Practitioners-The method presented in this paper aims to automate the inspection of nonaccessible pipe surfaces. The method was thought to be employed in the inspection of sewers; however, it could be used in many other industrial applications and could also be extended to other shapes rather than tubular structures. A laser ring profiler, consisting, for instance, of a laser diode and a ring projector, can be easily integrated into existing closed-circuit television systems. The proposed algorithm identifies defective areas and categorizes the types of defects, analyzing the successive recorded camera images that will contain the reflected ring of light. The algorithm, that can be used online, makes use of the deformation of the reflected laser ring together with its changes in intensity. The fact of combining the two kinds of data using artificial-intelligent algorithms makes the method robust enough to work in harsh environments

This paper presents the stability and performance design of a fuzzy-model-based control system subject to parameter uncertainties. A nonlinear controller with a favorable characteristic to relax the stability conditions is proposed to drive the system states of the nonlinear plant to follow those of a stable reference model. Stability and performance conditions in terms of bilinear matrix inequalities (BMIs) will be derived based on a Lyapunov-based approach. A combined genetic algorithm and convex programming technique process will be developed to solve the solution to the BMIs. An application example will be given to illustrate the merits of the proposed approach.

This paper presents a novel technique for identifying soil parameters for a wheeled vehicle travelling on an unknown terrain. The identified soil parameters are required for predicting vehicle drawbar pull and wheel drive torque which can be employed for traversability prediction, traction control, and performance optimization of a wheeled vehicle on unknown terrain. The Newton Raphson method is used as the identification technique applied on the modified form of the wheel-soil interaction dynamics model using the composite Simpson's rule. This work focuses on identifying the internal friction angle, the shear deformation modulus, and the lumped pressure-sinkage coefficient. The fourth parameter, cohesion, does not influence the vehicle drawbar pull and is assigned an average value during the identification process. In an experimental study, the identified parameters are compared with known values, and shown to be in good agreement. Soil parameter identification can be carried out on-line and thus our approach is suitable for real-time applications. The robustness of the method is also shown to be relatively good. The identified soil parameters can be used to predict drawbar pull and wheel drive torque with good accuracy

Accurate estimation of slip is essential in developing autonomous navigation strategies for mobile vehicles operating in unstructured terrain. In this paper, a sliding mode observer is firstly constructed to estimate slip parameters based on the kinematics model of a skid-steering vehicle and trajectory measurement. The stability of the sliding mode observer is given in a mathematical context. Slip estimation schemes using an extended Kalman filter and direct mathematical inversion of the kinematic equations are also presented for comparison purposes. It is shown that the non-linear sliding mode observer is more accurate than the other two methods. The robustness and superior performance of the sliding mode observer is demonstrated using both simulation and experimental results. A camera based system is used to measure the vehicle trajectory during experimental validation

Online identification of soil-bucket interaction is significant for the development of an autonomous excavation strategy. This paper presents a method for identifying multiple unknown soil parameters in real-time using the novel Modified Newton Raphson Method. The new approach involving a model selection strategy based on Parallel Execution Model, Flexible Parallel Execution Model and Hybrid Execution Model consisting of the Mohr-Coulomb soil model and the Chen and Liu Upper Bound soil model is examined. The identification of unknown soil parameters is achieved by minimizing the error between the measured forces and the modeled forces computed by the soil model. The results demonstrate that the proposed estimation scheme is accurate when comparing to the measured soil parameters. In addition, the reduced processing time of convergence and high robustness with respect to initial conditions show that the proposed method has real time estimation capability. The proposed method is very promising and highly suitable for soil-bucket interaction identification for an autonomous excavation system in an unpredictable, dynamic environment.

Standard CCTV (close circuit television) is currently used in many pipe inspection applications, such as sewers. This human-based approach is prone to error because of the exorbitant amount of data to be assessed, and smaller anomalies or defects are likely to be overlooked reducing the chance of detection of faults at an early stage. Laser profilers for pipe inspection have been recently proposed to overcome CCTV problems. Positional as well as intensity information, related to potential defects, can be extracted from the laser-camera acquired images. While most of these systems are based on the geometrical analysis of pipes, here the intensity distribution of the reflected light is also exploited. This paper describes the strategies developed for the automation of defect classification in pipes and explores new methods to fuse intensity and positional information and shows how they can be used to improve multi-variable defect classification. A neural network-based classification method is presented. Experimental results are provided.

This paper presents a novel approach for the internal inspection of sewers through the use of sonar techniques, generating enhanced 3D graphs which represent the inner sewer surfaces. These graphs not only show the inner contour of the pipe but also integrate the intensity of the received echoes. The enhanced profile is generated by superimposing the peak intensity from the returning echoes at the calculated x, y and z coordinates where they are reflected from the pipe wall. These coordinates are calculated by measuring the time of flight of the first reflections, which are extracted from consecutives B-mode images generated during the ultrasonic scanning of the pipe. The proposed method has been capable of showing anomalous conditions, inside pipes filled with liquid, with dimensions smaller than the theoretical lateral and axial resolution of the transducer, in contrast to traditional methods where these kinds of defects were not detected. The proposed inspection method and its capabilities were validated through the realization of simulations and experiments. The simulations were conducted to validate the proposed method and explore its limitations. The proposed approach was particularly developed with the aim of scanning internal sections of sewers or water pipes filled with liquid using rotary ultrasonic sonars where visual methods could not be employed. It is expected that this research could also be expanded to the inspection of other submerged structures, such as water tanks, or pressurized vessels.

The work presents the results of production trials of a gap sensing system for continuous extrusion machinery. It is critical to maintain a precise pre-defined extrusion gap for the efficient running of the continuous extrusion process and to maintain product quality. A high temperature capacitive gap sensing system is designed and implemented on a copper extrusion machine in a production plant. The results from the successful gap sensing production trials are presented and the benefits of the gap sensing system are demonstrated. First it is shown that machine setup times prior to production can be reduced from 35-40 minutes to 5 minutes with active gap sensing. The sensors can be used for on-line direct gap measurement and control, and for the first time provide a detailed view of extrusion zone gap behavior during a full production cycle. The gap sensor is used to evaluate the relationship between gap size, and waste levels. It is shown that there is a linear reduction of waste levels from 20% to 2%, when the gap size is reduced from 1 mm to 0.15 mm.

Real time estimation for soil-tool interaction is a key method for the development of an autonomous excavation strategy. This paper presents a method for identifying the unknown soil parameters in real-time using a novel hybrid soil model. The hybrid models consist of the Mohr-Coulomb soil model and the Chen and Liu Upper Bound soil model. A switching mode is utilized to select a more accurate soil model to compute the failure force depending on the position of the excavator bucket. The Newton Raphson method is proposed to identify the soil parameters by minimizing the error between the experimental forces and the forces computed by the hybrid soil model. The results demonstrate that the proposed estimation scheme is accurate when comparing to the measured soil parameters and the experimental data. In addition, the high speed estimation time shows that the proposed method has a real time estimation capability. A very high robustness is achieved when compared to the least square method. The proposed method is very promising and highly suitable for the soil-tool interaction identification of an autonomous excavator in an unpredictable, dynamical and potentially hazardous environment.

A robust, fast, and simple technique for the experimental identification of the link parameters (mass, inertia, and length) and friction coefficients of a full-scale excavator arm is presented. This new technique, based on the generalized Newton method (GNM), estimates unknown individual parameters of the excavator arm dynamic equations. The technique can be used when the number of equations is different from the number of estimated variables. Using experimental data from a full-scale field Combat Engineer Excavator (CEE), the values of link parameters and friction coefficients are successfully identified. The identified parameters are compared with known values, and shown to be in agreement. The method is compared with the least square method, and shows that the GNM is better in terms of prediction accuracy and robustness to noise. Further, the joint positions predicted by the analytical model using the identified parameters are validated against different experimental trajectories, showing very good agreement. The experimental data was obtained in collaboration with QinetiQ Ltd. (Hampshire, U.K.). The technique presented in this paper is general and can be applied to any manipulator.

Pipe inspection, for many industrial applications, is commonly carried out using CCTV (closed-circuit TV) cameras and off-line human analysis via raw image examination for defect detection. Researchers have proposed the use of additional sensors and lighting systems to create ring profiles onto pipe segments. Geometrical changes in the profile are analyzed to determine pipe deformation. However the light intensity information has not been considered in such works. In this paper we propose an intensity-based profiler system. The sensor consists of a laser profiler that is attached to a standard CCTV system. The physical sensor behavior is described using reflectance theory and experimental data. Finally, an image of the pipe wall is generated by extracting the intensity information existing in the pipe pictures. Defects and anomalies can be identified using this extracted image.

This paper presents the experimental results of an automated sensor system for the inspection of tubular structures. The method is applied to the autonomous inspection of sewers overcoming the drawbacks of standard CCTV-based inspection systems. The transducer consists of a low-cost laser-based profiler attached to a standard CCTV camera. Image analysis techniques and artificial neural networks are used to automatically locate and classify the defects in the pipe using the intensity distribution in the acquired camera images. A wide range of tests using data from different types of pipes in realistic conditions have been conducted and are presented here. It is shown that the proposed inspection approach is particularly well suited to complement existing CCTV inspection systems, providing automated and reliable detection of pipe defects in the millimeter range.

This paper presents an innovative approach for the development of ultrasonic solutions for sewer inspection, through the creation of a simulation tool capable of generating simulated images from pipes filled with water. The developed tool is capable of creating two dimensional and three dimensional surface scans of pipes simulating rotational sonar scanners submerged in water. Pipe deformations and anomalous conditions can be simulated. The properties of the simulation tool and the comparison of simulations against experimental tests, to validate its capability and accuracy to generate ultrasound images, are presented. Additionally, an optimization approach used to reduce the processing time to run the simulations, and a method to create 3D surfaces from B-mode images based on an edge detection method are described. The developed simulation routines represent a valuable tool for the development and design of pipe inspection systems and operator training.

This paper presents a novel method for estimating soil properties on-line during excavation tasks such as ground leveling, digging and sheet pilling. The proposed method computes key soil parameters by measuring the forces acting on the excavator bucket whilst being in contact with the soil and minimizing the error between measured forces and estimated forces produced by a real-time capable soil model. Two soil models, the Mohr-Coulomb soil model and the Chen and Liu upper bound soil model, are implemented and researched in the context of this estimation scheme. Parameter optimization is carried out employing the Newton Raphson method. The method is evaluated using experimental data and through comparison with an approach that makes use of graphical intersection for model optimization. The results demonstrate that the proposed Newton Raphson-based method is as accurate as the graphical intersection-based approach, but up to 2000 times faster, and thus, most suitable for on-line soil parameter estimation in an automated system which provides optimized digging trajectories for a given excavation task.

A robust and fast yet simple approach for experimental identification of the link (mass, inertia and length) parameters and friction coefficients for a full-scale closed-chain excavator arm is developed. The approach is based on a generalized Newton method, an excavation arm dynamic model and measured data. The new approach can be used where the number of equations is different from the number of variables, or if the Jacobian cannot be assumed nonsingular. The parameters are needed for improving the control actions in autonomous solution for excavation tasks and contributing to excavator design evaluation. Using experimental data obtained while moving the links of an instrumented full-scale combat engineer excavator, the values of the link parameters and friction coefficients for various links were successfully identified. The identified parameters are compared with physical values, and they are in agreement. Further, the joint torques and positions computed by the developed model using the identified parameters are validated against measured data, also showing excellent agreement. The experimental data was obtained in collaboration with QinetiQ. The novel technique presented in this paper is general and can be applied to a wide range of heavy-duty closed chain hydraulic manipulators.

Threaded fastenings are a common assembly method, and account for over a quarter of all assembly operations. These operations are very difficult to automate, with threaded to align and position the screw with respect to the hole, and to apply axial torque until the appropriate tightening forces are achieved. Screw insertions are typically carried out manually with the purpose of joining one component to another. They permit easy disassembly for maintenance, repair, and relocation. There is little research reported on automating screw insertions, with most automated assembly research focusing on the peg-in-hole assembly problem. This paper presents an on-line parameter estimation strategy employing the Newton Raphson method (NRM) for threaded fastenings. The motivation for the study is to employ the estimated parameters to develop strategies for monitoring threaded fastenings. The monitoring problem deals with predicting the integrity of self-tapping screw insertion process, based on the torque vs. insertion angle curves generated during the insertions. A technique for estimating three unknown parameters (friction coefficient, hole diameter, and screw major diameter) during a typical screw insertion is presented.

An innovative inspection method to assess the condition of sewer pipes is proposed in this paper. The standard sewer inspection technique, based on closed-circuit television systems, has a relatively poor performance; a video camera is mounted on a robot and the video recording is provided off-line to an engineer who classifies any defects. The focus of this research is the automated identification and location of discontinuities in the internal surface of sewers. The transducer used is an assembly of a CCD camera and optical elements to generate a ring-shaped laser pattern. The automated inspection method consists of several stages including the segmentation of the image into characteristic geometric features and potential defect regions. Automatic recognition, rating and classification of pipe defects are carried out by means of the computation of a partial histogram based on adaptive image processing techniques. Experiments in a realistic environment have been conducted and results are presented.

The backpropagation (BP) algorithm is commonly used in many applications, including robotics, automation and weight changes of artificial neural networks (ANNs). This paper proposes the addition of an extra term, a proportional factor (PF), to the standard BP algorithm to speed up the weight adjusting process. The proposed algorithm is tested and the results show that the proposed algorithm outperforms the conventional BP algorithm in convergence speed and the ability to escape from learning stalls. The paper presents a convergence analysis of the three-term BP algorithm. It is shown that if the learning parameters of the three-term BP algorithm satisfies certain conditions given in this paper, then it is guaranteed that the system is stable and will converge to a local minimum. The paper shows that all the local minima of the cost function are stable for the three-term backpropagation algorithm.

Standard sewer inspection systems are based on closed circuit television (CCTV) cameras mounted on wheeled platforms. One of the disadvantages of camera inspection systems is that they can detect only a small part of all possible sewer damage that could conclude in collapses. The inspection outcome of standard CCTV systems relies not only on the quality of the acquired images, but also on the off-line recognition and classification conducted by human operators. The objective of this research is the development of intelligent sensor systems that will enable the automation of the pipe condition assessment. Optical techniques are proposed to complement the existing CCTV-based approach and to improve inspection results. Besides that, automated defect recognition algorithms based on Artificial Neural Networks are proposed. Experiments to test the tolerance of the automated. algorithm to artificially-generated noise have been conducted and results are presented.

This paper presents an integrated physics based model for a front end mobile excavator. The model describes the dynamic relationship between the operator input commands and the position, orientation, speed and forces of the vehicle and the excavation arm. The dynamic model has the potential to be used in advanced controller design for automated excavation systems. The dynamic model for the excavation system is validated against measured data. The validation of the model is conducted in collaboration with QinetiQ, UK. A PID controller for trajectory tracking is implemented and tested using a computer simulation study. The graphical machine model is developed in the Zed3D graphical environment and all the inputs to the graphical model are taken from the simulation model.

The automation of screw insertions is a highly desirable task. An important part of the automation process is the monitoring of the insertion. The paper presents an application of artificial neural networks for monitoring this common manufacturing procedure. The research focuses on the insertion of self-tapping screws. A radial basis artificial neural network is employed to distinguish between successful and failed insertions. The network is tested with tasks of increasing complexity using simulation data. The approach is then validated with the use of experimental data, and the tests results are presented.

An automated parallel parking strategy for a car-like mobile robot is presented. The study considers general cases of parallel parking for a rectangular robot within a rectangular space. The system works in three phases. In scanning phase, the parking environment is detected by ultrasonic sensors mounted on the robot and a parking position and manoeuvring path is produced if the space is sufficient. Then in the positioning phase, the robot reverses to the edge of the parking space avoiding potential collisions. Finally, in manoeuvring phase, the robot moves to the parking position in the parking space in a unified pattern, which may requires backward and forward manoeuvres depending on the dimensions of the parking space. Motion characteristics of this kind of robots are modeled, taking into account the nonholonomic constraints acting on the car-like robot. A collision-free path is planned in reference to the surroundings. The strategy has been integrated into an automated parking system, and implemented in a modified B12 mobile robot, showing capable of safe parking in tight situations.

The automation of screw insertions represents a highly desirable task. An important part of the automation process is the monitoring of the insertion, The paper presents an application of artificial neural networks for monitoring this common manufacturing procedure. The research focuses on the insertion of self-tapping screws. Artificial neural networks have been employed to distinguish between successful and failed insertions. The networks under investigation use radial basis functions for the computation of the data. A range of networks, differing in size, has been implemented and thoroughly tested. Results and evaluations of the networks from the experiments are presented.

A weightless neural network based intelligent monitoring strategy for automated self-tapping screw insertions is presented. Problems encountered with automated screw insertion workstations include screw jamming, thread stripping and cross threading. If such problems are not detected early, this could lead to defective assemblies. A weightless neural network is designed and trained to monitor automated screw fastenings. The network is first trained and tested using computer simulations. An experimental test rig is constructed and the weightless neural network is tested using both seen and unseen cases. Experimental results are presented to confirm the effectiveness of the approach.

This paper describes a novel navigation and obstacle avoidance system for robotic manipulators. The system is divided into separate fuzzy units which individually control the links of a manipulator. The rule base of each unit combines the repelling influence of obstacles with the attracting influence of the target position in a fuzzy way to generate actuating commands for the link. Owing to its simplicity and hence its short response time, the fuzzy navigator is especially suitable in on-line applications with strong real-time requirements. Furthermore, this approach allows obstacle avoidance in dynamic environments. The functioning of the fuzzy navigator with respect to robotic manipulators and results of real-world experiments are presented.

It is common to use an affine transformation to approximate in dealing with the matching of plane curves under a projective transformation. The plane curve itself can be used as an identity to solve the parameters of an affine transformation. The objective of this paper is to obtain a closed form solution of the parameters using low order derivatives of the plane curve. A unique solution to the parameters of an affine transformation with up to second order derivatives is presented using differential invariants as well as the available global information. The computational time on verification has been significantly reduced. In computer vision, derivatives are obtained by numerical means. Achieving accurate numerical derivatives is an important application issue. Smoothing with a Gaussian filter modified by a linear combination of Hermite polynomials, can preserve the accuracy of continuous polynomials with powers up to the same order as the Hermite polynomials. In the discrete space however, the introduction of Hermite polynomials leads to a choice of a large smoothing scale /spl sigma/ in order to reduce computational errors at the expense of a reduction of local controllability and over-smoothing of the curve. It is shown that using a /spl sigma/ proportional to the order of the derivatives is more reliable in applications.

The invariant used as an index has shown many advantages over the pose dependent methods in model-based object recognition. Although perspective and even weak perspective invariants do not exist for general three dimensional point sets from a single view, invariants do exist for structured three dimensional point sets. However, such invariants are not easy to derive. A new special structure for calculating invariants of three dimensional objects is presented. The 3D invariant structure proposed by Rothwell (1993) requires seven points that lie on the vertices of a six-sided polyhedron and is applicable to position free objects. In comparison, the proposed algorithm requires only six points on adjacent (virtual) planes that provides two sets of four coplanar points and does not require the position free condition. Hence it is applicable to a wider class of objects. The algorithm is demonstrated on images of real scenes.

The invariant used as an index has shown many advantages over the pose dependent methods in model-based object recognition. Although perspective and even weak perspective invariants do not exist for general three dimensional point sets from a single view invariants do exist for structured three dimensional point sets. However, such invariants are not easy to derive. The 3D invariant structure proposed by Rothwell (1993) requires seven points that lie on the vertices of a six-sided polyhedron and is applicable to position free objects. A new special structure for calculating invariants of three dimensional objects is developed by the authors (1995). In comparison, the proposed algorithm requires only six points on adjacent (virtual) planes that provides two sets of four coplanar points and does not require the position free condition. Hence it is applicable to a wider class of objects This paper is the extension of previous work to discuss how to use the projection to the base plane to obtain invariant conditions for the more general situation. The algorithm is demonstrated on images of real scenes.

Finding a three dimensional shortest path is of importance in the development of automatic path planning for mobile robots and robot manipulators, and for practical implementation, the algorithms need to be efficient. Presented is a method for shortest path planning in three-dimensional space in the presence of convex polyhedra. It is based on the visibility graph approach, extended from two to three-dimensional space. A collineation is introduced for the identification of visible edges in the three-dimensional visibility graph. The principle of minimum potential energy is adopted for finding a set of sub-shortest paths via different edge sequences, and from them the global shortest path is selected. The three dimensional visibility graph is constructed in O(n/sup 3/v/sup k/) time, where n is the number of vertices of the polyhedra, k is the number of obstacles and v is the largest number of vertices on any one obstacle. The process to determine the shortest path runs recursively in polynomial time. Results of a computer simulation are given, showing the versatility and efficiency of the approach.

Presents a triangulation modelling algorithm for representing the working environment of a mobile robot such that planning a collision-free path on the corresponding constructed road map (a graph) is simplified. Both the nodes and the edges of the graph can be exactly calculated by expressions F/sub n/(V,B) and F/sub B/(V,B), where V and B represent the total numbers of vertices and obstacles respectively, and both F/sub n/(V,B) and F/sub e/(V,B) are of complexity O(V). The solution path planned on the resulting graph keeps the robot at some clearance from the obstacles.

Presented is the forward kinematic solution for the most general case of the 4-6 Stewart platform mechanism, in particular, the spherical joints of both the top and the base platforms are not restricted to be in a single plane. The problem is reduced to a 32nd order polynomial equation in a single unknown. This new theoretical analysis is numerically verified.

Two reduced, unconstrained robot models, in which the constraints are satisfied automatically, are introduced. The force tracking error is dependent on both the position tracking error and the estimated parameter error, so that in the constrained adaptive robot manipulator control, the convergence of the estimated parameter error becomes more important than in the unconstrained adaptive robot control. However, in the direct adaptive controller, the parameter adaptation is only driven by the tracking error in the joint motion, while in indirect adaptive controller, the parameter estimation is only driven by the prediction error in the filtered joint torque. Based on this observation, a combined adaptive controller for constrained robot manipulators, with uncertain dynamic model parameters, is proposed. The combined adaptive control law, which is driven by both the tracking error and the prediction error, gives much improved stability properties for parameter estimation and force tracking.

Abmcf- Presented is a novel motion strategy for a mobile, car like robot that is subject to kinematic constraints. The algorithm operates on the original obstacles, without needing to generate the configuration space obstacles for the dimensioned robot. The path for the dimensioned robot is generated in three stages: (i) the shortest path problem for a point robot is solved; (ii) free space relative to the point robot shortest path is locally evaluated by minimum distance computations; (iii) the point robot shortest path is locally modified to account for the size and kinematic constraints of the mobile robot. If the shortest point robot path fails to be modified into a feasible path for the robot, the process is repeated with a second candidate point robot path, and SO on until a feasible path is generated. Thus the proposed strategy combines a global scheme for point robot path generation with a local scheme for free space evaluation and point robot path modification. The algorithm is computationally efficient, being of computational time O(nk +nlogn) where n is the total number of vertices, including the two ends, and k is the number of obstacles. The algorithm has been tested in computer simulations, demonstrating its ability to automatically generate paths which may include reversals.

A computationally eftKcient assembly sequence planning algorithm for a multi-robot cell Is presented. A compact diagrammatic representation of the problem accounting for constraints which avoid collisions between the robots is introduced. The algorithm is based on dynamic programming. The computational complexity of the algorithm is O(logn/k) for m robots to assemble n elements in k groups.