Virtual reality (VR) simulation can improve the outcomes of percutaneous nephrolithotomy (PNCL) - a surgery to extract kidney stones using ultrasound (US) or fluoroscopy image guidance. These simulators almost exclusively employ fluoroscopy, and no commercial VR simulator is available for US-guided PNCL (usPCNL). In this paper, we proposed the first step towards developing an usPCNL simulator that integrates a volumetric US model of the patient's anatomy derived from parallel 2D computed tomography (CT) scans. A critical challenge in US image generation from CT scans is that the limited spatial resolution of CT slices may lead to inaccuracies in the simulated US images. The proposed algorithm interpolates successive CT images to create an augmented dataset with increased spatial resolution. Each CT slice is then converted into a US image based on principles of linear acoustics and spatial impulse response. These images are then combined to form two different volumetric US images, one derived from the original sparse CT scans, and one created with the augmented data. From these volumetric US images, new images can be formed along arbitrary imaging planes not captured in the original CT data. The obtained simulated images are compared with their corresponding real US images acquired experimentally, and further evaluated quantitatively using normalized root mean square error (NRMSE) and dice similarity coefficient (DSC). The results reveal an NRMSE of$\mathbf{0.235}\pm \mathbf{0.051}$and a DSC of$\mathbf{0.9139}\pm \mathbf{0.062}$, showcasing a close resemblance between simulated and actual ultrasound images. Additionally, we show that denser CT scan data leads to a 25% improvement in image quality based on peak signal-to-noise ratio compared to the original dataset. This initial work is laying the foundation for the development of the usPCNL simulator, which could potentially have significant benefits for training and enhancing skills in this medical procedure.

Two-dimensional (2D) simultaneous localization and mapping (SLAM) using a LIDAR is a method used to track the position and orientation of a moving platform. 2D-SLAM assumes that the platform translates in a 2D plane and can only rotate about an axis perpendicular to that plane. However, the assumption of no out-of-plane (OOP) motion does not hold true for platforms experiencing motion in six degrees-of-freedom (6-DOF), such as wearable technologies that have no 3D LIDAR. This paper proposes a new algorithm, called the Level Plane for SLAM (LPS) for removing OOP motion from 2D-LIDAR scans generated on platforms experiencing 6-DOF without requiring scan-matching in 3D. Like other existing methods, an IMU is combined with a 2D-LIDAR to determine the platform's orientation, capture OOP motion, and generate a scan in$3\mathrm{D}$. Unlike other methods, OOP motion is removed by projecting scans onto a globally stabilized coordinate frame in$2\mathbf{D}$where both scan matching and map alignment take place. The proposed algorithm is validated over a series of experiments with different levels of induced and observed OOP motion. Experimental results show that LPS is able to handle more OOP motion than other algorithms and run in real-time.

Ultrasound (US)-guided percutaneous nephrolithotomy is a surgical procedure for large kidney stone removal through an incision in the patient's back. To gain kidney access, the surgeon steers a needle towards the kidney while simultaneously controlling the position and orientation of a US probe to keep the needle in the image plane. To successfully reach the kidney while avoiding delicate structures, a significant level of skill and precision is required. To alleviate the surgeon's cognitive workload, robot-assisted needle tracking can be implemented to autonomously track the needle in the US images and adjust the US probe's position and orientation such that the same portion of the needle is visible in the images. This paper presents a US-guided visual servoing (VS) algorithm to track the translation and rotation of a needle in a plane. Image features representing the desired pose of the needle in the image are defined, through which an interaction matrix is devised to relate the rate of the change of the image features in US images to the required position and orientation of the US probe connected to a robotic manipulator. Experimental results in 4 experimental scenarios in a water tank demonstrate the capability of the proposed method in tracking the needle in real-time with an accuracy of 2.6 mm with a control rate of 20 Hz. Although VS has been used to track surgical targets in the past, this paper proposes the first implementation of VS for needle tracking in longitudinal US images subjected to 3-DOF motion in a plane without any prior knowledge of needle trajectory or additional position sensors.

Acoustoelectric impedance tomography (AET) is a new non-invasive medical imaging procedure used to map the electrical properties of biological tissues with higher spatial resolution than traditional electrical impedance tomography (EIT). It exploits the acoustoelectric effect where modulated ultrasonic pressure changes the local tissue conductivity. This provides additional information to reconstruct a tomographic image, and has a stabilizing effect on an otherwise highly unstable inverse problem.In this paper, a novel approach to solving the AET inverse problem for image reconstruction is proposed. In the algorithm, the acoustoelectric effect is assumed to create small perturbations in the local resistance of the medium under observation. A lumped model consisting of a finite mesh of resistors approximates the medium under observation, through which boundary voltage differences between the excited and unexcited medium are calculated. A variation of the Modified Newton Raphson (MNR) algorithm is then proposed, where each pattern in the algorithm is created from small perturbations of the tissue conductivity. A total of eight simulation scenarios are evaluated, where the conductivity perturbations are in the order of 1%, 2.5% to 5% of the nominal tissue conductivity. The algorithm can successfully reconstruct the images in the presence of random noise. The obtained images are compared against traditional EIT where the percentage error is calculated for each simulated tomographic image. The simulation results indicate that the proposed approach is superior to traditional EIT as it constructs more distinct and high contrasting images with less percentage error.

An active ankle foot orthosis (AAFO) is an assistive device that applies plantarflexion and dorsiflexion assistance to the ankle joint by means of a compliant actuator. The device must apply sufficient torque assistance to track the desired ankle trajectory. However, torque disturbances are prevalent throughout the gait cycle. Accurately modelling the AAFO in conjunction with the ankle joint disturbance torque is a difficult task, as the model parameters can change over time. As a result, parameters such as inertia and friction are often roughly estimated based on the user’s weight. The uncertainties due to unmodelled disturbances and errors in dynamics modelling can severely compromise the device’s ability to provide appropriate assistance.This paper presents a novel extended parameter estimation observer combined with a disturbance rejection controller to estimate the model’s inertia and friction. First, an extended state observer (ESO) is employed in which the extended state is the estimated disturbance. Knowing the nominal ankle torque trajectory and the disturbance, a novel control law is formulated to reject the effects of endogenous disturbance torque during trajectory tracking. Then, based on the observed difference between the observed disturbance and nominal ankle torque, the paper introduces a novel method to estimate the inertial and friction parameters of the AAFO.Simulation results show that state feedback with the ESO is able to reduce the root mean square tracking error by 5.2% and 71.1% for high and low feedback gains, respectively. The results also indicate that the estimated AAFO and ankle joint inertial and damping parameters converge close to the nominal plant parameters. Simulations also show the effectiveness of the estimation laws from various initial plant estimates.

In this paper, a novel optimization method named reference point-based particle sub-swarm optimization (RPB-PSWO) is presented. RPB-PSWO utilizes the particle position update method of PSO and with the non-dominance and diversity selection methods of NSGA-II. The multi-objective optimizer utilizes a reference point-based system to allocate particles into an equidistant sub-swarm, in which particles are attracted to a pareto optimal solution in that sub-swarm. To encourage diversity and avoid local minima, density and turbulence factors are included. RPB-PSWO is capable of optimizing problems with many dependent variables, as the position update method of PSO inherently preserves dependent relationships, but suffers from an increased computation cost compared to NSGA-II. The proposed algorithm, although less computationally efficient, is capable of creating diverse pareto front solutions for standardized and custom optimization problems.

One of the many challenges in autonomous robots is that they can enter an error state and are unable to continue operation without human intervention. Sensors in-stalled on the robot enable proprioception and could help the robot understand its error configuration. This paper proposes a method to determine from these sensor measurements, which are most critical in differentiating the error states such that the robot could understand its predicament, and could attempt at recovering without human aid. A classification model is built using the forward stepwise method and a scoring metric to overcome indecision in choosing between different features. This modified method is applied to three robot operating mode data sets. The experiments indicate an improvement to the classifier performance when using this the model built by the method compared to using all available predictor variables (features). With further refinement, this scoring metric could be a simple yet effective way to build classification models for increasing robot autonomy.

Percutaneous nephrolithotomy is the leading treatment for large or irregularly shaped kidney stones. Nevertheless, gaining access to the kidney remains a challenging component of the procedure with a steep learning curve. As a result, the procedure would benefit from robotic assistance to partially or fully automate this critical component of the intervention. A key component of automated kidney access, using a robotic manipulator, is to define and follow a tool path planning based on preoperative imaging and a target entry point.In this paper, the use of the multiobjective non-dominated sorting genetic algorithm II (NSGA-II) is proposed to plan a B-spline curve that will be used as the tool trajectory during the procedure. Here, NSGA-II is used to determine the anchor point locations for a uniform 3rd order B-spline curve. The optimal path minimizes path length, tissue potential energy due to tissue compression, and path smoothness while maximizing the distance to obstacles. The multiobjective optimization is evaluated using simulations and physical trials. The results show that the planned trajectories show minimal tissue deformation, are relatively short and smooth and do not collide with the internal kidney structures.

Series elastic actuators with passive compliance have been gaining increasing popularity in force-controlled robotic manipulators. One of the reasons is the actuator's ability to infer the applied torque by measuring the deflection of the elastic element as opposed to directly with dedicated torque sensors. Proper deflection control is pinnacle to achieve a desired output torque and, therefore, small deviances in positional measurements or a nonlinear deformation can have adverse effects on performance. In applications with larger torque requirements, the actuators typically use gear reductions which inherently result in mechanical backlash. This combined with the nonlinear behaviour of the elastic element and unmodelled dynamics, can severely compromise force fidelity.This paper proposes a backlash compensating active disturbance rejection controller (ADRC) for multi-input series elastic actuators. In addition to proper deflection control, a multiinput active disturbance rejection controller is derived and implemented experimentally to mitigate any unmodelled nonlinearities or perturbations to the plant model. The controller is experimentally validated on a hybrid motor-brake-clutch series elastic actuator and the controller performance is compared against traditional error-based controllers. It is shown that the backlash compensated ADRC outperforms classical PID and ADRC methods and is a viable solution to positional measurement error in elastic actuators.

Tissue classification from electrical impedance spectroscopy has several applications in diagnosis, surgical planning, and minimally invasive surgery. The method involves applying an alternating current to the sample and measuring its electric impedance at various frequencies. The spectrum is fit to a equivalent electric circuit that mimics the shape of the tissue's impedance spectrum. The model parameters are then used for classification. This paper proposes a new solution to decompose the model fitting problem into a form suitable for multi-objective optimisation, from which all the non-dominated solutions are used to form the database of parameters for a given tissue, as opposed to a single solution that is typically seen in impedance spectroscopy. The solution explores the use of the reference point dominance condition within Non-dominated Sorting Genetic Algorithm II to fit the data to the double dispersion Cole model. Each non-dominated solution contain values for the dispersion model elements. The multiple parameter value solutions from the optimiser are used as features in a weighted Naïve Bayes classifier to identify a new tissue sample. Experiments results in 3 different tissue samples shows that the method is successful in correctly labelling the data with an average accuracy of 89%.

Electrical impedance tomography (EIT) is a non-invasive medical imaging procedure. Image reconstruction in EIT is difficult because it involves solving a non-linear and ill-posed mathematical problem. One of the most commonly implemented inverse approaches is usually a variation of the Newton Raphson algorithm. However, this approach is not guaranteed to reach a global optimum or a local optimum and as such, it requires an accurate initial estimation of the resistance distribution, which is not always available in practice. In this paper, a new method is proposed to solve for the inverse problem in EIT while avoiding dependencies on the initial estimation of the resistance distribution. The proposed approach uses a differential evolution (DE) optimizer integrated with the Newton Raphson algorithm. The stochastic nature of DE allows the problem to be solved without having an accurate initial estimation and allows for solutions that will not be trapped in local minimal values. Simulation results indicate that the proposed approach outperforms the traditional differential evolution algorithm, and performs similarly to the traditional Modified Newton Raphson algorithm with accurate initial estimation. The proposed method does, however, have an advantage over the Modified Newton Raphson algorithm as it does not require an estimate of the initial resistance distribution.

Series elastic actuators have proven to be an elegant response to the issue of safety around human-robot interaction. The compliant nature of series elastic actuators provides the potential to be applied in robot-aided rehabilitation for patients with upper and lower limb musculoskeletal injuries. This paper proposes a new series elastic actuator to be used in robot-aided musculoskeletal rehabilitation. The actuator is composed of a DC motor, a torsion spring, and a magnetic particle brake coupled to one common output shaft through a differential gear. The proposed topology focuses on three types of actuation modes most commonly used in rehabilitation, i.e., free motion, elastic, and assistive/resistive motion. A dynamic model of the actuator is presented and validated experimentally and the ability of the actuator to follow a reference torque is shown in different experimental scenarios.

Stability and transparency are key design requirements in haptic devices. Transparency can be significantly improved by replacing conventional electric motors with passive actuators such as brakes or dampers. Passive actuators can display a wide range of impedance and since they can only dis-sipate energy, stability is guaranteed. However, passive haptic devices suffer from a serious drawback; the direction of the force output is difficult to control. This issue was addressed extensively for planar manipulators but devices with higher degrees-of-freedom (DOF) have not been examined. In this paper, we introduce a new analytical framework to evaluate the feasibility and performance of non-redundant passive haptic manipulators with any DOF. The method identifies different regions in the workspace where a force can be created or approximated, and regions where a passive system cannot create force at all for a given user input. The results indicate that the range of forces a passive device can display increases with the number of DOF. This framework can aid in the design of control methods for multi-DOF passive haptic devices.

Robotic-assisted needle steering can enhance the accuracy of needle-based interventions. Application of current needle steering techniques are restricted by the limited deflection curvature of needles. Here, a novel steerable needle with improved curvature is developed and used with an online motion planner to steer the needle along curved paths inside tissue. The needle is developed by carving series of small notches on the shaft of a standard needle. The notches decrease the needle flexural stiffness, allowing the needle to follow tightly curved paths with small radius of curvature. In this paper, first, a finite element model of the notched needle deflection in tissue is presented. Next, the model is used to estimate the optimal location for the notches on needle's shaft for achieving a desired curvature. Finally, an ultrasound-guided motion planner for needle steering inside tissue is developed and used to demonstrate the capability of the notched needle in achieving high curvature and maneuvering around obstacles in tissue. We simulated a clinical scenario in brachytherapy, where the target is obstructed by the pubic bone and cannot be reached using regular needles. Experimental results show that the target can be reached using the notched needle with a mean accuracy of 1.2 mm. Thus, the proposed needle enables future research on needle steering toward deeper or more difficult-to-reach targets.

Telerobotic manipulation allows patients living with upper limb impairments to interact with a variety of environments and accomplish through teleoperation daily activities such as playing, feeding, self-care, and leisure, that would otherwise be difficult to perform. In this paper, we propose a nonlinear mapping between the patient's range of motion and the workspace of an environment being manipulated. The objective is to identify the patient's workspace and span it to that of the environment or an object, thus optimizing the scaling factor while soliciting the entire patient's range of motion. The boundaries of each workspace are obtained from scattered measurements of the master and slave robots end-effector position. The nonlinear mapping is then achieved through thin plate spline interpolation that describes deformation between two surfaces by scattered point-to-point preponderances. Experimental results reported in three different scenarios confirm the suitability of the nonlinear transformation to map diverse workspace volumes.

In this paper, we propose an automated method to reconstruct the three-dimensional (3-D) needle shape during needle insertion procedures using only 2-D transverse ultrasound (US) images. Using a set of transverse US images, image processing and random sample consensus are used to locate the needle within each image and estimate the needle shape. The method is validated with an in vitro needle insertion setup and a transparent tissue phantom, where two orthogonal cameras are used to capture the true 3-D needle shape for verification. Results showed that the use of at least three images obtained at 75% of the maximum insertion depth or greater allows for maximum needle shape estimation errors of less than 2 mm. In addition, the needle shape can be calculated consistently as long as the needle can be identified in 30% of the transverse US images obtained. Application to permanent prostate brachytherapy is also presented, where the estimated needle shape is compared to manual segmentation and sagittal US images. Our method is intended to help to assess needle placement during manual or robot-assisted needle insertion procedures after the needle has been inserted.

We propose a method to estimate the entire shape of a long flexible needle, suitable for a needle insertion assistant robot. This method bases its prediction on only a small segment of a needle, imaged via ultrasound, after insertion. An algorithm is developed that can segment a needle observed partially in ultrasound images and fully in camera images, returning a polynomial representation of the needle shape after RANSAC processing. The polynomial corresponding to the partial needle observation in ultrasound images is used as the input to a needle-tissue interaction model that predicts the entire needle shape. The needle shape predicted by the model is compared to the segmented needle shape based on camera images to validate the proposed approach. The results show that the entire needle shape can be accurately predicted in tissues of varying stiffness based on observation of parts of the needle in an ultrasound image.

In needle-based medical procedures, beveled-tip flexible needles are steered inside soft tissue with the aim of reaching pre-defined target locations. The efficiency of needle-based interventions depends on accurate control of the needle tip. This paper presents a comprehensive mechanics-based model for simulation of planar needle insertion in soft tissue. The proposed model for needle deflection is based on beam theory, works in real-time, and accepts the insertion velocity as an input that can later be used as a control command for needle steering. The model takes into account the effects of tissue deformation, needle-tissue friction, tissue cutting force, and needle bevel angle on needle deflection. Using a robot that inserts a flexible needle into a phantom tissue, various experiments are conducted to separately identify different subsets of the model parameters. The validity of the proposed model is verified by comparing the simulation results to the empirical data. The results demonstrate the accuracy of the proposed model in predicting the needle tip deflection for different insertion velocities.

A tissue-independent model to estimate needle deflection during insertion in soft tissue is presented in this paper. A force/torque sensor is connected to the needle base in order to measure forces and moments during insertion due to needle deflection. A static mechanical model, which is based on the Euler-Bernoulli beam equation and the balance of forces applied by the tissue onto the needle takes these force and moment measurements as input. The needle tip deflection can then be calculated based on the beam model undergoing these forces. Three different needle-tissue interaction models are presented. Their estimation performance is evaluated and experimentally compared by carrying out insertion experiments into phantom tissue. The experimental results show a precise estimate of needle tip deflection for a novel virtual sensor introduced in this work. The main advantage of this virtual sensor approach is that measurements obtained from the force/torque sensor are the only necessary model inputs. Furthermore, the approach does not rely on ultrasound or other image-based needle observation techniques. This makes the virtual sensor suitable for real-time feedback of needle tip deflection.

This paper introduces an automatic method to visualize 3D needle shapes for reliable assessment of needle placement during needle insertion procedures. Based on partial observations of the needle within a small sample of 2D transverse ultrasound images, the 3D shape of the entire needle is reconstructed. An intensity thresholding technique is used to identify points representing possible needle locations within each 2D ultrasound image. Then, a Random Sample and Consensus (RANSAC) algorithm is used to filter out false positives and fit the remaining points to a polynomial model. To test this method, a set of 21 transverse ultrasound images of a brachytherapy needle embedded within a transparent tissue phantom are obtained and used to reconstruct the needle shape. Results are validated using camera images which capture the true needle shape. For this experimental data, obtaining at least three images from an insertion depth of 50 mm or greater allows the entire needle shape to be calculated with an average error of 0.5 mm with respect to the measured needle curve obtained from the camera image. Future work and application to robotics is also discussed.

This paper represents an extension to the kinematic bicycle model for beveled-tip needle motion in soft tissue, which accounts for non-constant curvature paths for the needle tip. For a tissue that is not stiff relative to the needle, the tissue deformation caused by needle insertion deviates the needle tip position from a constant curvature path. The proposed model is obtained by replacing the bicycle wheels with omnidirectional wheels that move in two orthogonal directions independently. Such wheels can move sideways, providing a means for modeling the deviations of the needle tip from a constant curvature path by incorporating new parameters in the model. Using an experimental setup, the needle is inserted into soft phantom tissue at different constant velocities and model parameters are fitted to experimental data. The model is verified by comparing the results from the model to empirical data.

This paper presents a stable control method for a hybrid haptic device comprising a brake and a motor. A review of stability condition via describing function analysis is first presented. The results show that while brakes are intrinsically stable, an active device is limited in terms of stiffness. The stability is however improved if the brake simulates a physical damping. Subsequently, the stability condition is obtained via passivity condition analysis. The results demonstrate that the stiffness is improved by engaging both actuators to create resistive forces and the passivity is respected assuming a passive virtual environment. An energy and a stiffness-bounding algorithms have been developed in order to assure the stability of the coupled system in this case. It has been tested and validated using a 1-DOF hybrid haptic device by the simulation of an unstable and an active virtual environments respectively . Experimental results show that the displayable stiffness is improved under stability conditions using the control method. Furthermore, it allows the hybrid system to simulate nonlinear and unstable virtual environments and the controller remains independent of the virtual environment model.

A new concept of hybrid actuator for haptic devices is proposed. This system combines a controllable magnetorheological brake with a conventional DC motor. Both actuators are linked through an overrunning clutch. Thus, the motor is connected to the handle while the brake can exert a resistive force only in a defined direction. This configuration enables the brake and the motor to be engaged at the same time because the torque imposed by the motor is not canceled by the brake. The concept and its control laws have been investigated using a 1-DOF haptic device. The experimental results show that is possible to combine a powerful brake with a small DC motor. This approach reduces the power consumption, expand the range of forces, achieve global stability in the system providing thereby safety to the user. Besides, the proposed independent control laws enable the actuator to be adaptable in many different haptic applications.